Control Improvement for Jump-Diffusion Processes with Applications to Finance Nicole Bäuerle joint work with Ulrich Rieder Toronto, June 2010
Outline Motivation: MDPs Controlled Jump-Diffusion Processes Control Improvement Algorithm Financial Applications
Motivation: MDPs Markov Decision Processes Let (X n ) be a controlled Markov process with state space S, action space A, transition kernel Q( x, a). Let f : S A be a decision rule and β (0, 1) a discount factor, r(x, a) a bounded reward function. Consider the infinite-horizon Markov Decision Problem [ ] J(x) := sup J f (x) = sup IE x β n r(x n, f (X n )). f F f F n=0
Motivation: MDPs Notation IB := {v : S R : v < }. For v IB and f : S A let T f v(x) := r ( x, f (x) ) + β v(x )Q ( dx x, f (x) ). f is called maximizer of v if T f v = sup T f v. f F It holds that J f = T f J f and J = sup f T f J.
Motivation: MDPs Howard s Policy Improvement Algorithm 1. Choose f 0 arbitrary and set k = 0. 2. Compute J fk as solution v IB of the equation v = T fk v. 3. Compute f k+1 as a maximizer of J fk. Then J fk+1 J fk. If f k+1 = f k then J fk = J and (f k, f k,...) is optimal. Else set k := k + 1 and go to step 2.
Controlled Jump-Diffusion Processes Controlled Jump-Diffusion Processes W = (W 1,..., W m ) is an m-dimensional Brownian motion, N = (N 1,..., N l ) are indep. Poisson random measures, ν j (B) := IE N j (1, B) are the Lévy measures, Ñ j (dt, dz j ) := N j (dt, dz j ) ν j (dz j )dt. The n-dimensional controlled state process X = (X 1,..., X n ) is m dx i (t) = µ i (t, X t, π t )dt + σ ij (t, X t, π t )dw j (t) + j=1 j=1 l + γ ij (t, X t, π t, z j )Ñj(dt, dz j )
Controlled Jump-Diffusion Processes Controlled Jump-Diffusion Processes π = (π t ) is a càdlàg control process with values in D R d, the coefficient functions µ, σ, γ are continuous, g, h are reward functions. Consider the problem [ ] T J π (t, x) := IE t,x g(s, X s, π s )ds + h(x T ). t J(t, x) = sup J π (t, x). π
Controlled Jump-Diffusion Processes Generator of the state process Av(t, x, u) = v t (t, x) + + 1 2 + n v xi (t, x)µ i (t, x, u) + i=1 n (σσ T ) ij (t, x, u)v xi x j (t, x) + i,j=1 l ( v ( t, x + γ (j) (t, x, u, z j ) ) v(t, x) j=1 ) x v(t, x)γ (j) (t, x, u, z j ) ν j (dz j ).
Controlled Jump-Diffusion Processes Control Improvement Algorithm 1. Suppose π 0 is an admissible control. 2. Compute the corresponding value function J 0 and suppose J 0 C 1,2. 3. Compute π 1 (t, x) such that it maximizes u g(t, x, u) + AJ 0 (t, x, u), u D and suppose that πt 1 := π 1 (t, Xt 1 ) is an admissible control.
Controlled Jump-Diffusion Processes Control Improvement Algorithm Under some technical conditions it holds: Theorem Let I := {(t, x) : g(t, x, π 1 (t, x)) + AJ 0 (t, x, π 1 (t, x)) > 0}. a) If I, then J 1 (t, x) J 0 (t, x) for all (t, x) and J 1 (t, x) > J 0 (t, x) for (t, x) I. b) If I = then π 1 is an optimal control.
Controlled Jump-Diffusion Processes Limit Considerations Theorem Suppose that the following assumptions are satisfied: (i) lim k J k =: J C 1,2 and J k t J t, J k x J x, J k xx J xx uniformly. (ii) µ, σ, γ are bounded. Let π be a policy defined by the maximizer of J as in step (b) of the algorithm, then J = J and π is optimal.
Application: Portfolio Optimization Financial Market The price process (St 0 ) of the riskless bond is given by S 0 t := e rt, where r 0 denotes the fixed continuous interest rate. The price process (S t ) of the risky asset satisfies: ds t = S t ( µdt + σdwt + 1 where µ R, σ > 0 and 1 zν(dz) <. Øksendal and Sulem (2005) zñ(dt, dz))
Application: Portfolio Optimization Portfolio Optimization U : (0, ) R is a (strictly increasing, concave) utility function. (π t ) with π t [0, 1] is the portfolio strategy where π t = fraction of wealth invested in the stock at time t. The dynamics of the wealth process is dx π t = X π t (rdt + π t (µ r)dt + π t σdw t + π t ). zñ(dt, dz) The portfolio problem is J(t, x) := sup IE[U(X T π ) X t π = x]. π 1
Application: Portfolio Optimization When is the invest all the money in the bond -strategy optimal? Theorem Let U C 2 (0, ) be an arbitrary utility function. The invest all the money in the bond -strategy is optimal if and only if µ r.
Application: Portfolio Optimization Proof Consider π t 0 with J π (t, x) = U(xe r(t t) ). π 0 is again a maximum point of u AJ π (t, x, u) on [0, 1] if and only if u AJπ (t, x, u) u=0 = (µ r)xj π x 0.
Application: Portfolio Optimization Special Case: Black-Scholes Model Suppose now we have a Black-Scholes market. In case µ > r, the first improvement of the invest all the money in the bond -strategy is given by U (xe r(t t) ) (µ r) π 1 (t, x) = U (xe r(t t) )xer(t t) σ 2. It relies on the Arrow-Pratt-Relative-Risk-Aversion Coefficient and the Merton-ratio. When the utility function is the power or logarithmic utility function, the first improvement yields already the optimal investment strategy.
Application: Portfolio Optimization When is a constant fraction optimal? Suppose ν is concentrated on (0, ), i.e. jumps are only upwards and that 2 xν(dx) < µ r. Under these assumptions it holds: Theorem The logarithm- and the power-utility are the only utility functions U C 2 (0, ) with U C 2 (up to a multiplicative constant) where the optimal portfolio invests a constant positive fraction of the wealth in the stock.
Application: Portfolio Optimization Proof J π and π t π are optimal if and only if π is a maximum point of u AJ π (t, x, u), u 0, i.e. (µ r)j π x + J π xxσ 2 xπ + and we must have AJ π (t, x, π) = 0, i.e. 0 0 ( ) Jx π (t, x + πxz)z Jx π (t, x) ν(dz) = 0 Jt π + (r + (µ r)π)xjx π + 1 2 Jπ xxσ 2 x 2 π 2 + ( ) + J π (t, x + πxz) J π (t, x) Jx π (t, x)πxz ν(dz) = 0.
References Bäuerle, N., Rieder, U. (2010) : Control improvement for jump-diffusion processes with applications to finance. Preprint. Bäuerle, N., Rieder, U. (2010) : Markov Decision Processes with Applications to Finance. To appear. Fleming, W. H., Rishel, R. (1975) : Deterministic and stochastic optimal control. Springer-Verlag. Howard, R. (1960) : Dynamic programming and Markov processes. The Technology Press of M.I.T., Cambridge, Mass. Øksendal, B,,Sulem, A. (2005) : Applied stochastic control of jump diffusions. Springer-Verlag Thank you very much for your attention!