Portfolio Optimization with Position Constraints: an Approximate Dynamic Programming Approach

Transcription

1 Portfolo Optmzaton wth Poston Constrants: an Approxmate Dynamc Programmng Approach Martn B. Haugh Leond Kogan Sloan School of Management, MIT, Massachusetts, MA 02142, Zhen Wu Department of IE and OR, Columba Unversty, New York, NY 10027, Ths research for ths paper was completed whle the frst author was at the Department of Industral Engneerng and Operatons Research at Columba Unversty. He can be contacted at

2 Abstract We analyze dynamc portfolo choce problems usng an approxmate dynamc programmng (ADP) algorthm. We extend the algorthm to the case of constrants on borrowng and mplement a dualty-based smulaton procedure for estmatng bounds on the true value functon. We demonstrate that the ADP soluton exhbts a hgh degree of accuracy n the consdered examples, ndcatng that ths s a promsng approach to tacklng challengng practcal problems n the area of asset allocaton and portfolo choce. We present addtonal evdence on the performance of the dualty-based method for estmatng performance of approxmate portfolo rules, showng that t provdes a valuable tool n conjuncton wth ADP-style algorthms. Subject Classfcatons: Fnance: portfolo optmzaton. optmal control, dualty theory. Dynamc Programmng:

3 1 Introducton Recent years have seen ncreased nterest n the problem of optmal portfolo choce. Ths nterest has been fueled partly by methodologcal advances, and partly by the growng practcal mportance of such problems, partcularly snce the ncreased emphass on defned contrbuton penson plans has recently shfted the burden of lfe-tme asset allocaton onto ndvduals. A sgnfcant effort n academc lterature has been drected towards obtanng explct solutons to selected problems (e.g., Merton 1971, 1990, Karatzas, Lehoczky, Seth and Shreve, 1986, Km and Omberg, 1986, Cox and Huang, 1989, Lu, 1998, Wachter, 2002) and analyzng calbrated versons of smple models (e.g., Brennan, Schwartz and Lagnado, 1997, Munk, 2000, Campbell and Vcera, 1999, 2002, Brennan and Xa, 2002, Barbers, 2002, Campbell, Chan and Vcera, 2003, Chacko and Vcera, 2005, Wachter and Sangvnatsos, 2005), avodng the challenge posed by lack of analytcal tractablty and hgh dmensonalty of many problems of practcal nterest. Recently, Brandt, Goyal, Santa-Clara and Stroud (2005) suggested a computatonal algorthm, based on approxmate dynamc programmng (ADP) deas, amed at hgh-dmensonal and computatonally ntensve problems. Ther algorthm reles on functonal approxmatons to the value functon and apples to problems wth ncomplete fnancal markets. An mportant unexplored aspect of ther algorthm s the qualty of approxmaton. Whle the procedure can be shown to perform well on a few sample problems wth known solutons, t s clearly not a guarantee that t wll far as well on more challengng problems for whch solutons cannot be obtaned usng standard methods. In ths paper, we show how ADP approach can be strengthened by a rgorous dualtybased smulaton method for evaluatng the qualty of approxmate solutons, developed by Haugh, Kogan, and Wang (2006) (HKW). Ther smulaton method can handle problems wth poston constrants, ncludng ncomplete markets. For the examples we consder, the ADP algorthm delvers accurate approxmatons to the optmal polcy, as verfed by our smulaton technque. Second, our analyss also demonstrates the practcal potental of the dualty-based method of HKW. The latter demonstrated that ther smulaton method shows promse n several examples based on a smple myopc portfolo polcy. In the problems consdered, the myopc soluton was close to the optmum, leavng open the queston of how well the algorthm would perform on problems wth a sgnfcant hedgng component n the optmal polcy. In ths paper, we show that the smulaton method of HKW also performs well on problems for whch myopc strateges are far from optmal. 1

4 2 The Model We consder a portfolo choce problem under ncomplete markets and portfolo constrants. We formulate our model n dscrete tme: the economy exsts between 0 and T, and tradng takes place at equally spaced perods t [0, T ], = (0, 1,..., I), t 0 = 0, t I = T. In later sectons, when we compute dualty-based performance bounds, we defne the dscrete-tme model as an Euler approxmaton to a contnuous-tme dffuson model. The nvestment opportunty set There are N rsky assets and a rskfree asset. Wthout loss of generalty, we assume that the assets pay no dvdends. The nstantaneous moments of asset returns depend on the M-dmensonal vector of state varables X t. We assume that the vector of state varables X t follows a general Markov process. The vector of rsky returns s assumed to be condtonally log-normally dstrbuted,.e., return on the rsky asset n over the tme nterval [t, t +1 ) s gven by R (n) = exp(r (n) ), where the vector r = (r (1), r (2),..., r (N) ) has a condtonal mult-varate normal dstrbuton, gven the state X at tme t. Thus, n order to specfy the return process, we only need to defne the vector of condtonal means and the varance-covarance matrx of r. We denote the return on the rskfree asset over the same tme nterval by = exp(r (f) ). The objectve functon We assume that the portfolo polcy s chosen to maxmze the expected utlty of wealth at the termnal date T, E 0 [U(W T )]. We further assume that the utlty functon s of constant relatve rsk averson (CRRA) type, so that U (W ) = W 1 γ /(1 γ). In addton to beng a very popular specfcaton of preferences, the CRRA utlty functon has an advantage that optmal portfolo polces are ndependent of wealth, whch makes the problem more tractable computatonally. We let θ denote the vector of portfolo shares nvested n the rsky assets, so that the return on the portfolo s gven by where ı = (1,..., 1). + θ (R ı), In summary, the portfolo choce problem s to solve for [ ] 1 sup E 0 {θ } 1 γ W 1 γ T subject to the budget constrant ( ) W +1 = W + θ (R ı). (1) 2

5 3 Approxmate Dynamc Programmng for Portfolo Optmzaton We now descrbe our method for constructng approxmate solutons to the portfolo choce problems. We use the ADP algorthm for ncomplete markets, developed by Brandt et al. (2005). We also extend ther method to handle no-borrowng constrants. Brandt et al (2005) propose solvng the portfolo choce problem by approxmatng the value functon V (X, W ). The value functon satsfes the Bellman equaton [ ( ( ( )))] V (X, W ) = max E V +1 X +1, W + θ R ı. (2) θ wth the termnal condton V T (X T, W T ) = W 1 γ T /(1 γ). Snce n most cases the dynamc program (2) cannot be solved exactly we nstead look for an approxmate soluton. Followng Brandt et al (2005), we use a 4 th -order Taylor expanson of V +1 around W n the rght-hand-sde of (2) and obtan [ V (X, W ) max E V +1 (X +1, W ) + 2 V +1 (X +1, W )W R θ θ V +1 (X +1, W )(W R θ ) V +1 (X +1, W )(W R θ ) V +1 (X +1, W )(W R θ ) 4 ] where R ( ) = R 1 s the vector of excess returns on the rsky assets and 2 n V denotes the n-th order partal dervatve of the value functon wth respect to ts second argument, wealth. If we use ˆθ to denote the optmal weghts n (3), then we can characterze the partal dervatves of the value functon as V +1 (X +1, W ) = (W 1 γ ) 1 γ E +1 [H +1 ] 2 V +1 (X +1, W ) = (W ) γ E +1 [H +1 ] 2V 2 +1 (X +1, W ) = γ(w ) γ 1 E +1 [H +1 ] 2V 3 +1 (X +1, W ) = γ(γ + 1)(W ) γ 2 E +1 [H +1 ] 2V 4 +1 (X +1, W ) = γ(γ + 1)(γ + 2)(W ) γ 3 E +1 [H +1 ] (3) where H +1 = I 1 k=+1 ( k + R k ˆθ k ) 1 γ. 3

6 We can then rewrte (3) to obtan [ 1 ( ) 1 γe+1 ( ) γ V (X, W ) = W 1 γ max E [H +1 ] + E+1 [H +1 ] θ 1 γ R θ 1 ( ) γ 1 2 γ E+1 [H +1 ]( R θ ) ( ) γ 2 6 γ(γ + 1) E+1 [H +1 ]( R θ ) 3 1 ( ) ] γ 3 24 γ(γ + 1)(γ + 2) E+1 [H +1 ]( R θ ) 4. (4) The optmal soluton to (4) s clearly ndependent of W. condtons correspondng to (4) are gven by [ ( ) γ E E+1 [H +1 ] R ( γ ( + 1γ(γ + 1) γ(γ + 1)(γ + 2) ( ) γ 1 E+1 [H +1 ] R ˆθ R ) γ 2 E+1 [H +1 ]( R ˆθ ) 2 R ) ] γ 3 E+1 [H +1 ]( R ˆθ ) 3 R The frst-order optmalty mplyng n partcular that { 1 ˆθ = (E [B +1 ]) 1 γ E [A +1 ] (1 + γ)e [C +1 (ˆθ )] 1 } 6 (1 + γ)(2 + γ)e [D +1 (ˆθ )] = 0 where A +1 = B +1 = C +1 (θ ) = D +1 (θ ) = ( ( ( ( (5) ) γ E+1 [H +1 ] R (6) ) γ 1 E+1 [H +1 ] R R (7) ) γ 2 E+1 [H +1 ]( R θ ) 2 R (8) ) γ 3 E+1 [H +1 ]( R θ ) 3 R. (9) We dscuss how functons A +1, B +1, C +1 and D +1 can be estmated n Secton 3.1 below. For now, assume that these quanttes can be estmated wth suffcent accuracy. Then, we can solve (5) by a smple teratve procedure. In partcular, f we start wth a good ntal approxmaton to ˆθ, θ 0, then we can substtute θ 0 nto the rght-hand-sde of (5) and obtan a new estmate, θ 1. We terate n ths manner untl the sequence of estmates, θ k, converges. There are many possble ways to fnd accurate ntal approxmatons, θ 0, for the teraton. Brandt et al. (2005) suggest usng the soluton, to the ADP algorthm that s 4

7 based on usng a 2 nd order Taylor expanson of the value functon n (3). It s easly seen that θ 0 s then gven explctly as θ 0 = (E [B +1 ]) 1 { 1 γ E [A +1 ]}. (10) Alternatvely, one can use smple suboptmal portfolo strateges, such as the statc and myopc strateges of Secton 5.2, as startng ponts for teraton. There s no theoretcal guarantee that the teratve procedure above converges. sometmes, one may need to use a hgher order approxmaton to acheve convergence. For example, the correspondng teraton for the 5 th -order approxmaton s gven by θ k+1 = (E [B +1 ]) 1 { 1 γ E [A +1 ] (1 + γ)e [C +1 (θ k )] 1 6 (1 + γ)(2 + γ)e [D +1 (θ k )] ( where F +1 (θ k ) = ) γ 4 E+1 [H +1 ]( R θ k ) 4 R (1 + γ)(2 + γ)(3 + γ)e [F +1 (θ k )]}(11) The procedure descrbed above s suggested n Brandt et al. (2005). In the followng sectons, we generalze ther method to handle portfolo constrants. We ntroduce our technque usng several examples wth common portfolo constrants. However, n order to motvate our procedure for handlng constrants, we frst dscuss the method for estmatng condtonal expectatons n (6 9). 3.1 Numercal Implementaton: Estmatng Condtonal Expectatons In order to fnd the optmal ˆθ va the teratve procedures of (5) or (15) t s necessary to estmate the quanttes E [A +1 ], E [B +1 ], E [C +1 ] and E [D +1 ] where A +1, B +1, C +1 and D +1 are gven by equatons (6), (7), (8) and (9), respectvely. Indeed snce C +1 and D +1 are actually functons of θ t s necessary to re-estmate them n each teraton of (5) and (15). The key to a useful computatonal algorthm s to be able to estmate requred condtonal expectatons effcently. A nave approach of usng a full-scale wthn smulaton Monte Carlo smulaton loop to estmate each condtonal expectaton s too costly computatonally and would make mult-perod problems extremely dffcult to solve. Besdes, n order to estmate condtonal expectatons, one would have to have nformaton about the optmal polcy along each of the paths n the nternal smulaton loop, whch s a non-trval obstacle. To get around the need to conduct smulatons wthn smulatons, Brandt et al. (2005) suggest a clever across-path regresson procedure, based on the approaches of Longstaff and Schwartz (2001) and Tstskls and Van Roy (2001) for prcng Amercan optons. We now summarze ths approach and then dscuss how t can be extended to constraned problems. 5

8 The Across-Path Regresson Approach The algorthm s based upon smulatng S sample paths of the underlyng state varables from t = 0 to t = T. We now descrbe how to estmate h(x ) = E [G], where G s a random varable, dependng on the future values of state varables. The S sample paths provde us wth S realzatons of the par, (X, G). Usng (X s, G s ) for s = 1,..., S to denote these realzatons, we may now consder a lnear regresson of the form G = β f(x ) + ɛ, (12) where f(x ) s a matrx contanng S rows, each of whch s a vector of values of bass functons and ɛ s an orthogonal error term. Each bass functon should be a functon of tme nformaton, X. We choose a complete set of bass functons, whch guarantees an arbtrarly accurate approxmaton wth a suffcent number of bass functons and a large enough number of smulated paths. For example, usng polynomal bass functons, and truncatng our expanson at the second order, we can take a constant, lnear functons, and quadratc functons to form our bass, n whch case an n-th row of f(x ) s gven by [ ( ) ] f n (X ) = 1 (X n ) X n X n Hgher-order terms of the form (X n X n... X n ) can be added to mprove approxmaton accuracy. Of course, to avod over-fttng, the number of bass functons should reman suffcently small compared to the number of smulated paths, S. After estmatng the regresson (12), we use the ftted values ˆβ to approxmate the condtonal expectaton, h(x ): h(x ) ĥ(x ) = ˆβ f(x ) 3.2 No-borrowng Constrant The across-path regresson approach of the prevous Secton s an effectve tool for handlng ncomplete-markets problems, for whch explct expressons (5) are avalable. However, a straghtforward applcaton of such an approach to problems wth portfolo constrants runs nto sgnfcant computatonal obstacles: as soon as constrants on the portfolo composton, θ, are mposed, one has to solve a constraned optmzaton problem for every smulaton path at every tradng perod. Ths precludes one from estmatng the optmal portfolo polcy usng an effcent regresson approach. In ths paper we do not attempt to provde a general treatment of portfolo constrants. Instead we consder the specal case of no-borrowng constrants and solve the problem usng an ntutve applcaton of Lagrangan relaxaton. In partcular, we relax constrants usng Lagrange multplers and obtan explct expressons for the optmal 6

9 polcy n terms of such multplers. We then perform teratons, smlar to the case of ncomplete markets, updatng the multplers at each step. We do t n a manner that guarantees feasblty of our approxmate soluton. The resultng teratve procedure s well suted for mplementaton usng the across-path regressons. We can then apply the dualty-based performance evaluaton algorthm to the constraned portfolo choce problem. When the no-borrowng constrant s mposed, the optmzaton problem correspondng to the 4 th -order approxmaton to the value functon takes form [ 1 ( ) 1 γ ( ) γ V (X, W ) = max E ı E+1 [H +1 ] + E+1 [H +1 ] θ 1 1 γ R θ 1 ( ) γ 1 2 γ E+1 [H +1 ]( R θ ) ( ) γ 2 6 γ(γ + 1) E+1 [H +1 ]( R θ ) γ(γ + 1)(γ + 2) ( ) ] γ 3 E+1 [H +1 ]( R θ ) 4 (13) Let α 0 denote a Lagrange multpler on the portfolo constrant n (13). We relax the no-borrowng constrant to obtan a problem of the form ( ) 1 γ ( ) γ V (X, W ) = max E E +1 [H +1 ] + E+1 [H +1 ] θ 1 γ R θ 1 2 γ ( ) γ 1 E+1 [H +1 ]( R θ ) ( ) γ 2 6 γ(γ + 1) E+1 [H +1 ]( R θ ) 3 1 ( ) ] γ 3 24 γ(γ + 1)(γ + 2) E+1 [H +1 ]( R θ ) 4 + α(ı θ 1) (14) Note that the effect of the term α(ı θ 1) n (14) s to penalze the objectve functon when the no-borrowng constrant s volated. The frst-order condtons for the optmzaton problem n (14) are gven by [ ( ) γ E E+1 [H +1 ] R ( ) γ 1 γ E+1 [H +1 ] R ˆθ R + 1 ( ) γ 2 2 γ(γ + 1) E+1 [H +1 ]( R ˆθ ) 2 R 1 ( ) ] γ 3 6 γ(γ + 1)(γ + 2) E+1 [H +1 ]( R ˆθ ) 3 R + αı = 0. As was the case for the ncomplete markets problem wthout no-borrowng constrants, we can rewrte these condtons n the form ˆθ nb = ˆθ nc + α γ (E [B +1 ]) 1 ı (15) ˆθ nc where refers to the expresson for ˆθ n the ncomplete market wthout any constrants. Startng wth a feasble ntal approxmaton to the portfolo polcy, θ 0, we 7

10 then employ the same teratve procedure as n Secton 3 to compute the optmal ˆθ. We mantan feasblty of the sequence θ 0, θ 1,..., θ k by varyng α wth each teraton. In partcular, t s easy to see that we can mantan feasblty of θ k f we let α k be the multpler for the k-th teraton and choose t to satsfy γ(1 ı θ nc,k ) α k =, f ı (E [B +1 ]) 1 ı ı θ nc,k 1, (16) 0, otherwse. The multpler α defned by (16) remans non-postve after each teraton, snce the matrx B +1 s postve-defnte. It s easy to check that our choce of the multpler n (16) s such that, f the k-th teraton θ nb,k 1 were already optmal, then we would also recover the optmal soluton as θ nb,k, and our teratve procedure would ndcate convergence. Ths s because, by constructon, the soluton θ nb,k s feasble for the orgnal problem, satsfes the frstorder optmalty condtons for the relaxed problem, satsfes complmentary slackness condtons together wth α k, and the multpler α k s non-postve. Thus, to guarantee that θ nb,k s optmal, t suffces that E [B +1 ] s the same as under the optmal portfolo polcy, whch s the case f θ nb,k 1 s optmal. 4 The Algorthm Estmatng Dervatves of Condtonal Expectatons In order to compute the upper bound on the true value functon usng our approxmaton, Ṽ, t s necessary to compute partal dervatves of Ṽ. One of the advantages of usng a power utlty functon s that we only need to compute Ṽ/ x and do not need hgher order dervatves. It s easy to see that fndng Ṽ/ x amounts to fndng E [H ]/ x. Whle dfferentaton s an nherently unstable procedure, we obtaned satsfactory results by usng the dervatve of our approxmaton to E [H ] as our approxmaton to E [H ]/ x. Ths observaton may be explaned n part by the fact that n our numercal examples we found the magntude of the θ component contrbutng to the market-prce-of-rsk to be sgnfcantly larger than the partal dervatve component. 4.1 The ADP Algorthm 1. Smulate m paths of the state varables orgnatng from X Startng from t n = T, work backwards computng (and storng: see the Secton 8

11 below on constructng the polcy) approxmately optmal ˆθ s. We do ths at tme t for each of the m paths by: (a) We choose a feasble startng pont of θ 0 for each sample path. Set k = 0. (b) We estmate E [A +1 ], E [B +1 ], E [C +1 (θ k )] and E [D +1 (θ k )] for each sample path usng the across-path regresson approach. Then usng (5) (or (15) n the no-borrowng case), we obtan θ k+1 for each sample path. (c) If max 1 j m θ j,k+1 θ j,k s suffcently small, we termnate at ths tme step. Otherwse we contnue teratng usng equaton (5), or equaton (15) n the no-borrowng case. Constructng an Approxmate Portfolo Polcy The ADP algorthm generates an approxmate optmal polcy along each sample path. In order to evaluate the overall qualty of the approxmate polcy, as well as for practcal applcatons of the algorthm, one must be able to extrapolate the approxmate tradng polcy to arbtrary ponts n the state space. Clearly, there are many ways to perform such an extrapolaton. We use a smple method, based on an dea of local averagng. In partcular, we generate n j equal ntervals along the j th dmenson of the state space. Together wth I equal ntervals, ths creates a total of In 1 n 2 n M (M + 1)-dmensonal rectangles, whch cover the parts of the state space most lkely to be vsted by the state vector X t. Then, f a pont (t, X t ) falls nsde one of the rectangles, we defne the correspondng extrapolated value of the portfolo polcy as an arthmetc average over the values at all the ponts used by the ADP algorthm that happen to fall nsde the same rectangle. If a pont (t, X t ) happens to be outsde of the pre-defned rectangles, or f we cannot fnd a sngle path used n the ADP algorthm wth a pont nsde the rectangle correspondng to (t, X t ), then we smply assgn the value of the approxmate portfolo polcy based on a smple analytcal approxmaton (e.g., a myopc approxmaton, see below). In our numercal experments, we found that we end up usng such an analytcal approxmaton nfrequently, typcally less than 2% of the tme. 5 Numercal Experments We perform several numercal experments, n whch we llustrate the performance of the ADP algorthm. Based on the results n Haugh, Kogan and Wang (2006), we know that the myopc portfolo polcy often provdes a good approxmaton to the optmal polcy. The man promse of the ADP algorthm s that t can be used to obtan hgh-qualty approxmate solutons n stuatons where the optmal polcy s far from myopc,.e., the hedgng component of the optmal polcy s sgnfcant. Wth that n mnd, we desgn some of our experments so that, a pror, the myopc component of the optmal polcy 9

12 s small relatve to the hedgng component. As we shall see below, the ADP algorthm captures the hedgng demand qute well. In order to further evaluate the performance of the ADP algorthm n our experments, we compare the resultng lower and upper bounds on the value functon wth those correspondng to smple analytcal approxmaton, the myopc polcy defned below. In the numercal experments n Haugh, Kogan and Wang (2006), the bounds based on the myopc polcy are relatvely tght. Ths ndcates that for the portfolo choce problems consdered there the hedgng demand can be safely gnored, resultng n lttle loss of utlty. Below we consder settngs n whch capturng the hedgng demand s crucal, and the ADP algorthm s able to accomplsh that wth sgnfcant accuracy. 5.1 The Data-Generatng Process Recall that there are N rsky securtes and one short-term rsk-free bond (a bank account) wth M state varable processes drvng ther short term returns. We vary the numbers of assets and state varables n our experments, but the general structure of the data-generatng process s always the same. We defne all the processes n contnuous tme, and then generate the correspondng dscrete-tme dynamcs usng Euler approxmatons. Euler approxmatons add a certan amount of numercal error to the estmates of the upper bound on the value functon of the dscrete-tme problem, snce the theoretcal results n Haugh, Kogan and Wang (2006) apply to contnuous-tme models. However, by shrnkng the tme between tradng perods, we ensure that numercal approxmaton errors are qute small. The reported results where obtaned by applyng the ADP algorthm wth a tme-step equal to 1/15, and usng a tme-step of 1/100 n out-of-sample smulatons to estmate the bounds on the true value functon. In contnuous tme, the vector of state varables X t s characterzed by a lnear stochastc dfferental equaton, drven by a J-dmensonal vector of ndependent standard Brownan motons z t : dx t = K X t dt + σ X dz t, (17) where X t s an M by 1 vector, K s an M by J matrx, and σ X s an M by J matrx. We assume that the nstantaneously rsk-free rate s stochastc and gven by r t = δ 0 + δ 1 X t, where δ 0 and δ 1 are constant. In order to defne return processes for the rsky assets, we frst ntroduce ther market prce of rsk: Λ t = λ 1 + λ 2 X t, (18) where λ 1 s a constant N by 1 vector and λ 2 s a constant N by M matrx. Then, returns on the rsky assets satsfy dp t P t = (r t + σ Λ t ) dt + σ dz t, (19) 10

13 where σ s the 1 by N dffuson vector for asset. We allow σ to be a determnstc functon of tme. Our defnton of rsky assets s qute general and can be used to descrbe returns on both stocks and bonds. In fact, n some of our numercal experments, we assume that the portfolo conssts of both stocks and bonds and use ths to calbrate our model. Calbraton We now consder two specal cases of our general data-generatng process. Our frst model corresponds to the market wth two assets: a rsk-free short-term bond (a bank account) and a rsky long-term bond. We defne parameter values so that the myopc component of the optmal polcy s dentcally equal to zero. We then evaluate the ablty of the ADP algorthm to recover the hedgng component, whch completely characterzes the optmal portfolo polcy. Model I: There are three state varables and one rsky asset n our frst model, therefore M = 3 and N = 1. The rsky asset s a long-term bond maturng at tme T. Gven that the rsk-free rate s an affne functon of the state vector X t, t follows that the market prce of rsk n (18) s also an affne functon of the state vector X t. The rskneutral process for the rsk-free rate s therefore Gaussan, and hence the term structure of nterest rates s affne (e.g., Duffe 2001, Chapter 7). Specfcally, the dffuson matrx of the long-term bond, σ 1, s an explct functon of tme, gven by where e τ Q denotes matrx exponentaton and τ = T t. σ 1 = b (1 e τ M ), b = δ 1 Q 1, (20) Q = K + σ X λ 2, To calbrate the model, we adopt parameter values for the state-varable process from Wachter and Sangvnatsos (2005). We devate n one respect: we set the market prce of rsk to zero, to guarantee that the myopc component of demand s dentcally equal to zero. Our parameter values are summarzed n Table 1. Model II: Our second model s a more realstc example, n whch we construct an optmal portfolo of the rsk-free short-term bond, two bonds of three- and ten-year maturty, and a stock ndex. The three- and ten-year bonds are n fact dynamc rollover strateges, accordng to whch one must constantly renvest the funds to mantan the duraton of the bonds at three and ten years respectvely. Specfcally, an nvestment at tme t nto a three-year bond maturng at t + 3 must be lqudated at tme t + t and renvested n another bond, maturng at t + t + 3. Both stock and bond returns are predctable by a vector of state varables. The vector of state varables s now fourdmensonal. Its frst three components are the same as n our frst model and are taken from Wachter and Sangvnatsos (2005). The fourth component represents the logarthm 11

14 of the dvdend yeld on the NYSE value-weghted portfolo, whch we use to predct returns on the stock market, the frst rsky asset n our model. We assume a Gaussan model for stock returns and, furthermore, we assume that the shocks drvng changes n the dvdend yeld and stock returns are ndependent of the shocks to the frst three components of the state vector and are not prced. Ths assumpton s justfed mostly by ts convenence: snce we are not strvng to have an emprcally accurate model of the jont behavor of stocks and bonds, the smplfyng assumpton allows us to use parameter estmates of Wachter and Sangvnatsos (2005) to descrbe the behavor of bonds n our model. The dffuson coeffcents of bond returns can be computed accordng to (20). The dffuson coeffcents of stock returns, denoted by σ S, are gve n Table 2, together wth other model parameters. 5.2 Numercal Results We frst defne formally the myopc portfolo polcy, whch we use to ntate the teratve soluton procedure descrbed n Secton 4.1. We also compare the performance of the portfolo strategy generated by the ADP algorthm, captured by the resultng bounds on the maxmal expected utlty, to the bounds correspondng to the myopc strategy. Let Σ t denote the dffuson matrx of returns on the rsky assets. Let µ t denote the vector of nstantaneous expected returns. Gven our lnear specfcaton of the market prces of rsk, µ t = µ 1 + µ 2 X t. When no-borrowng constrants are mposed, we lmt feasble portfolos to the set K = {ı θ t 1}. Then, the myopc portfolo polcy s defned as θ myopc t = arg max θ t K θ t (µ t r t ) 1 2 θ t Σ t Σ t θ t. (21) The results for Model I are summarzed n Table 3. We choose a rather hgh value of rsk averson, γ = 15, to make sure that the myopc polcy s far from the optmum. As we know from pror lterature (e.g., Wachter, 2003), long-horzon nvestors wth hgh rsk averson tend to gravtate towards holdng a bond wth maturty matchng ther nvestment horzon. Thus, for comparson, we nclude the results for a strategy of nvestng 100% of the portfolo nto the long-term bond. As we see, the ADP algorthm produces a relatvely tght set of bounds on the optmal expected utlty. In contrast, the myopc polcy performs rather poorly, as expected: due to the lack of rsk premum, the myopc polcy prescrbes holdng 100% of the portfolo n the short-term rsk-free nstrument, whch s pretty much the opposte of what a long-horzon hghly rsk-averse nvestor would lke to do. Another aspect of the results n Table 3 s that the dualtybased method for estmatng upper bounds on the value functon performs well when the optmal soluton s far from the myopc approxmaton. Ths provdes further evdence of the method s potental. The results for Model II are summarzed n Table 4. Our calbraton mples a sgnfcant degree of predctablty n stock returns, therefore the certanty-equvalent returns 12

15 achevable by the constructed portfolos are qute hgh. We see a clear pattern across the dfferent rsk averson values: the ADP polcy outperforms the myopc polcy n terms of the acheved expected utlty, whch s expressed as a lower bound on the optmal value. The dfference s not large, due to the fact that expected returns on the stock ndex n our model are qute volatle and capturng ths predctablty through a myopc polcy seems to have a frst-order affect on expected utlty. Moreover, the varable predctng stock returns s not well hedged by any of the rsky assets, dmnshng the mportance of the hedgng component of demand. It s nterestng that the upper bound obtaned from the ADP polcy s not always superor to the one based on the myopc polcy. Ths must be due to the fact that to evaluate the upper bound one must approxmate the partal dervatves of the value functon wth respect to the state varables. If such approxmaton turns out to be naccurate, one can obtan tghter bounds by usng the myopc strategy and smply settng the above partal dervatves to zero. 6 Concluson As our results ndcate, the ADP algorthm s a vable tool for solvng practcally nterestng portfolo choce problems. The method s accurate n the consdered examples, whether the myopc polcy s close to optmalty or not. As our numercal experments demonstrate, the accuracy of the ADP-based approxmate solutons can be relably gauged by the dualty-based smulaton method of HKW. Together, the two algorthms form a powerful set of tools whch can be used to tackle dffcult, hgh-dmensonal problems. 13

16 Barbers, N Investng for the Long Run When Returns are Predctable. Journal of Fnance Brandt, M., A. Goyal, P. Santa-Clara, and R. Stroud A Smulaton Approach to Dynamc Portfolo Choce wth an Applcaton to Learnng About Return Predctablty. Revew of Fnancal Studes 18, Brennan, M. and Y. Xa Dynamc Asset Allocaton Under Inflaton. Journal of Fnance Brennan, M., E. Schwartz, and R. Lagnado Strategc Asset Allocaton. Journal of Economc Dynamcs and Control Campbell, J. and L. Vcera Consumpton and Portfolo Decsons When Expected Returns are Tme Varyng. The Quarterly Journal of Economcs Campbell, J. and L. Vcera Strategc Asset Allocaton Portfolo Choce for Long-Term Investors. Oxford Unversty Press, USA. Campbell, J., Y. Chan and L. Vcera A Multvarate Model of Strategc Asset Allocaton. Journal of Fnancal Economcs Chacko, G. and L. Vcera Dynamc Consumpton and Portfolo Choce wth Stochastc Volatlty n Incomplete Markets. Revew of Fnancal Studes Cox, J. and C.F. Huang Optmal Consumpton and Portfolo Polces When Asset Prces Follow a Dffuson Process. Journal of Economc Theory Duffe, D USA. Dynamc Asset Prcng Theory. 3rd ed. Prnceton Unversty Press, Haugh, M., L. Kogan and J. Wang Portfolo Evaluaton: A Dualty Approach. Forthcomng, Operatons Research. Karatzas, I., J. Lehoczky, S. Seth and S. Shreve Explct Soluton of a General Consumpton/Investment Problem. Mathematcs of Operatonas Research Km, T. and E. Omberg Dynamc Nonmyopc Portfolo Behavor. Revew of Fnancal Studes Merton, R Optmum Consumpton and Portfolo Rules n a Contnuous-Tme Model, Journal of Economc Theory 3, Merton, R Contnuous-Tme Fnance. New York: Basl Blackwell. 14

17 Munk, C Optmal Consumpton/Investment Polces wth Undversfable Income Rsk and Lqudty Constrants. Journal of Economc Dynamcs and Control Lu, J Dynamc Portfolo Choce and Rsk Averson. Workng paper. Stanford Unversty, Palo Alto. Longstaff, F. and E. Schwartz Valung Amercan Optons by Smulaton: A Smple Least-Squares Approach. Revew of Fnancal Studes Tstskls, J., and B. Van Roy Regresson Methods for Prcng Complex Amercan Style Optons. IEEE Transactons on Neural Networks Wachter, J Portfolo and Consumpton Decsons under Mean-Revertng Returns: An Exact Soluton for Complete Markets. Journal of Fnancal and Quanttatve Analyss Wachter, J Rsk Averson and Allocaton to Long-Term Bonds. Economc Theory Journal of Wachter, J. and A. Sangvnatsos Does the Falure of the Expectatons Hypothess Matter for Long-Term Investors?, Journal of Fnance

18 Parameter Values K σ X δ δ λ λ Table 1: Model I, calbrated parameters. The market conssts of the rsk-free asset and a long-term bond. The data-generatng process s Gaussan. See Secton 5.1 for detals. 16

19 Parameter Values K σ X σ S δ δ λ λ Table 2: Model II, calbrated parameters. The market conssts of the rsk-free asset, two long-term bonds, and a stock ndex. Stock returns are predctable and the process for the predctve varable s ndependent of the processes drvng the term structure of nterest rates. The data-generatng process s Gaussan. See Secton 5.1 for detals. 17

20 Incomplete Markets LB ADP 5.22 [5.22,5.23] UB ADP 5.53 [5.53,5.54] LB m 4.42 [4.41,4.42] UB m 5.59 [5.58,5.60] LB LT 5.51 [5.51,5.51] UB LT 5.53 [5.52,5.53] No-borrowng LB ADP 5.29 [5.28,5.29] UB ADP 5.67 [5.65,5.69] LB m 4.42 [4.41,4.42] UB m 6.87 [6.80,6.94] LB LT 5.51 [5.51,5.51] UB LT 5.58 [5.57,5.59] Table 3: Model I, portfolo performance. The parameter set s defned n Table 1 and the problem horzon s T = 5 years. The rsk averson coeffcent s γ = 15. All results are computed for the ntal value of the state varable X 0 = 0. The rows marked LB ADP and UB ADP report the estmates of the bounds on the expected utlty acheved by usng the ADP portfolo strategy. The ADP algorthm uses fourth-order approxmatons and reles on a myopc polcy for ntal approxmaton and extrapolaton. The rows marked LB m and UB m report the correspondng results based on the myopc portfolo strategy. Expected utlty s reported as a contnuously compounded certanty equvalent return. Approxmate 95% confdence ntervals are reported n brackets. The rows marked LB LT and UB LT report the bounds based on the polcy of holdng all of the portfolo n a long-term bond maturng at tme T. We report the results both for ncomplete markets and the case of no-borrowng constrants. 18

21 γ = 1.5 γ = 3 γ = 5 Incomplete Markets LB ADP [56.78,56.91] [34.66,34.81] [24.18,24.34] UB ADP [60.31,60.56] [37.45,38.28] [27.85,29.73] LB m [56.88,57.03] [34.33,34.45] [23.75,23.85] UB m [60.31,60.56] [37.14,37.98] [26.82,28.90] No-borrowng LB ADP [31.67,31.77] [19.69,19.76] [15.14,15.32] UB ADP [33.39,33.55] [21.41,21.76] [16.34,16.83] LB m [30.52,30.61] [19.44,19.49] [14.67,14.71] UB m [33.01,33.18] [21.01,21.38] [15.57,16.26] Table 4: Model II, portfolo performance. The parameter set s defned n Table 2 and the problem horzon s T = 5 years. The rsk averson coeffcent takes values γ = 1.5, 3, 5. All results are computed for the ntal value of the state varable X 0 = 0. The rows marked LB ADP and UB ADP report the estmates of the bounds on the expected utlty acheved by usng the ADP portfolo strategy. The ADP algorthm uses fourth-order approxmatons and reles on a myopc polcy for ntal approxmaton and extrapolaton. The rows marked LB m and UB m report the correspondng results based on the myopc portfolo strategy. Expected utlty s reported as a contnuously compounded certanty equvalent return. Approxmate 95% confdence ntervals are reported n brackets. We report the results both for ncomplete markets and the case of no-borrowng constrants. 19

22 A Upper bound on expected utlty: a quadratc subproblem In ths secton we derve explct solutons to constraned quadratc optmzaton problems nvolved n estmatng an upper bound on the optmal expected utlty. See HKW for a general formulaton and defntons related to the dual formulaton. We consder here the case of ncomplete markets and no-borrowng constrants. We start wth a smpler case of Model I. The quadratc programmng problem that needs to be solved repeatedly durng Monte Carlo smulatons s mn s.t 1 2 Λ ˆΛ ν 2 σ ˆΛν 1 = σ 1 Λ + ν ν s feasble Above, Λ s the canddate for a market prce of rsk n a fcttous market, obtaned from an approxmate value functon produced by the ADP algorthm (see HKW for defntons). ˆΛ ν s the market prce of rsk n a fcttous complete market, whch we use to compute an upper bound on the expected utlty. ν s a scalar n ths case, whch parameterzes feasble fcttous markets. When markets are ncomplete, the only feasble value of ν s zero. We are thus faced wth a smple projecton problem, mn 1 2 Λ ˆΛ ν 2 s.t σ 1 (ˆΛ ν Λ) = 0 whch has an explct soluton: ˆΛ ν = Λ [ ] (σ 1 σ1 ) 1 σ 1 ( Λ Λ) σ1. (22) In the case of no-borrowng constrants, the feasble set s ν 0. Therefore, we are solvng mn 1 2 Λ ˆΛ ν 2 s.t σ 1 (ˆΛ ν Λ) 0. We are thus faced wth two possbltes. If σ 1 ( Λ Λ) 0, then ˆΛ ν = Λ. Otherwse, ˆΛ ν s gven by (22). 20

23 We now consder Model II. The quadratc subproblem has a smlar form. Let Σ denote the dffuson matrx of returns on rsky assets and defne varables y = ˆΛ ν Λ and b = Σ(Λ Λ). Then, the quadratc subproblem takes form mn s.t. 1 2 y 2 Σy νı = b ν s feasble ı s a vector of ones of the same dmenson as the number of rsky assets and, agan, ν s a constant. When markets are ncomplete, ν = 0 and the optmzaton problem reduces to whch has the soluton mn s.t. 1 2 y 2 Σy = b y = Σ (ΣΣ ) 1 b. In the case of no-borrowng constrants, the feasble set s gven by ν 0. We are now facng two dstnct cases. Relaxng the equalty constrants wth Lagrange multplers π, we see that f the system of lnear equatons y Σ π = 0 ı π = 0 Σy νı = b has a soluton wth ν 0, then we have an optmal y. Otherwse, we must set ν = 0, and the optmal y s gven by the soluton for the case of ncomplete markets. 21