Lecture 5: Applying Dynamic Programming to Consumption & Asset Choice Note: pages -28 repeat material from prior lectures, but are included as an alternative presentation may be useful
Outline. Two Period Consumption and Saving 2. Consumption/Saving/Portfolio Problem 3. Dynamic Programming g Approach 4. Efficiency conditions 5. How do people behave? 6. Behavior with serially independent returns: Levhari and Srinvasan 7. Sandmo s three risk concepts 8. An alternative way of increasing risk 2
. Basic two period consumption and saving problem Consider a household that has preferences U = u( c ) + βu( c ) 0 with u () c ( ) 0 c γ = and γ > γ and has constraints of the form a + y = c + a and a + y = c + r or 0 0 0 a + y + y = c + c + r + r 0 0 0 3
Optimal consumption Marginal benefits and costs of saving are equated (Euler/Fisher view) uc( c0) = βu( c) + r Marginal utility in each period is equated to cost along budget constraint (Hicks view) u ( c ) = Λ c 0 βuc ( ) =Λ + r + 4
Optimal consumption Restriction on growth rate of consumption (Fisher s rule) = [ β ( + )] γ 0 c r c Restriction on level of consumption (Λ- constant demand for consumption) c c 0 = [ Λ] γ = Λ [ ] + r γ 5
Pinning down the level Using pdv budget constraint and efficient consumption results c0 + c = a0 + y0 + y + r + r γ c { + [ β ( + r)] } = a + y + y + r + r a0 + y0 + y c 0 = + r [ γ ( r γ + β + ) ] 0 0 0 6
Key aspects of this example Higher income in present or future raises consumption in both periods Higher wealth raises consumption in both periods Higher interest t rate raises growth rate of consumption (more so if γ is smaller) Higher interest rate has ambiguous effect on level of c 0. 7
What s missing Multiple periods (for permanent income or lifecycle model) Effect of uncertainty about future income Effect of uncertainty about returns On overall consumption On allocation of wealth about various assets 8
2. A dynamic model of optimal consumption, saving and portfolio allocation Consider an individual who has random labor income, y, and can invest in a variety of assets which bring stochastic returns. This individual views both his income and asset returns as a function of a vector of exogenous state variables. 9
Budget constraint J ( p ( ς ) + d ( ς )) a + y ( ς ) = c + ( p ( ς )) a + j t j t jt t t t j t j, t j= j= J In words: There are J different assets. Financial i wealth is based on the prices of those assets and their payouts Financial wealth plus current income (y) can be used to finance consumption or asset holding Asset holding this period leads to financial wealth next period 0
Setting up a dynamic program Think about what the state variables are (asset positions and variables useful for forecasting income and asset prices). Form Bellman equation va (, ς ) = max [ uc ( ) + β Eva (, ς ) ( a, ς )] t t ct, at+ t t+ t+ t t st.. pa + da + y = c + pa t t t t t t t t+ with p = [ p... p ]; d = [ d... d ]; and a = [ a... a ]' so that p a t t Jt t t Jt t t Jt J = p d t t jt jt j=
Time Invariant Problem va (, ς ) = max [ uc ( ) + βeva ( ', ς ') ( a, ς)] ca, ' st.. p ( ς ) a + d ( ς ) a + y ( ς ) = c + p ( ς ) a' 2
Maximize subject to constraint(s) What are possible approaches? Substitute out for consumption (Euler equation) Add multiplier, select consumption and asset holdings What are possible problems? Investment might be so productive that infinite horizon utility could be infinite (choices not well defined) People might have so much wealth that they are satiated (rule out with marginal utility of consumption which is strictly positive) Some assets could be dominated by others (bad assets should be shorted or at least not held, in which case there is a missing inequality constraint ) We ll assume that none of these is an issue, but we must be alert to situations in which our assumption is violated.. 3
Maximize: What do we get? Policy functions: Rules for how people consume and invest c a = π ( a, ς ) ' = m( a, ς ) Value function: Level of utility associated with those rules. ues va (, ς ) = [ u( π( a, ς)) + βevma ( (, ς)) ( a, ς)] 4
3. Efficiency conditions A. Euler Equations which are derived when budget constraint is substituted in and consumption is substituted out B. Euler Equations which h come from a comparable approach to a DP problem C. FOCs which come from constrained optimization of u+βv 5
Euler Equations Create the reduced form objective J J ' (, ', ς ) = ( ( j ( ς) + j( ς)) j + ( ς) ( j ( ς )) j ) j= j= raa u p d a y p a And then maximize i with respect to a. Can do this with Bellman equation, but also directly with expected utility. 6
Approach : Directly calculating l utility consequences of varying asset i 0 = { ( )} = { ( ) + ( ) +...} a E l t β u ct+ j u ct βeu t ct+ it, + l= ait, + J J = { u( ( p ( ς ) + d ( ς )) a + y ( ς ) ( p ( ς )) a ) a it, + j t j t jt t t j t j, t+ j= j= J + β Eu ( ( p ( ς ) + d ( ς )) a + y ( ς ) ( p ( ς )) a ) +...} J t j t+ j t+ j, t+ t t+ j t+ j, t+ 2 j= j= = p ( ς ) u ( c ) + β E {[ p ( ς ) + d ( ς )] u ( c )} i t c t t i t + i t + c t + J with c = ( p ( ς ) + d ( ς )) a + y ( ς ) ( p ( ς )) a t j t j t jt t t j t j, t+ j = j = J 7
Economic Rule Adjust the holdings of each asset to the point where the current cost, measured in terms of foregone utility of consumption p ( ς ) u ( c ) i t c t Equals the future benefit, measured in terms of expected utility of consumption gained. β E{[ p ( ς ) + d ( ς )] u ( c )} t i t+ i t+ c t+ 8
Approach #2: Obtaining the Euler Equation from the direct DP efficiency conditions Using the reduced form objective e and the value function, va (, ς ) = max [ raa (, ', ς) + βeva ( ', ς ') ( a, ς)] a' We can find that t raa (, ', ς ) va ( ', ς ') 0 = E { } ( a, ) a + β a ς ' i ' i 9
Substituting, we get the same economic efficiency condition (Euler equation) raa (, ', ς ) va ( ', ς ') 0 = + β E{ } ( a, ς ) ' ' a a i = pu () c + ( p ' + d ') u (') c i c i i c defnof fr + envelopetheorem i 20
Approach #3: Constrained optimization i i Alternatively, we can think of this as a constrained optimization problem and study it using Lagrangian techniques. This is convenient now and even more so if there are additional constraints on states (e.g., no short sales), when we would move to the Kuhn-Tucker approach. L= u() c + β Ev( a', ς ') (, a ς) + λ[ p( ς) a+ d( ς) a+ y( ς) p( ς) a' c] 2
Consumption Efficiency conditions Asset holdings Multiplier 0 = u ( c) c v va ( ', ς ') 0 = β E ( a, ς ) λ p ( ς ) i a ' i λ 0 = [ p( ς ) a+ d( ς) a+ y( ς) p( ς) a' c] 22
Solving these FOCs, if we knew the v function We would get the policy functions discussed above c= π ( a, ς ) a' = m( a, ς ) An additional implication that the mulitiplier also depends ds on the parameters a λ = la (, ς ) 23
The value function Is the Lagrangian g (Objective) evaluated at optimal policies and associated shadow prices (optimal policies) va (, ς ) = u( π( a, ς)) + βevma ( (, ς), ς ') ( a, ς) + λ ( a, ς)[ p( ς) a + d( ς) a + y( ς ) p ( ς) m( a, ς) π( a, ς)] 24
Partial derivatives of v The Envelope Theorem J ' va (, ς ) π( a, ς) v' a j = uc * + βe{ } ( a, ς) ' a a a a i i j= j i λ ( a, ς ) + [ p ( ς ) a + d ( ς ) a + y ( ς ) p ( ς ) m ( a, ς ) π ( a, ς )] a i J ' a j + λ [( pi + di ) pj ] a = λ( p + d ) i i j= i 25
Link back Asset holding FOC + ET implies va ( ', ς ') 0 = βe ( a, ς) λ( a, ς) p ( ) ' i ς a i = βe[ λ( a', ς ')( pi( ς ') + di( ς '))] λ( a, ς) pi( ς) = β Eu [ c( π( a', ς '))( p i( ς ') + d i( ς '))] u c( π( a, ς)) p i( ς) or = βeu [ ( c')( p( ς ') + d( ς '))] u( c) p( ς) c i i c i The last lines also use the consumption efficiency condition That is: the constrained optimization approach implies the Euler equation. Jargon ( our model is a number of euler equations and constraints ) 26
4. How do people behave? According to our modeling so far, they behave efficiently. But, without other assumptions, we can t be a lot more specific. One route - exemplified by Levhari and Srinvasan--is to make specific functional form and environmental assumptions, then to work to characterize the policy functions. They assume No income (or certain income and a certain bond) Independently and identically distributed returns 27
5. Optimal consumption in some particular cases Outline of discussion of various infinite horizon models A. Dynamic programming with a single, certain interest rate, but no labor income B. Dynamic programming with a single, stochastic interest rate, but no labor income C. Dynamic programming with multiple returns, but no labor income D. Adding back in labor income 28
5.A. Dynamic Program with ihcertain Rate of freturn r = rate of return on a discount bond. A discount bond is an asset with a variable price, but a unit fact value, so that there is a a natural state equation similar to those studied above pa '+ + c = a with p = + r 29
Bellman equation va ( ) = max {() uc + βva ( ')} ca, ' st.. a= pa' + c γ γ Assume c with γ > 0 30
FOCs For consumption and asset accumulation 0 = u λ u c 0 : v p + β a 0 : a pa ' c ' ' 3
Using the FOCs to determine a policy function One approach is to try to determine policy function off Euler equation via guess and verify In this case, a simple guess that consumption is proportional to assets works. But if we had guessed c=k+a then we would reach a contradiction (try it!). This is like undetermined coefficients for RE models which we will study later. 32
Policy function (cont d) Working off the EE γ Efficient Consumption : c = β ( c ') + r Guess : a ' = ga and c= κa : ( ) [ '] [ ] [ ( )] + r γ γ γ / γ Hence κa = β κa = β κga so β + r = g Implication of asset accumulation a' = ( + r)[ a c] ga= ( + r)[ a κa] g κ = ( ) r γ + 33
Policy (g) depends on Time preference: consumption growth and asset growth increases (g rises) if people discount the future less heavily (β rises). Rate of return: consumption and asset growth increases if the rate of return is higher. Preference parameter: γ controls shape of indifference curves (with / γ being the intertemporal elasticity of substitution), so that it is natural that consumption and asset growth are larger (with (+r)β >) if people are more willing to substitute future for current consumption. 34
Policy (κ ) depends on Rate of return: sign of effect depends on (-γ) due to offsetting income and substitution effects Preference parameters: level and growth rate of consumption are inversely related. 35
The value function We know the policy function, so that we can evaluate the utility function so as to determine the value function, t t γ { β uc ( )} = { t β [ ( ct ) } t= 0 t= 0 γ = t t γ { β [ ( κa 0 g ) } t= 0 γ t γ = ( κag 0 ) γ β g γ 36
Restrictions on (+r) Factor in utility function must be finite Since g is increasing in r this sets and upper bound on r Get restriction using above and definition of g, i.e., > β [ ( + )] = γ / γ g and β r g γ γ γ γ γ > ββ [ ( + r )] = β [( + r )] 37
What else? Not necessary in this case, but widely used in other settings: construct a numerical approximation to the value and/or policy functions. Why necessary? Because we can readily imagine more complicated and realistic settings than we can solve analytically. 38
What else can we do? Build up a series of value functions analytically, looking for instruction on the likely infinite horizon form of the value function Then, look for a policy function which maximizes the limiting value function (or take a limit of policy functions) 39
5.B Dynamic program with an uncertain rate of return Assume there is a single asset with an uncertain return. We require that the rate of return be a serially independent random variables p( ς ') + d( ς ') p ( ς ) = + r In words: we rule out ALL predicable variations in the moments of asset returns (mean, variance, ). Hence, there is no need to track history. It is enough to know the realized return as a state. 40
Bellman Equation var (, ) = max { uc ( ) +β Eva ( ', r ')} ca, ' st.. ra = a' + c Assume c γ with γ γ > 0 4
Euler Equation and Policy Function γ : [( γ Consumption c = β E + r ')( c ') ] or E[( r γ = β + ')( c'/ c ) ] Guess : c = κ ( a ( + r )) and a ' = g ( r ) a Since a ' + c = a( + r ), g( r ) = ( κ)( + r ) Hence : c '/ c = a '/ a = ( κ )( + r ') γ γ Hence :[ κ] = βe( + r ') γ / γ Hence :[ βe( + r ') ] = κ 42
LS results and puzzle Savings rate (c scaled by wealth) does not depend on wealth, even in presence of uncertainty Savings rate does not depend on the distribution under particular parameter restriction (γ=, which corresponds to log utility). 43
(Frequently employed) special case Lognormal returns Suppose that x is distributed N(μ, σ 2 ) Then X=exp(x) is lognormal, i.e., log(x)=x is normal Useful Properties of Normal Linear combinations of normals are also normal with mean and variance readily calculated. z = [ z z z ]~ N( μ, Σ) 2 n T lz = ( l z + l z +... l z ) ~ Nl ( μ, l Σl ) 2 2 n n Summarized by just two parameters, corresponding to first and second moments 44
Related useful properties of LN Since log(xy)=log(x)+log(y), log(x) log(y), products of log normals are lognormal Since log(x a )=a*log(x), then powers of lognormals are lognormal. How use on Euler equation? Log(+r)=normal Log[(c /c) -γ ]=normal Log[(+r)(c /c) -γ ]=linear combination of normals = normal 45
Useful properties (cont d) Suppose that x is distributed N(μ, σ 2 ) and X=exp(x). Then Mean of lognormal is given by E(X)=E(exp(x))=exp(μ+.5σ +5 2 ) Variance of lognormal is given by =exp(2μ+2σ 2 )- exp(2μ+σ 2 ) A mean preserving spread on X involves increasing σ 2 holding (μ+.5σ 2 ) constant, i.e., involves adjustment in both normal parameters. Drawback: linear combination of LNs is not LN (difficulty for application to portfolio theory) 46
Back to dynamic model taking an approach like that above Consumption : = β E[( + r ')( c '/ c ) γ ] γ γ From above :[ κ ] = β E( + r ') γ 2 2 From LN [ κ] = βexp(( γ) μ+ ( γ) σ ) 2 2 2 Hence :log( κ) = [(( γ) μ+ ( γ) σ + log( β)] γ 2 Hence :log( g) = log( + r ) Elog( + r ) + + + + γ 2 2 2 [ μ {( γ) γ} σ log( β)] 47
Interpretation Consumption level now depends on parameters of normal (log normal) Growth rate involves Effects of shocks (unexpected returns affect wealth) Effects of plans (need to interpret further and will do so later) 48
Mean-preserving spread Increase variability holding fixed mean: corresponds to pure effect of increased uncertainty Requires simultaneous adjustment of two parameters, (dμ+.5dσ 2 =0). Effect on consumption (saving) is of interest t (neglect stock-flow distinction) 49
Effect of uncertainty on saving Depends on level of γ parameter 2 2 Hence : log( κ) = [(( γ) μ+ ( γ) σ + log( β)] γ 2 = + + γ 2 2 2 2 log( κ) [(( γ)( μ σ ) γ( γ) σ log( β )] κ = ( γ ) 2 2 κ σ dμ+ dσ = 0 2 2 50
5.B.2. Risk and Economic Responses Sandmo s classic paper distinguishes between three types of economic responses Risk Aversion Precautionary Saving Consequences of interest rate uncertainty for saving Another important effect is effect of return uncertainty on portfolio composition, but this is not treated in Sandmo s paper although he (and many others) have worked on it subsequently 5
Risk Aversion An individual is risk averse if increased risk reduces his welfare. Measured in terms of attitude toward gambles: when confronted with an uncertain prospect with a mean of zero, a risk averter will decline it unless he is paid to take it. Two standard gambles Fixed dollar amount (absolute risk aversion) Fraction of current wealth (relative risk aversion) 52
Figure 53
Absolute Risk Aversion Consider an individual with a utility function u(c) which is increasing and concave. Suppose that the individual is initially certain about consumption, but is then told: you must now take c+b+e with probability ½ and c+b-e with probability ½, but b will be set so that you have the same expected utility In this expression, e is a measure of the extent of the risk and b is a measure of the amount of a fixed benefit the individual must be paid to take the bet. 54
The Arrow-Pratt ARA measure A measure of relative risk aversion is found as follows. Indifference implies uc () = uc ( + b+ e) uc ( + b e) 2 2 This makes b an implicit function of e. b(0)=0 0 since u(c) = u(c): no risk, no benefit (bribe) 55
Differentiate the equation Working out other derivatives of b(e) b b 0 = uc( c+ b+ e)( + ) + uc( c+ b e)( + ) 2 e 2 e b uc( c+ b e) uc( c+ b+ e) = e uc( c+ b+ e) + uc( c+ b e) b e 0 = 0 e = Words: no premium necessary for really small bet Arrow used same approach to show that a risk averse individual would always take a little bit of a favorable (positive mean) bet. 56
Working out Differentiating again 0 = { uc ( c+ b + e ) + u ( c+ b + e )( + ) 2 ( e) e 2 b b 2 b 2 cc b b + + + + + + 2 ( e) e 2 2 { u c( c b e) u ( )( ) 2 cc c b e } 2 b ucc 0 = 0 2 e= > ( e) u c 57
Implication The premium is approximately (in the sense of a second order Taylor series approximation around zero) 2 b b b= b(0) + ( e 0) + ( e 0) 2 e 2( e ) u u = e = ε = 2 u 2 u 2 cc 2 cc ( ) ( )var( ) ( )var( ) c c 2 A c ε + e with prob= /2 whereεε = e with prob = /2 58
In words Absolute risk aversion A(c) is shows the size of a premium demanded as function of variance OR absolute risk aversion is twice the premium demanded per unit of variance, with a small but not infinitessimal it i bet 59
Relative risk aversion Consider proportional risks (c(+e) or c(-e)). Another measure R(c) which is frequently used in many contexts. It is defined implicitly by the equation below and takes the form indicated uc () = uc ( + bc+ ce) + uc ( + bc ce) 2 2 cucc b= ( ) var( ε ) = R( c) var( ε ) 2 u 2 c 60
Constant risk aversion functions Constant absolute risk aversion u cc uc () = exp( Ac ) Ac () = = A u Constant relative risk aversion c R cucc uc () = c Rc () = = R R u c 6
Saving with uncertain income Sandmo considers a two-period problem in which an individual Has general preferences U(c,c 2 ) which are increasing in each consumption and concave Faces uncertain future income (y 2 ) Can save at a given rate so that c 2 =y 2 +R(y -c ) Note: R=+r for simplicity not relative risk aversion 62
Our analysis Will focus on additively separable case and will begin with a graphical approach Thus, the utility function is Uc (, c ) = uc ( ) + βuc ( ) 2 2 We use expected utility as in Sandmo 63
Euler Equation A necessary condition for optimal saving is u ( c ) = β REu ( c ) c c 2 u ( y s) = β REu ( y + Rs) c c 2 64
Figure 65
Income uncertainty What is the effect of greater uncertainty about future income on the expected marginal utility of consumption? One strategy is to look at increasing e in Euc( y2 + Rs) = uc( y2 + Rs + e) + uc( y2 + Rs e) 2 2 = uc( c2 + e) + uc( c2 e) 2 2 66
Figure 67
Comments and implication Greater marginal utility with greater uncertainty: Arises whenever marginal utility is a strictly convex function of c Is absent if marginal utility is linear (which occurs if utility is quadratic) Results in the upward shift that we saw in the previous figure and thus leads to a rise in saving. 68
Comments and interpretation Convexity of marginal utility requires that the third derivative (u ccc ) is positive, so that we have a simple test for the presence of this effect. Sandmo shows (in a more general setting) that a sufficient condition for a positive saving effect is a declining (absolute) risk premium: this is interesting because it is suggestive of two behaviors that we can link: attitude toward gambles (expressed in insurance, for example) and attitude toward saving. 69
Deriving the sufficient condition ucc premium : b( c) = var( e) 2 u uccc ucc 2 derivative : bc ( c) = var( e)[ ( ) ] 2 u u uccc ucc 2 Hence : bc ( c) < 0 [ ( ) ] > 0 u u 2 ( ucc ) Or : uccc > > 0 u c c c c c c 70