Decoupling and Agricultural Investment with Disinvestment Flexibility: A Case Study with Decreasing Expectations

Size: px
Start display at page:

Download "Decoupling and Agricultural Investment with Disinvestment Flexibility: A Case Study with Decreasing Expectations"

Transcription

1 Decoupling and Agricultural Investment with Disinvestment Flexibility: A Case Study with Decreasing Expectations T. Heikkinen MTT Economic Research Luutnantintie 13, Helsinki FINLAND tiina.heikkinen@mtt.fi June 30, 2007 Abstract This paper studies the effect of decoupling on optimal investment in agriculture assuming disinvestment flexibility. Agricultural income being determined by policy processes is subject to policy uncertainty. Case study examples suggest that assuming disinvestment flexibility, decoupling increases income stability, and a higher level of investments can be achieved even with lower subsides. Increased income stability may also diminish the dynamic cost of income uncertainty. When decoupling of income from production, the stability of the compensating direct payments should be ensured. Keywords: decoupling in agriculture, investment analysis 1 Introduction Decoupling is one of the key issues in EU common agricultural policy. This paper addresses the dynamic effects of decoupling on investments, focusing on the role of expectations. Previous work in [OECD2000] and [Heikkinen and Pietola2006] addresses the role of expectations about future payments to investment decisions in a dynamic setting. This paper extends [Heikkinen and Pietola2006] to study the effect of decoupling the national support on investment in agriculture assuming disinvestment flexibility 1. 1 Disinvestment was not considered in [Heikkinen and Pietola2006]. 1

2 The investment model is based on the dynamic optimization model presented in [Heikkinen and Pietola2006], where investments are studied from the point of view of a representative dairy farm, assuming a Markov model of expectations: farmers believe that with high probability the current income level remains the same next year; however, due to decreasing expectations regarding future income support, a positive probability is assigned to lower levels of income 2. In addition to making future income uncertain, political processes may also make the farmer s current income unobservable at the time when the investment decision is made. In [Heikkinen1998] the dynamic uncertainty cost is defined as the loss in the value and amount of investment due to unobservable return; Appendix A summarizes the definition for dynamic uncertainty cost. Assuming nonincreasing income expectations, decoupling the support from production may make production an unprofitable alternative compared to zero production. For example, assuming zero production implies zero labor cost and production cost, the income of an inactive farmer can be defined as equal to a direct payment. This corresponds to assuming a (partial) disinvestment takes place whenever net income from production is negative: most animals are sold under nonincreasing income expectations. If the direct payment is fixed, decoupling associated with disinvestment flexibility may increase income stability. Decoupling accompanied by a compensating fixed income subsidy may potentially be welfare-enhancing in comparison to a more costly subsidy program with more volatile price-based support. The following observations based on a case study illustrate the potential benefits of a fixed national support: A fixed compensating payment stabilizes income, thereby increasing investments even if the aggregate subsidy level is decreased. Assuming disinvestment flexibility with nonincreasing income expectations, when removing both national support and investment subsidies in the case study, a compensating payment 90 % of previous national support increases investment probability to one (from 16 % before decoupling national support). The dynamic uncertainty cost can be reduced by a fixed compensating payment even if at a lower aggregate subsidy level. Assuming disinvestment flexibility, when decoupling both national support and investment subsidies, a compensating payment 90 % of previous national support im- 2 [Heikkinen and Pietola2006] implicitly made a simplifying assumption that investment increases productivity: after investment took place, a new transition matrix was assumed to apply, corresponding to a higher expected return (for simplicity, corresponding to the transition matrix at the beginning of time horizon). Even though there is some evidence of productivity-enhancing effects of investment [A.-M. Heikkilä2004], this paper studies the effect of decoupling on investment and uncertainty cost without assuming a productivity-enhancing effect. The main effect of removing the (somewhat arbitrary) assumption of productivityenhancement is to reduce the investment level. 2

3 plies a saving in uncertainty cost corresponding to approximately 7 % of the value of investment in the case study. The dynamic cost of uncertainty at a given time is the cost due to an unobservable return that time period, e.g. year. Even if the return is not observable at a given time, the dynamic uncertainty cost can be zero if dynamic decision making is based on a relatively more stable income process. When enforcing decoupling, it is important to consider the stability of the compensating payments. The organization of the paper is as follows. Section 2 summarizes the investment problem of an optimizing representative agent following [Heikkinen and Pietola2006]. Section 3 studies the effect of decoupling on investment over time. A decision tree representation for problem (3) subject to (2) it outlined in Appendix B. A simplified decision tree illustrates the complexity of the decision tree representation of the investment problem, motivating a dynamic programming approach to the investment problem. 2 Optimizing Investment At each time t = 1,..., T a representative farm decides the investment at t, I t. Denoting by I the total available budget for the investment at t, the decision I t at t satisfies I t {0, I}, t = 1,..., T. (1) Let r t denote the return on investment (%) per time period at time t. Due to variability in the rate of return, the future value of investment is random. Letting I a denote the aggregate budget, the aggregate budget constraint requires: T I t I a. (2) t=1 Letting ρ denote the internal rate of return, the discount factor b (0, 1] is defined as b = 1/(1 + ρ). The dynamic optimization problem of a representative firm can be written as: max E[ b t (r t I t + b k t r k I t I t (1 p))] (3) t=1 k=t+1 where p denotes the investment subsidy % and E denotes the expectation operator. Two versions of problem (3) subject to (1)-(2) are studied in [Heikkinen and Pietola2006]; in the first model (Model 1), it is assumed that r t is observable when the investment decision is made at time t; in the second model (Model 2) only r t 1 is observable at time t. Since the future values of the investment are unknown in both models, there is an opportunity cost to making the investment decision at the beginning of the time horizon [Dixit and Pindyck1994]; the firm has the option to postpone the investment decision. 3

4 3 Decoupling and Optimal Investment with Disinvestment Flexibility In a dynamic model, expectations matter to optimal investment behaviour. Present policies may affect farmers expectations of future policies. For example, in a dynamic investment model in [OECD2000], a perceived probability of a subsidy payment (consisting of a lump-sum payment and a payment per hectare) is assumed to depend on the level of production in the previous period. [Heikkinen and Pietola2006] assumes the probability distribution of a direct payment depends on the previous payment level. Specifically, [Heikkinen and Pietola2006] studies optimal investment assuming a timecorrelated income, modelled as a Markov chain. In this paper the same numerical optimization model will be applied to evaluate the effect of decoupling of national income and investment support on investment, assuming disinvestment flexibility. The model is based on considering a case of a representative dairy farm with a herd size of 130. Table 1 summarizes the assumed transition probabilities between states in terms of return on investment (ROI %). For example, 5 % corresponds to 2003 ROI % accompanied by a decrease by 20 % in total (subsidized) income 3. To focus on the role of policy expectations, consider a simplified model assuming that costs and parameters other than subsidies remain at 2003 level. The transition probabilities can be interpreted as reflecting uncertainty regarding the compensating direct payments 4, varying between 0 (corresponding to ROI 5 %) and (ROI 30 %). With high probability, the return on investment remains unchanged (except for the highest rate of return r = 0.3 that is assumed to decrease with probability 99 % in Table 1). Disinvestment Flexibility Consider the above case study based on a representative large dairy farm. After decoupling the subsidy no longer depends on production and it is optimal to produce only if the net return from production is positive. Otherwise, it is better to set production to zero and take only the direct decoupled support. With no production, both the labour cost of the farmer and the production cost 3 On average, 2005 income from sales in dairy production of large farms was at 2003 level whereas total subsidies slightly increased. However, profitability decreased 2005 due to increased cost level. 4 A direct payment can be defined to match a producer price change given production. Assuming price-based payments are continued, the transition probabilities could alternatively be interpreted as modeling uncertainty regarding the subsidized price level. However, unlike with direct payments, price-based support is not obtained if ROI % is negative (production is zero). 4

5 Table 1: Transition Probabilities between States (ROI %) are assumed to be zero. This corresponds to assuming a (partial) disinvestment takes place whenever net income from production would be negative: most animals are sold under nonincreasing income expectations. Examples of the effect of decoupling will be studied shortly, assuming disinvestment flexibility under nonincreasing income expectations. Disinvestment flexibility in this paper refers to partial disinvestment as outlined above. In contrast, allowing for full capital disinvestment, reducing available production facilities, would modify constraint (1) to: I t {0, I, I}, t = 1,..., T (4) where I I a is the maximum change per period in cumulative investments. However, in examples with I = I a, full capital disinvestment, implying zero return, was not observed to take place. To study the effect of decoupling on investments, consider the following two modifications to the case study in [Heikkinen and Pietola2006]: 1. the removal of price-based national support, assuming a fixed compensating direct payment and disinvestment flexibility. 2. the simultaneous removal of national support and investment subsides, assuming a fixed direct compensating payment and disinvestment flexibility. Specifically, the national-level compensating direct payments are assumed to be fixed, whereas the direct payments due to previous decoupling are assumed to be uncertain according to Table 1. The farm is assumed to make the production decision based on net return; assuming zero production implies zero labor cost, the income of an inactive farmer is defined as equal to a fixed direct payment. Decoupling National Support Consider scenario 1 first, assuming the complete removal of price-based national support. In the case study example, this implies a saving of approximately per farm. A 100 % direct payment compensation corresponds to a subsidy 7 5

6 %- 9 % of gross revenue depending on the uncertain component of the income subsidy. In Table 1, ROI % before decoupling national support may take 5 values: r = [0.3, 0.16, 0.11, 0.05, 0]. The ROI % after decoupling depends on the level of direct compensation. Let r c [4] denote the ROI % corresponding to the fourth element in ROI-vector at direct compensation level c. For example, letting b = 0.94 and the start state r c 0 = r c [4] for each compensation level c, the effects of decoupling with a fixed compensating payment can be summarized as follows: A fixed income compensation corresponding to 70 % of the support level before decoupling suffices for making the cumulative investment probability equal to one. The dynamic cost of uncertainty is zero assuming a fixed compensation corresponding to 70 % (or higher) of the previous support. This corresponds to a saving in average uncertainty cost by approximately 7 % of the value of investment. Figure 1 plots the cumulative investment probability over 6 years as function of the fixed direct payment in the case study assuming b = 0.94 and r c 0 = r c [4]. The cumulative investment probability over a six year period before decoupling the national support was approximately 16 %. Assuming a fixed compensating payment, a higher level of investment can be achieved at a much lower cost. By increasing income stability, it is thus possible simultaneously to reduce the total support increase investments decrease the dynamic cost of uncertainty Decoupling Investment Subsidies Investment subsidies in terms of a given percentage of investment expenditure are by definition directly linked to the value of investment and to production. Full decoupling investment subsidies here means setting the subsidy percentage (0.35 % in the case study) to zero. For example, letting b = 0.94 and r c 0 = r c [4], the effects of removing both national support and the investment subsidy with a fixed compensating payment can be summarized as follows: Assuming a compensating payment 90 % or higher of the previous national subsidy increases the investment probability to one; before decoupling national support the investment probability was 16 % over 6 years. Assuming a compensating payment 90 % or higher implies the dynamic uncertainty cost is zero. 6

7 cumulative investment probability direct compensating payment relative to national subsidy before decoupling Figure 1: Cumulative investment probability over 6 years as function of direct payment, when removing national price-based support, with b = 0.94, r c 0 = r c [4], assuming observable value (solid curve) and with unobservable value (*) 7

8 cumulative investment direct compensating payment relative to national subsidy before decoupling Figure 2: Cumulative investment probability over 6 years as function of direct payment, when removing both national support and investment subsidy assuming observable value (solid curve) and unobservable value (*), with b = 0.94, r0 c = r c [4] Figure 2 depicts the investment probability over 6 years as function of direct payment (% previous national subsidy level), when removing both national support and the investment subsidy in the case study. Figure 3 depicts the dynamic cost of uncertainty over time as function of direct compensating payment. With a compensating payment 90 % or more of previous national support the uncertainty cost is zero. With compensation lower than 90 %, the dynamic uncertainty cost averaged over time depends on the compensating payment, varying between 2 % (70 % compensation) and 8 % (85 % compensation), as compared to 7 % before decoupling. The underlying reason for the outcomes listed above is the greater income stability due to the assumed fixed compensating support along with assumed disinvestment flexibility. Assuming production is zero unless the net return is positive, ROI % will improve relatively more in cases where ROI % would be zero or negative without direct decoupled support. To illustrate the effect of the flexibility in production (zero disinvestment cost), consider the above example assuming production is always at full capacity level. Figure 4 depicts the investment probability when unprofitable production is continued. This scenario could be motivated e.g. by a delivery contract implying a penalty for failing to meet a target supply. Comparing Figures 2 and 4, it can be seen that 8

9 REVPI(t) time index Figure 3: Dynamic uncertainty cost relative to expected investment, when removing both national support and investment subsidy assuming direct payment 70 % of previous national support (+ sign), 80 % (solid curve), 85 % (dashed curve), 90 % or more (*), with b = 0.94, r0 c = r c [4] 9

10 optimal investment can be sensitive to the production decision. A special case of decoupling is to assume a fully deterministic compensating payment. In this case investment decision is made at t=1 provided ROI % meets the internal interest rate. Continuing the example discussed above assuming b = 0.94, the direct fixed compensation required for investment to take place corresponds to approximately 95 % national support before decoupling (cf. Figure (2)). On the Size of the Investment Problem The dynamic investment model is a simplified framework that can be extended in many ways to a more realistic model. For example, a more realistic scenario model would result allowing both national policy and common agricultural policy each have 5 possible states. A single aggregate budget constraint corresponds to two possible budget states: {0, I a }. The vector of possible states then can be defined as S = [z, s 11, s 12,..., s 54, s 55 ] where z is a 25-vector of zeros corresponding to each possible state where the budget has been used up, and s ij is the state of return before investment has taken place when the state of national policy is i and the state of common policy is j. In a Markov decision process, each possible action (invest, do not invest) is associated with a transition matrix. Letting the submatrix P1 s denote the transition matrix between 25 possible states in terms of national and CAPsupport, the transition matrix P1 when investing at time t will be P1 = [ P1s 0 P1 s 0 ], (5) where 0 is a matrix of zeros. The transition matrix when not investing at time t is: [ ] P1s 0 P2 =. (6) O P1 s Thus, allowing both national policy and common agricultural policy each have 5 possible states would increase the size of the transition matrix to 50 50, assuming a single aggregate financial constraint. The size of the transition matrix can be reduced by introducing an artificial state modeling the situation when the budget has been used up 5 ; the size of the transition matrix will then be reduced to There is a trade-off between the accuracy of the scenario model and the size of the computational problem. An extension to explicitly allowing a disinvestment option as formalized in (4) further increases the computational cost. In this case, the decision-maker 5 the additional state corresponds to ROI=0; after investment has been made, a new transition probability matrix applies: one where each state leads to this additional state with probability one. 10

11 cumulative investment probability direct compensating payment relative to national subsidy before decoupling Figure 4: Cumulative investment probability over 6 years as function of direct payment, when removing both national support and investment subsidy assuming production at full capacity with observable value (solid line) and unobservable value (*), b = 0.94, r0 c = r c [4] faces three possible actions: to invest, to hold current investment and to disinvest. Letting I a = I, the transition matrix P1 in (5) corresponds to making investment and P2 in (6) corresponds to holding current investment. The transition matrix when disinvesting can be foramlized as [ ] 0 P1s P3 =. (7) 0 P1 s With m possible budget states the size of each transition matrix will be 25m 25m. 4 Conclusion In a dynamic setting expectations of future agricultural policies are relevant for investment decisions. This paper studies the effect of decoupling on optimal investment assuming disinvestment flexibility. A case study in Finnish agriculture is studied assuming decreasing income expectations. If the compensating payment is paid as a fixed income subsidy, decoupling implies a greater income stability. Greater income stability may increase investments even if accompanied by a lower subsidy level. The stability of the compensating payment is a key issue in policy planning. 11

12 More work on the role of expectations for agricultural investment is needed. Appendix A Consider problem (3) subject to (1)-(2). First, it is assumed that r t is observable when the investment decision is made at time t (Model 1); in the second model (Model 2) only r t 1 is observable at time t. Let {It } denote the solution to Model 1 and let {It } denote the solution to Model 2, assuming the investment decision can be made at any time. Let y t denote the net value of investment in terms of income obtained when investing I t at t: y t (r t, I t ) = r t I t + E[ k=t+1 b k t r k ]I t I t (1 p) (8) where p denotes the investment subsidy percentage. cost, EVPI(t), can be defined for period t as A dynamic uncertainty EV P I(t) = E[y t (I t )] E[y t (E(r t r t 1 ), I t )] (9) where the first term corresponds to the expected value obtained at t when solving the wait-and-see model (Model 1) and the second term formalizes the corresponding expected value when the investment decision at time t is based on E(r t ), given the observed return r t 1 (Model 2). Let P r t denote the probability of investment at time t when the state r t is observed, and let P r t denote the corresponding probability with expected value maximization (Model 2). Assuming P r t > 0, define the unit value of investment at time t, v 1 (t), in Model 1 as v 1 (t) = E[y t(i t )] P r t I a and assuming P r t > 0 define the unit value in Model 2 as: v 2 (t) = E[y t(e(r t r t 1 ), I t )] P r ti a. If P r t = 0, let v 1 (t) = 0 and if P r t = 0 let v 2 (t) = 0. An uncertain state in terms of ROI % lowers the value of investment in two ways: by lowering the unit value of investment given the expected amount of investment and by reducing the expected investment for a given unit value. Accordingly, the dynamic value of information, EVPI(t) can be decomposed into two components: EV P I(t) = (v 1 (t) v 2 (t))p r t I a + v 2 (t)(p r t P r t)i a. (10) EVPI(t) can be negative due to overinvestment under uncertainty. E.g. in the special case of a stationary distribution, at t = 1, P r t = 1 or P r t = 0. In 12

13 this case, [Heikkinen and Pietola2006] suggests a dynamic option value O(t) for as a measure for a dynamic uncertainty cost relative to expected investment: O(t) = E[y t(i t )] P r t I a max{max t E(y t ), 0} I a, (11) where the nominator in the second term models the net value obtained when maximizing expected forward start NPV at the beginning of the time horizon. If P r t = 0, O(t) is not defined. Note that at t = 1, (11) is equivalent to the first term in (10) (if P r 1 = 1, the quantity cost due to uncertainty could be considered as zero; on the other hand, if P r t = 0 t, the first term is equivalent to the value lost due to uncertainty: EVPI(t)=E[y t (I t )]). In general, whenever overinvestment due to uncertainty takes place with P r 1 = 1, (11) could be applied for the dynamic cost of uncertainty. Appendix B Figure 5 outlines a decision tree representation for the first three time periods for the investment problem (3) subject to aggregate budget constraint (2) with I a = I. In Figure 5 y t (r t, I t ) denotes the net value of investment made at time t as formalized in (8). At time t = 1 there are five decision problems, one for each possible state r i1, i = 1,..., 5. If at time t I t = I, the expected net value of investment at t, y t (r t, I), is obtained, assuming r t is observable (Model 1). If r t is not observable at time t, the decision maker obtains y t (E(r t ) r t 1, I) when investing at time t (Model 2). After each branch with decision I t = 0, 5 new branches will emerge. Thus, at t = 2, there are 25 decision problems, three of which are explicitly depicted in Figure 5: the investment problem when r 1 = r 11 and r 2 = r 12 and the corresponding problems when r 1 = r 51 and r 2 = r 12 or r 2 = r 52. With 20 time periods there are 5 20 decision problems in the decision tree representation. This motivates applying dynamic programming (DP) to the investment problem. DP Formalization for Model 1 Problem (3) subject to (1)-(2) can be solved recursively applying Bellman equation: v(r t ) = max{(y t ) + bev(r t+1 )}, (12) I t where v(r t ) denotes the value function given state r t and where y t denotes the expected net value of investment at t, as formalized in (8). DP Formalization for Model 2 In Model 1, like in [Dixit and Pindyck1994], the value of investment at any time t is observable. In the case study summarized above, due to a high degree of 13

14 policy uncertainty, the return rate r t can be uncertain at the beginning of period t. Assuming r t is observed at the end of period t, all terms affecting the value of investment are random. A risk-neutral decision maker in this case solves the Bellman equation: v(e(r t )) = max I t {E[y t (r t, I t )] + bv[e(r t+1 )]}, (13) where y t denotes the expected net value of investment at t, as formalized in (8). According to formulation (13) the decision maker has the flexibility to make the investment decision at any time; the expected return can be determined based on the return observed previous time period. Assuming a stationary income process, however, there is no motivation for postponing investment; in this case NPV maximization is optimal. Using the terminology in [Keswani and Shackleton2006], the special case where the investment decision is made at the beginning of the time horizon corresponds to optimizing forward start net present value (NPV). References [A.-M. Heikkilä2004] A.-M. Heikkilä, L. Riepponen, A. H.: 2004, Investment in New Technology to Improve Productivity of Dairy Farms. EAAE Seminar University of Crete September 2004 CD-ROM. [Dixit and Pindyck1994] Dixit, A. and R. Pindyck: 1994, Investment under Uncertainty. Princeton: Princeton University Press. [Heikkinen1998] Heikkinen, T.: 1998, Dynamic Pricing and Power Control of A Multimedia Network. In: Conference on Information Sciences and Systems. Princeton, NJ, USA. [Heikkinen and Pietola2006] Heikkinen, T. and K. Pietola: 2006, Investment and the Dynamic Cost of Income Uncertainty: the Case of Diminishing Expectations in Agriculture. MTT Discussion Paper 5, 2006 (previous version also in proc. ERSA 2006, Volos). [Keswani and Shackleton2006] Keswani, A. and M. Shackleton: 2006, How real option disinvestment flexibility augments project NPV. European Journal of Operational Research 168, [OECD2000] OECD: 2000, Decoupling: a conceptual overview. OECD Publication. 14

15 Figure 5: Outline of a Decision Tree Representation of the Investment Problem 15

Optimization of a Real Estate Portfolio with Contingent Portfolio Programming

Optimization of a Real Estate Portfolio with Contingent Portfolio Programming Mat-2.108 Independent research projects in applied mathematics Optimization of a Real Estate Portfolio with Contingent Portfolio Programming 3 March, 2005 HELSINKI UNIVERSITY OF TECHNOLOGY System Analysis

More information

Reservation Rate, Risk and Equilibrium Credit Rationing

Reservation Rate, Risk and Equilibrium Credit Rationing Reservation Rate, Risk and Equilibrium Credit Rationing Kanak Patel Department of Land Economy University of Cambridge Magdalene College Cambridge, CB3 0AG United Kingdom e-mail: kp10005@cam.ac.uk Kirill

More information

Chapter 19 Optimal Fiscal Policy

Chapter 19 Optimal Fiscal Policy Chapter 19 Optimal Fiscal Policy We now proceed to study optimal fiscal policy. We should make clear at the outset what we mean by this. In general, fiscal policy entails the government choosing its spending

More information

A simple wealth model

A simple wealth model Quantitative Macroeconomics Raül Santaeulàlia-Llopis, MOVE-UAB and Barcelona GSE Homework 5, due Thu Nov 1 I A simple wealth model Consider the sequential problem of a household that maximizes over streams

More information

Financial risks and factors affecting them on Finnish farms

Financial risks and factors affecting them on Finnish farms 1 Financial risks and factors affecting them on Finnish farms Pyykkönen, P. 1, Yrjölä, T. 1 and Latukka, A. 2 1 Pellervo Economic Research Institute PTT, Helsinki, Finland 2 MTT Agrifood Research Finland,

More information

17 MAKING COMPLEX DECISIONS

17 MAKING COMPLEX DECISIONS 267 17 MAKING COMPLEX DECISIONS The agent s utility now depends on a sequence of decisions In the following 4 3grid environment the agent makes a decision to move (U, R, D, L) at each time step When the

More information

16 MAKING SIMPLE DECISIONS

16 MAKING SIMPLE DECISIONS 247 16 MAKING SIMPLE DECISIONS Let us associate each state S with a numeric utility U(S), which expresses the desirability of the state A nondeterministic action A will have possible outcome states Result

More information

Non-Deterministic Search

Non-Deterministic Search Non-Deterministic Search MDP s 1 Non-Deterministic Search How do you plan (search) when your actions might fail? In general case, how do you plan, when the actions have multiple possible outcomes? 2 Example:

More information

Luca Taschini. 6th Bachelier World Congress Toronto, June 25, 2010

Luca Taschini. 6th Bachelier World Congress Toronto, June 25, 2010 6th Bachelier World Congress Toronto, June 25, 2010 1 / 21 Theory of externalities: Problems & solutions Problem: The problem of air pollution (so-called negative externalities) and the associated market

More information

Econ 8602, Fall 2017 Homework 2

Econ 8602, Fall 2017 Homework 2 Econ 8602, Fall 2017 Homework 2 Due Tues Oct 3. Question 1 Consider the following model of entry. There are two firms. There are two entry scenarios in each period. With probability only one firm is able

More information

Valuing Capacity Investment Decisions: Binomial vs. Markov Models

Valuing Capacity Investment Decisions: Binomial vs. Markov Models Valuing Capacity Investment Decisions: Binomial vs. Markov Models Dalila B. M. M. Fontes 1 and Fernando A. C. C. Fontes 2 1 LIACC, Faculdade de Economia da Universidade do Porto Rua Dr. Roberto Frias,

More information

Handout 8: Introduction to Stochastic Dynamic Programming. 2 Examples of Stochastic Dynamic Programming Problems

Handout 8: Introduction to Stochastic Dynamic Programming. 2 Examples of Stochastic Dynamic Programming Problems SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 8: Introduction to Stochastic Dynamic Programming Instructor: Shiqian Ma March 10, 2014 Suggested Reading: Chapter 1 of Bertsekas,

More information

16 MAKING SIMPLE DECISIONS

16 MAKING SIMPLE DECISIONS 253 16 MAKING SIMPLE DECISIONS Let us associate each state S with a numeric utility U(S), which expresses the desirability of the state A nondeterministic action a will have possible outcome states Result(a)

More information

A VALUE-BASED APPROACH FOR COMMERCIAL AIRCRAFT CONCEPTUAL DESIGN

A VALUE-BASED APPROACH FOR COMMERCIAL AIRCRAFT CONCEPTUAL DESIGN ICAS2002 CONGRESS A VALUE-BASED APPROACH FOR COMMERCIAL AIRCRAFT CONCEPTUAL DESIGN Jacob Markish, Karen Willcox Massachusetts Institute of Technology Keywords: aircraft design, value, dynamic programming,

More information

Price volatility and return on pig fattening under different pricequantity

Price volatility and return on pig fattening under different pricequantity Price volatility and return on pig fattening under different pricequantity contract regimes JARKKO K. NIEMI, XING LIU and KYÖSTI PIETOLA MTT Economic Research Latokartanonkaari 9 FI-00790 Helsinki Finland

More information

Dynamic Replication of Non-Maturing Assets and Liabilities

Dynamic Replication of Non-Maturing Assets and Liabilities Dynamic Replication of Non-Maturing Assets and Liabilities Michael Schürle Institute for Operations Research and Computational Finance, University of St. Gallen, Bodanstr. 6, CH-9000 St. Gallen, Switzerland

More information

1 What does sustainability gap show?

1 What does sustainability gap show? Description of methods Economics Department 19 December 2018 Public Sustainability gap calculations of the Ministry of Finance - description of methods 1 What does sustainability gap show? The long-term

More information

Complex Decisions. Sequential Decision Making

Complex Decisions. Sequential Decision Making Sequential Decision Making Outline Sequential decision problems Value iteration Policy iteration POMDPs (basic concepts) Slides partially based on the Book "Reinforcement Learning: an introduction" by

More information

ROBUST OPTIMIZATION OF MULTI-PERIOD PRODUCTION PLANNING UNDER DEMAND UNCERTAINTY. A. Ben-Tal, B. Golany and M. Rozenblit

ROBUST OPTIMIZATION OF MULTI-PERIOD PRODUCTION PLANNING UNDER DEMAND UNCERTAINTY. A. Ben-Tal, B. Golany and M. Rozenblit ROBUST OPTIMIZATION OF MULTI-PERIOD PRODUCTION PLANNING UNDER DEMAND UNCERTAINTY A. Ben-Tal, B. Golany and M. Rozenblit Faculty of Industrial Engineering and Management, Technion, Haifa 32000, Israel ABSTRACT

More information

Valuation of Exit Strategy under Decaying Abandonment Value

Valuation of Exit Strategy under Decaying Abandonment Value Communications in Mathematical Finance, vol. 4, no., 05, 3-4 ISSN: 4-95X (print version), 4-968 (online) Scienpress Ltd, 05 Valuation of Exit Strategy under Decaying Abandonment Value Ming-Long Wang and

More information

Chapter 10 Inventory Theory

Chapter 10 Inventory Theory Chapter 10 Inventory Theory 10.1. (a) Find the smallest n such that g(n) 0. g(1) = 3 g(2) =2 n = 2 (b) Find the smallest n such that g(n) 0. g(1) = 1 25 1 64 g(2) = 1 4 1 25 g(3) =1 1 4 g(4) = 1 16 1

More information

MYOPIC INVENTORY POLICIES USING INDIVIDUAL CUSTOMER ARRIVAL INFORMATION

MYOPIC INVENTORY POLICIES USING INDIVIDUAL CUSTOMER ARRIVAL INFORMATION Working Paper WP no 719 November, 2007 MYOPIC INVENTORY POLICIES USING INDIVIDUAL CUSTOMER ARRIVAL INFORMATION Víctor Martínez de Albéniz 1 Alejandro Lago 1 1 Professor, Operations Management and Technology,

More information

9. Real business cycles in a two period economy

9. Real business cycles in a two period economy 9. Real business cycles in a two period economy Index: 9. Real business cycles in a two period economy... 9. Introduction... 9. The Representative Agent Two Period Production Economy... 9.. The representative

More information

Using discounted flexibility values to solve for decision costs in sequential investment policies.

Using discounted flexibility values to solve for decision costs in sequential investment policies. Using discounted flexibility values to solve for decision costs in sequential investment policies. Steinar Ekern, NHH, 5045 Bergen, Norway Mark B. Shackleton, LUMS, Lancaster, LA1 4YX, UK Sigbjørn Sødal,

More information

Revenue Management Under the Markov Chain Choice Model

Revenue Management Under the Markov Chain Choice Model Revenue Management Under the Markov Chain Choice Model Jacob B. Feldman School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14853, USA jbf232@cornell.edu Huseyin

More information

Dynamic Portfolio Choice II

Dynamic Portfolio Choice II Dynamic Portfolio Choice II Dynamic Programming Leonid Kogan MIT, Sloan 15.450, Fall 2010 c Leonid Kogan ( MIT, Sloan ) Dynamic Portfolio Choice II 15.450, Fall 2010 1 / 35 Outline 1 Introduction to Dynamic

More information

Measuring the Benefits from Futures Markets: Conceptual Issues

Measuring the Benefits from Futures Markets: Conceptual Issues International Journal of Business and Economics, 00, Vol., No., 53-58 Measuring the Benefits from Futures Markets: Conceptual Issues Donald Lien * Department of Economics, University of Texas at San Antonio,

More information

BSc (Hons) Software Engineering BSc (Hons) Computer Science with Network Security

BSc (Hons) Software Engineering BSc (Hons) Computer Science with Network Security BSc (Hons) Software Engineering BSc (Hons) Computer Science with Network Security Cohorts BCNS/ 06 / Full Time & BSE/ 06 / Full Time Resit Examinations for 2008-2009 / Semester 1 Examinations for 2008-2009

More information

2D5362 Machine Learning

2D5362 Machine Learning 2D5362 Machine Learning Reinforcement Learning MIT GALib Available at http://lancet.mit.edu/ga/ download galib245.tar.gz gunzip galib245.tar.gz tar xvf galib245.tar cd galib245 make or access my files

More information

The application of linear programming to management accounting

The application of linear programming to management accounting The application of linear programming to management accounting After studying this chapter, you should be able to: formulate the linear programming model and calculate marginal rates of substitution and

More information

How Much to Spend on Flexibility? Determining the Value of Information System Flexibility. Abstract

How Much to Spend on Flexibility? Determining the Value of Information System Flexibility. Abstract How Much to Spend on Flexibility? Determining the Value of Information System Flexibility Franz Schober Professor Emeritus, Albert Ludwigs University Freiburg, Germany Judith Gebauer Assistant Professor

More information

Online Appendix: Extensions

Online Appendix: Extensions B Online Appendix: Extensions In this online appendix we demonstrate that many important variations of the exact cost-basis LUL framework remain tractable. In particular, dual problem instances corresponding

More information

Conditional versus Unconditional Utility as Welfare Criterion: Two Examples

Conditional versus Unconditional Utility as Welfare Criterion: Two Examples Conditional versus Unconditional Utility as Welfare Criterion: Two Examples Jinill Kim, Korea University Sunghyun Kim, Sungkyunkwan University March 015 Abstract This paper provides two illustrative examples

More information

Chapter 9 Dynamic Models of Investment

Chapter 9 Dynamic Models of Investment George Alogoskoufis, Dynamic Macroeconomic Theory, 2015 Chapter 9 Dynamic Models of Investment In this chapter we present the main neoclassical model of investment, under convex adjustment costs. This

More information

B. Online Appendix. where ɛ may be arbitrarily chosen to satisfy 0 < ɛ < s 1 and s 1 is defined in (B1). This can be rewritten as

B. Online Appendix. where ɛ may be arbitrarily chosen to satisfy 0 < ɛ < s 1 and s 1 is defined in (B1). This can be rewritten as B Online Appendix B1 Constructing examples with nonmonotonic adoption policies Assume c > 0 and the utility function u(w) is increasing and approaches as w approaches 0 Suppose we have a prior distribution

More information

A Note on Competitive Investment under Uncertainty. Robert S. Pindyck. MIT-CEPR WP August 1991

A Note on Competitive Investment under Uncertainty. Robert S. Pindyck. MIT-CEPR WP August 1991 A Note on Competitive Investment under Uncertainty by Robert S. Pindyck MIT-CEPR 91-009WP August 1991 ", i i r L~ ---. C A Note on Competitive Investment under Uncertainty by Robert S. Pindyck Abstract

More information

Problems. the net marginal product of capital, MP'

Problems. the net marginal product of capital, MP' Problems 1. There are two effects of an increase in the depreciation rate. First, there is the direct effect, which implies that, given the marginal product of capital in period two, MP, the net marginal

More information

A DYNAMIC CAPITAL BUDGETING MODEL OF A PORTFOLIO OF RISKY MULTI-STAGE PROJECTS

A DYNAMIC CAPITAL BUDGETING MODEL OF A PORTFOLIO OF RISKY MULTI-STAGE PROJECTS A DYNAMIC CAPITAL BUDGETING MODEL OF A PORTFOLIO OF RISKY MULTI-STAGE PROJECTS Janne Gustafsson, Tommi Gustafsson, and Paula Jantunen Abstract This paper presents a linear programming model in which a

More information

Sequential Coalition Formation for Uncertain Environments

Sequential Coalition Formation for Uncertain Environments Sequential Coalition Formation for Uncertain Environments Hosam Hanna Computer Sciences Department GREYC - University of Caen 14032 Caen - France hanna@info.unicaen.fr Abstract In several applications,

More information

Optimal Dam Management

Optimal Dam Management Optimal Dam Management Michel De Lara et Vincent Leclère July 3, 2012 Contents 1 Problem statement 1 1.1 Dam dynamics.................................. 2 1.2 Intertemporal payoff criterion..........................

More information

Combining Real Options and game theory in incomplete markets.

Combining Real Options and game theory in incomplete markets. Combining Real Options and game theory in incomplete markets. M. R. Grasselli Mathematics and Statistics McMaster University Further Developments in Quantitative Finance Edinburgh, July 11, 2007 Successes

More information

Dynamic Programming: An overview. 1 Preliminaries: The basic principle underlying dynamic programming

Dynamic Programming: An overview. 1 Preliminaries: The basic principle underlying dynamic programming Dynamic Programming: An overview These notes summarize some key properties of the Dynamic Programming principle to optimize a function or cost that depends on an interval or stages. This plays a key role

More information

A Newsvendor Model with Initial Inventory and Two Salvage Opportunities

A Newsvendor Model with Initial Inventory and Two Salvage Opportunities A Newsvendor Model with Initial Inventory and Two Salvage Opportunities Ali CHEAITOU Euromed Management Marseille, 13288, France Christian VAN DELFT HEC School of Management, Paris (GREGHEC) Jouys-en-Josas,

More information

ECON 459 Game Theory. Lecture Notes Auctions. Luca Anderlini Spring 2017

ECON 459 Game Theory. Lecture Notes Auctions. Luca Anderlini Spring 2017 ECON 459 Game Theory Lecture Notes Auctions Luca Anderlini Spring 2017 These notes have been used and commented on before. If you can still spot any errors or have any suggestions for improvement, please

More information

Sequential Decision Making

Sequential Decision Making Sequential Decision Making Dynamic programming Christos Dimitrakakis Intelligent Autonomous Systems, IvI, University of Amsterdam, The Netherlands March 18, 2008 Introduction Some examples Dynamic programming

More information

UNIT 5 DECISION MAKING

UNIT 5 DECISION MAKING UNIT 5 DECISION MAKING This unit: UNDER UNCERTAINTY Discusses the techniques to deal with uncertainties 1 INTRODUCTION Few decisions in construction industry are made with certainty. Need to look at: The

More information

Optimal Long-Term Supply Contracts with Asymmetric Demand Information. Appendix

Optimal Long-Term Supply Contracts with Asymmetric Demand Information. Appendix Optimal Long-Term Supply Contracts with Asymmetric Demand Information Ilan Lobel Appendix Wenqiang iao {ilobel, wxiao}@stern.nyu.edu Stern School of Business, New York University Appendix A: Proofs Proof

More information

Pricing Problems under the Markov Chain Choice Model

Pricing Problems under the Markov Chain Choice Model Pricing Problems under the Markov Chain Choice Model James Dong School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14853, USA jd748@cornell.edu A. Serdar Simsek

More information

Smooth pasting as rate of return equalisation: A note

Smooth pasting as rate of return equalisation: A note mooth pasting as rate of return equalisation: A note Mark hackleton & igbjørn ødal May 2004 Abstract In this short paper we further elucidate the smooth pasting condition that is behind the optimal early

More information

The investment game in incomplete markets.

The investment game in incomplete markets. The investment game in incomplete markets. M. R. Grasselli Mathematics and Statistics McMaster University RIO 27 Buzios, October 24, 27 Successes and imitations of Real Options Real options accurately

More information

CS 188: Artificial Intelligence Spring Announcements

CS 188: Artificial Intelligence Spring Announcements CS 188: Artificial Intelligence Spring 2011 Lecture 9: MDPs 2/16/2011 Pieter Abbeel UC Berkeley Many slides over the course adapted from either Dan Klein, Stuart Russell or Andrew Moore 1 Announcements

More information

The test has 13 questions. Answer any four. All questions carry equal (25) marks.

The test has 13 questions. Answer any four. All questions carry equal (25) marks. 2014 Booklet No. TEST CODE: QEB Afternoon Questions: 4 Time: 2 hours Write your Name, Registration Number, Test Code, Question Booklet Number etc. in the appropriate places of the answer booklet. The test

More information

A Newsvendor Model with Initial Inventory and Two Salvage Opportunities

A Newsvendor Model with Initial Inventory and Two Salvage Opportunities A Newsvendor Model with Initial Inventory and Two Salvage Opportunities Ali Cheaitou Euromed Management Domaine de Luminy BP 921, 13288 Marseille Cedex 9, France Fax +33() 491 827 983 E-mail: ali.cheaitou@euromed-management.com

More information

Dynamic and Stochastic Knapsack-Type Models for Foreclosed Housing Acquisition and Redevelopment

Dynamic and Stochastic Knapsack-Type Models for Foreclosed Housing Acquisition and Redevelopment Proceedings of the 2012 International Conference on Industrial Engineering and Operations Management Istanbul, Turkey, July 3-6, 2012 Dynamic and Stochastic Knapsack-Type Models for Foreclosed Housing

More information

CSE 473: Artificial Intelligence

CSE 473: Artificial Intelligence CSE 473: Artificial Intelligence Markov Decision Processes (MDPs) Luke Zettlemoyer Many slides over the course adapted from Dan Klein, Stuart Russell or Andrew Moore 1 Announcements PS2 online now Due

More information

Department of Agricultural Economics. PhD Qualifier Examination. August 2010

Department of Agricultural Economics. PhD Qualifier Examination. August 2010 Department of Agricultural Economics PhD Qualifier Examination August 200 Instructions: The exam consists of six questions. You must answer all questions. If you need an assumption to complete a question,

More information

Towards a Theory of Policy Making

Towards a Theory of Policy Making Towards a Theory of Policy Making KLAUS MITTENZWEI, DAVID S. BULLOCK 2, KLAUS SALHOFER 3 AND JUKKA KOLA 4 Norwegian Agricultural Economics Research Institute, Oslo, Norway 2 University of Illinois at Urbana-Champaign,

More information

Lecture 2 Dynamic Equilibrium Models: Three and More (Finite) Periods

Lecture 2 Dynamic Equilibrium Models: Three and More (Finite) Periods Lecture 2 Dynamic Equilibrium Models: Three and More (Finite) Periods. Introduction In ECON 50, we discussed the structure of two-period dynamic general equilibrium models, some solution methods, and their

More information

The Optimization Process: An example of portfolio optimization

The Optimization Process: An example of portfolio optimization ISyE 6669: Deterministic Optimization The Optimization Process: An example of portfolio optimization Shabbir Ahmed Fall 2002 1 Introduction Optimization can be roughly defined as a quantitative approach

More information

CS364B: Frontiers in Mechanism Design Lecture #18: Multi-Parameter Revenue-Maximization

CS364B: Frontiers in Mechanism Design Lecture #18: Multi-Parameter Revenue-Maximization CS364B: Frontiers in Mechanism Design Lecture #18: Multi-Parameter Revenue-Maximization Tim Roughgarden March 5, 2014 1 Review of Single-Parameter Revenue Maximization With this lecture we commence the

More information

Stochastic Optimization

Stochastic Optimization Stochastic Optimization Introduction and Examples Alireza Ghaffari-Hadigheh Azarbaijan Shahid Madani University (ASMU) hadigheha@azaruniv.edu Fall 2017 Alireza Ghaffari-Hadigheh (ASMU) Stochastic Optimization

More information

Farmland Values, Government Payments, and the Overall Risk to U.S. Agriculture: A Structural Equation-Latent Variable Model

Farmland Values, Government Payments, and the Overall Risk to U.S. Agriculture: A Structural Equation-Latent Variable Model Farmland Values, Government Payments, and the Overall Risk to U.S. Agriculture: A Structural Equation-Latent Variable Model Ashok K. Mishra 1 and Cheikhna Dedah 1 Associate Professor and graduate student,

More information

THE OPTIMAL ASSET ALLOCATION PROBLEMFOR AN INVESTOR THROUGH UTILITY MAXIMIZATION

THE OPTIMAL ASSET ALLOCATION PROBLEMFOR AN INVESTOR THROUGH UTILITY MAXIMIZATION THE OPTIMAL ASSET ALLOCATION PROBLEMFOR AN INVESTOR THROUGH UTILITY MAXIMIZATION SILAS A. IHEDIOHA 1, BRIGHT O. OSU 2 1 Department of Mathematics, Plateau State University, Bokkos, P. M. B. 2012, Jos,

More information

Luca Taschini. King s College London London, November 23, 2010

Luca Taschini. King s College London London, November 23, 2010 of Pollution King s College London London, November 23, 2010 1 / 27 Theory of externalities: Problems & solutions Problem: The problem of (air) pollution and the associated market failure had long been

More information

Math Models of OR: More on Equipment Replacement

Math Models of OR: More on Equipment Replacement Math Models of OR: More on Equipment Replacement John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA December 2018 Mitchell More on Equipment Replacement 1 / 9 Equipment replacement

More information

Optimal Inventory Policies with Non-stationary Supply Disruptions and Advance Supply Information

Optimal Inventory Policies with Non-stationary Supply Disruptions and Advance Supply Information Optimal Inventory Policies with Non-stationary Supply Disruptions and Advance Supply Information Bilge Atasoy (TRANSP-OR, EPFL) with Refik Güllü (Boğaziçi University) and Tarkan Tan (TU/e) July 11, 2011

More information

{List Sales (1 Trade Discount) Total Cost} (1 Tax Rate) = 0.06K

{List Sales (1 Trade Discount) Total Cost} (1 Tax Rate) = 0.06K FINAL CA MAY 2018 ADVANCED MANAGEMENT ACCOUNTING Test Code F84 Branch: Date : 04.03.2018 (50 Marks) Note: All questions are compulsory. Question 1(4 Marks) (c) Selling Price to Yield 20% Return on Investment

More information

3.4 Copula approach for modeling default dependency. Two aspects of modeling the default times of several obligors

3.4 Copula approach for modeling default dependency. Two aspects of modeling the default times of several obligors 3.4 Copula approach for modeling default dependency Two aspects of modeling the default times of several obligors 1. Default dynamics of a single obligor. 2. Model the dependence structure of defaults

More information

Optimal Bidding Strategies in Sequential Auctions 1

Optimal Bidding Strategies in Sequential Auctions 1 Auction- Inventory Optimal Bidding Strategies in Sequential Auctions 1 Management Science and Information Systems Michael N. Katehakis, CDDA Spring 2014 Workshop & IAB Meeting May 7th and 8th, 2014 1 Joint

More information

Markowitz portfolio theory

Markowitz portfolio theory Markowitz portfolio theory Farhad Amu, Marcus Millegård February 9, 2009 1 Introduction Optimizing a portfolio is a major area in nance. The objective is to maximize the yield and simultaneously minimize

More information

Characterization of the Optimum

Characterization of the Optimum ECO 317 Economics of Uncertainty Fall Term 2009 Notes for lectures 5. Portfolio Allocation with One Riskless, One Risky Asset Characterization of the Optimum Consider a risk-averse, expected-utility-maximizing

More information

Market Liberalization, Regulatory Uncertainty, and Firm Investment

Market Liberalization, Regulatory Uncertainty, and Firm Investment University of Konstanz Department of Economics Market Liberalization, Regulatory Uncertainty, and Firm Investment Florian Baumann and Tim Friehe Working Paper Series 2011-08 http://www.wiwi.uni-konstanz.de/workingpaperseries

More information

DAKOTA FURNITURE COMPANY

DAKOTA FURNITURE COMPANY DAKOTA FURNITURE COMPANY AYMAN H. RAGAB 1. Introduction The Dakota Furniture Company (DFC) manufactures three products, namely desks, tables and chairs. To produce each of the items, three types of resources

More information

Optimizing Modular Expansions in an Industrial Setting Using Real Options

Optimizing Modular Expansions in an Industrial Setting Using Real Options Optimizing Modular Expansions in an Industrial Setting Using Real Options Abstract Matt Davison Yuri Lawryshyn Biyun Zhang The optimization of a modular expansion strategy, while extremely relevant in

More information

WHEN TO SELL THE COW? University of Szeged. Institute of Informatics, 2 Faculty of Agriculture Szeged, Árpád tér 2.

WHEN TO SELL THE COW? University of Szeged. Institute of Informatics, 2 Faculty of Agriculture Szeged, Árpád tér 2. WHEN TO SELL THE COW? BALÁZS BÁNHELYI, TIBOR CSENDES, ABIGÉL MESTER, JÓZSEFNÉ MIKÓ 2, AND JÓZSEF HORVÁTH 2 University of Szeged Institute of Informatics, 2 Faculty of Agriculture 6720 Szeged, Árpád tér

More information

Dynamic tax depreciation strategies

Dynamic tax depreciation strategies OR Spectrum (2011) 33:419 444 DOI 10.1007/s00291-010-0214-3 REGULAR ARTICLE Dynamic tax depreciation strategies Anja De Waegenaere Jacco L. Wielhouwer Published online: 22 May 2010 The Author(s) 2010.

More information

Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration

Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration Piyush Rai CS5350/6350: Machine Learning November 29, 2011 Reinforcement Learning Supervised Learning: Uses explicit supervision

More information

A Markov decision model for optimising economic production lot size under stochastic demand

A Markov decision model for optimising economic production lot size under stochastic demand Volume 26 (1) pp. 45 52 http://www.orssa.org.za ORiON IN 0529-191-X c 2010 A Markov decision model for optimising economic production lot size under stochastic demand Paul Kizito Mubiru Received: 2 October

More information

Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration

Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration Piyush Rai CS5350/6350: Machine Learning November 29, 2011 Reinforcement Learning Supervised Learning: Uses explicit supervision

More information

Reinforcement Learning. Slides based on those used in Berkeley's AI class taught by Dan Klein

Reinforcement Learning. Slides based on those used in Berkeley's AI class taught by Dan Klein Reinforcement Learning Slides based on those used in Berkeley's AI class taught by Dan Klein Reinforcement Learning Basic idea: Receive feedback in the form of rewards Agent s utility is defined by the

More information

Notes 10: Risk and Uncertainty

Notes 10: Risk and Uncertainty Economics 335 April 19, 1999 A. Introduction Notes 10: Risk and Uncertainty 1. Basic Types of Uncertainty in Agriculture a. production b. prices 2. Examples of Uncertainty in Agriculture a. crop yields

More information

91.420/543: Artificial Intelligence UMass Lowell CS Fall 2010

91.420/543: Artificial Intelligence UMass Lowell CS Fall 2010 91.420/543: Artificial Intelligence UMass Lowell CS Fall 2010 Lecture 17 & 18: Markov Decision Processes Oct 12 13, 2010 A subset of Lecture 9 slides from Dan Klein UC Berkeley Many slides over the course

More information

A portfolio approach to the optimal funding of pensions

A portfolio approach to the optimal funding of pensions A portfolio approach to the optimal funding of pensions Jayasri Dutta, Sandeep Kapur, J. Michael Orszag Faculty of Economics, University of Cambridge, Cambridge UK Department of Economics, Birkbeck College

More information

arxiv: v1 [q-fin.rm] 1 Jan 2017

arxiv: v1 [q-fin.rm] 1 Jan 2017 Net Stable Funding Ratio: Impact on Funding Value Adjustment Medya Siadat 1 and Ola Hammarlid 2 arxiv:1701.00540v1 [q-fin.rm] 1 Jan 2017 1 SEB, Stockholm, Sweden medya.siadat@seb.se 2 Swedbank, Stockholm,

More information

Research Memo: Adding Nonfarm Employment to the Mixed-Frequency VAR Model

Research Memo: Adding Nonfarm Employment to the Mixed-Frequency VAR Model Research Memo: Adding Nonfarm Employment to the Mixed-Frequency VAR Model Kenneth Beauchemin Federal Reserve Bank of Minneapolis January 2015 Abstract This memo describes a revision to the mixed-frequency

More information

1 Dynamic programming

1 Dynamic programming 1 Dynamic programming A country has just discovered a natural resource which yields an income per period R measured in terms of traded goods. The cost of exploitation is negligible. The government wants

More information

Optimal Actuarial Fairness in Pension Systems

Optimal Actuarial Fairness in Pension Systems Optimal Actuarial Fairness in Pension Systems a Note by John Hassler * and Assar Lindbeck * Institute for International Economic Studies This revision: April 2, 1996 Preliminary Abstract A rationale for

More information

Some Problems. 3. Consider the Cournot model with inverse demand p(y) = 9 y and marginal cost equal to 0.

Some Problems. 3. Consider the Cournot model with inverse demand p(y) = 9 y and marginal cost equal to 0. Econ 301 Peter Norman Some Problems 1. Suppose that Bruce leaves Sheila behind for a while and goes to a bar where Claude is having a beer for breakfast. Each must now choose between ghting the other,

More information

Optimal Dual-Sourcing: A Real Options Approach

Optimal Dual-Sourcing: A Real Options Approach Optimal Dual-Sourcing: A Real Options Approach Davison, att University of Western Ontario Lawryshyn, Yuri University of Toronto iklyukh, Volodymyr University of Toronto February 16, 217 1 1 Introduction

More information

1 Consumption and saving under uncertainty

1 Consumption and saving under uncertainty 1 Consumption and saving under uncertainty 1.1 Modelling uncertainty As in the deterministic case, we keep assuming that agents live for two periods. The novelty here is that their earnings in the second

More information

Econ 101A Final exam May 14, 2013.

Econ 101A Final exam May 14, 2013. Econ 101A Final exam May 14, 2013. Do not turn the page until instructed to. Do not forget to write Problems 1 in the first Blue Book and Problems 2, 3 and 4 in the second Blue Book. 1 Econ 101A Final

More information

Making Complex Decisions

Making Complex Decisions Ch. 17 p.1/29 Making Complex Decisions Chapter 17 Ch. 17 p.2/29 Outline Sequential decision problems Value iteration algorithm Policy iteration algorithm Ch. 17 p.3/29 A simple environment 3 +1 p=0.8 2

More information

Singular Stochastic Control Models for Optimal Dynamic Withdrawal Policies in Variable Annuities

Singular Stochastic Control Models for Optimal Dynamic Withdrawal Policies in Variable Annuities 1/ 46 Singular Stochastic Control Models for Optimal Dynamic Withdrawal Policies in Variable Annuities Yue Kuen KWOK Department of Mathematics Hong Kong University of Science and Technology * Joint work

More information

6.262: Discrete Stochastic Processes 3/2/11. Lecture 9: Markov rewards and dynamic prog.

6.262: Discrete Stochastic Processes 3/2/11. Lecture 9: Markov rewards and dynamic prog. 6.262: Discrete Stochastic Processes 3/2/11 Lecture 9: Marov rewards and dynamic prog. Outline: Review plus of eigenvalues and eigenvectors Rewards for Marov chains Expected first-passage-times Aggregate

More information

Chapter 3. Dynamic discrete games and auctions: an introduction

Chapter 3. Dynamic discrete games and auctions: an introduction Chapter 3. Dynamic discrete games and auctions: an introduction Joan Llull Structural Micro. IDEA PhD Program I. Dynamic Discrete Games with Imperfect Information A. Motivating example: firm entry and

More information

Lecture 7: Bayesian approach to MAB - Gittins index

Lecture 7: Bayesian approach to MAB - Gittins index Advanced Topics in Machine Learning and Algorithmic Game Theory Lecture 7: Bayesian approach to MAB - Gittins index Lecturer: Yishay Mansour Scribe: Mariano Schain 7.1 Introduction In the Bayesian approach

More information

How to Calculate Your Personal Safe Withdrawal Rate

How to Calculate Your Personal Safe Withdrawal Rate How to Calculate Your Personal Safe Withdrawal Rate July 6, 2010 by Lloyd Nirenberg, Ph.D Advisor Perspectives welcomes guest contributions. The views presented here do not necessarily represent those

More information

Solving dynamic portfolio choice problems by recursing on optimized portfolio weights or on the value function?

Solving dynamic portfolio choice problems by recursing on optimized portfolio weights or on the value function? DOI 0.007/s064-006-9073-z ORIGINAL PAPER Solving dynamic portfolio choice problems by recursing on optimized portfolio weights or on the value function? Jules H. van Binsbergen Michael W. Brandt Received:

More information

Optimal Stopping. Nick Hay (presentation follows Thomas Ferguson s Optimal Stopping and Applications) November 6, 2008

Optimal Stopping. Nick Hay (presentation follows Thomas Ferguson s Optimal Stopping and Applications) November 6, 2008 (presentation follows Thomas Ferguson s and Applications) November 6, 2008 1 / 35 Contents: Introduction Problems Markov Models Monotone Stopping Problems Summary 2 / 35 The Secretary problem You have

More information

Problem set Fall 2012.

Problem set Fall 2012. Problem set 1. 14.461 Fall 2012. Ivan Werning September 13, 2012 References: 1. Ljungqvist L., and Thomas J. Sargent (2000), Recursive Macroeconomic Theory, sections 17.2 for Problem 1,2. 2. Werning Ivan

More information