Lec 1: Single Agent Dynamic Models: Nested Fixed Point Approach. K. Sudhir MGT 756: Empirical Methods in Marketing

Transcription

1 Lec 1: Single Agent Dynamic Models: Nested Fixed Point Approach K. Sudhir MGT 756: Empirical Methods in Marketing

2 RUST (1987) MODEL AND ESTIMATION APPROACH

3 A Model of Harold Zurcher Rust (1987) Empirical Model of Engine Maintenance/ Replacement Decisions Tradeoff High Cost to Replace, but low future cost to maintain No cost of replacement, but higher cost of maintenance Optimal Stopping Problem Structure of Solution Replace if miles exceed a threshold, else continue with old machine

4 Optimal Stopping Problems: Examples Search Models Marketing When to stop search and buy a new product (e.g., camera, ipod) Threshold: Has the price dropped enough? Labor Economics When to stop searching and accept a job Threshold: Have you got a high enough salary?

5 The Model

6 Notation

7 Notes

8 Value Function

9 Solving the DO Problem Bellman Equation See Similarities to how you would have done estimation for the static G&L Model if only you knew the Value Function V

10 Choice Specific Value Function Notice the ~ We solve this value function using a recursive technique called value function iteration

11 Note: Infinite Horizon vs Finite Horizon Dynamic Optimization (DO) Problems This is an Infinite Horizon Problem We assume stationarity (time homogeneity) We will solve the DO using value function iteration We also have finite horizon problems They are non-stationary DO Solved using backward induction from final period E.g., sales-force compensation problem in Chung et al.

12 Going Back to Rust Additive Separability

13 What do we mean by structural errors?

14 Parameters to be Estimated Discount Factor beta is not typically estimated. Essentially there is an identification problem

15 The Identification Problem (Magnac and Thesmar 2002)

16 What kinds of data may help identify discount factor One needs variables that does not affect current payoff, but only future payoffs Chevalier and Goolsbee (2005) Student choice of purchasing new text depends on whether a new edition is to be released soon Chung, Steenburgh and Sudhir (2009) Effort is related to not just current payoffs but also how far one is from a future bonus

17 ECONOMETRIC MODEL

18 Econometric Model Data: Buses are assumed homogeneous and independent

19 Assumptions Transition probabilities are Markov Conditional Independence Two types of conditional Independence Given x, e independent over time Conditional on x and i, x is independent of e

20 Likelihood Function for a Bus Markovian Assumption Conditional Independence Assumption

21 Log-Likelihood of the Model Likelihood separates into two components: 1. Structural parameters: maintenance cost and engine replacement cost 2. Markov Transition Probabilities Therefore you can do estimation in two independent steps (this is not two step estimation you hear of ) 1. Estimate transition probabilities 3 (relatively easy empirical frequencies) 2. Estimate the other structural parameters

22 Estimating Assumption: i.i.d. Extreme Value for e Dynamic Logit Model

23 General Notation Where

24 Estimation Method for Second Step: Nested Fixed Point Algorithm

25 Doing the inner Loop Trick: Iterate over the Expected Value Function, rather than value function This avoids having to compute value functions at values of e 0, e 1 that are additional state vars

26 Getting the Expected Value Function Expectation of the maximum for extreme value distribution

27 Expected Value Function Discrete state space: Take a weighted sum over the probabilities of being in each state Continuous state space: Discretize the state space and do interpolation to estimate EV at other state space values

28 Value Function Iteration Let t index iterations Iterate over Till the EV on LHS and RHS are within tolerance

29 Summary: Assumptions Key Assumptions Additive Separability of Error Term (AS) Markov Errors Conditional Independence (CI) i.i.d. errors (no serial correlation in errors) Extreme Value Distribution (convenient)

30 Summary: Estimation Discount Factor is Assumed Transition Probabilities estimated non-parametrically for continuous states empirical frequencies for discrete states Key Structural Parameters Using Nested Fixed Point Algorithm Outer loop estimate theta Inner Loop Solve DP Value Function Iteration for infinite horizon Backward Induction for finite horizon

31 PROGRAMMING EXERCISE

32 Programming a Simple Optimal Stopping Problem: Rust Adaptation Pay off Function Where a t is age of machine at time t i t =1 (replace), 0 (not replace) R-replacement cost 1 a t -maintenance cost of machine of age a t

33 State Space Evolution

34 Loading and setting up the Data % Load Data load rust N=rows(rust) % Convert Data into a Choice Vector Ch=zeros(N,2); Ch(:,1)=(rust(:,2)==0); Ch(:,2)=(rust(:,2)==1); %Setting up the number of states, actions, and discount factor NState=5; NAct=2; beta=0.9; %Starting Values theta=[-2;-3];

35 Setting up the Action-Specific Transition Probabilities F0= [ ; ; ; ; ]; F1= [ ; ; ; ; ]; This is completely deterministic in this problem. So we are not estimating it. If it were unknown, we would have to estimate the probabilities from the data

36 Maximizing Likelihood Function (Lik) [theta,fval,exitflag,output,grad,hessian]= fminunc('lik',theta,options);

37 Likelihood Function function f=lik(theta) global beta NState NAct differ F0 F1 N Ch rust %Static Period Utility U=zeros(N, NAct); U(:,1)=theta(1)*rust(:,1); U(:,2)=theta(2); %Expected Value Function through Value Function Iteration EV=ValFuncIter(theta); %The dynamic logit part of the model V(:,1)=exp(U(:,1)+beta*EV(rust(:,1),1)+1e-30); V(:,2)=exp(U(:,2)+beta*EV(rust(:,1),2)+1e-30); SV=sum(V,2); Prob=V./SV(:,[1 1]); % negative log Likelihood f=-sum(sum((log(prob).*ch)));

38 function EV=ValFuncIter(theta) global beta NState NAct F0 F1 %Setting up Toler=0.5; EV1=zeros(NState, NAct); EV0=zeros(NState, NAct); Value Function Iteration %State grid is discrete already, so easy, but otherwise create a grid StateGrid=seqa(1,1,5); %Filling out the static part of the utility at all combinations of states and actions U=zeros(NState, NAct); U(:,1)=theta(1)*StateGrid; U(:,2)=theta(2); %Doing the value function iterations while(abs(toler)>0.0005) EV1(:,1)= F0*log(sum(exp(U+beta*EV0),2)); EV1(:,2)= F1*log(sum(exp(U+beta*EV0),2)); Toler=max(max(abs(EV1-EV0))) EV0=EV1; end; %Returning the value function at all states and actions EV=EV1;

39 HOMEWORK EXERCISE

40 Homework (to be done by Next Friday) See posted assignment of Holger Seig: Part A Questions 1, 2 These can be easily done by following lecture Questions 3 and 4 You have to do Value function iteration to solve, which is essentially in the code I gave you Question 5: Given you the answer with the program Question 6: Answer and Program Question 6 a Simply answer questions 6b and 6c.