Semiparametric Estimation of a Finite Horizon Dynamic Discrete Choice Model with a Terminating Action 1

Transcription

1 Semiparametric Estimation of a Finite Horizon Dynamic Discrete Choice Model with a Terminating Action 1 Patrick Bajari, University of Washington and NBER Chenghuan Sean Chu, Facebook Denis Nekipelov, University of Virginia Minjung Park, University of California, Berkeley This version: April 27, 2015 Abstract We provide new identification results for finite horizon dynamic discrete choice models with a terminal action. We prove the conditions under which agents time preference is identified without relying on exclusion restrictions or observations from the final period; and the conditions under which one can identify the utility of all choices without normalizing any per-period payoffs. We then develop a computationally tractable semiparametric estimator for the considered models. Our proposed estimator is a two-step approach that involves only linear projections without simulation. Monte Carlo studies demonstrate that our estimator performs better than existing methods in terms of computational time and robustness. Keywords: Finite Horizon Optimal Stopping Problem, Time Preferences, Semiparametric Estimation, Inference with Estimated Regressors JEL Classifications: C14, C18, C50 1 We thank the co-editor and two anonymous referees for detailed and constructive suggestions. The paper has benefited from helpful comments by seminar participants at Stanford GSB, Chicago Booth, Olin Business School, Berkeley ARE, IO fest, Cirpée Conference on Industrial Organization, and Conference on Recent Contributions to Inference in Game Theoretic Models at University College London. All remaining errors are our own. Correspondence: bajari@uw.edu; seanchu@gmail.com; denis.nekipelov@gmail.com; mpark@haas.berkeley.edu. 1

2 1 Introduction Dynamic discrete choice problems have traditionally been considered to be easier to study empirically when there is a finite horizon, such as when there is a predetermined future point in time when a contract expires, a treatment program ends, or the evaluation period for workers performance pay is periodically reset. The reason is that, in such cases, the value function can be found using backward induction. The value function is trivial to compute in the final period, in which the agent s decision problem becomes static, and can be constructed iteratively for all previous periods by taking the value function one period ahead as given. However, this approach has two practical limitations. First, implementing the iterative procedure may require a prohibitively large amount of computer memory to store the value functions for all periods, which is especially problematic when the dimensionality of the state space is large and the decision horizon is long. Second, the iterative approach relies on being able to observe agents decisions in the final period. Such data are not available in many realistic settings. For example, panel data based on surveys that follow individuals over time may suffer from censoring if individuals drop out of the sample. A similar problem exists when evaluating the treatment effect of a program that has only recently been introduced. In those cases one needs to integrate out the unobserved periods under the iterative procedure, which is not only computationally burdensome but also unattractive since the resulting estimator could be sensitive to assumptions regarding primitives of the unobserved periods. In this paper, we focus on finite horizon dynamic discrete choice problems in which the agent s potential choices include a terminating action that ends the decision problem, and develop identification results and a novel multi-step estimation procedure for such problems. The estimation procedure does not rely on backward induction and provides consistent estimates of the structural parameters even if the available data do not extend to the final periods of the decision problem. The procedure is conceptually different from multi-step estimation procedures that have been proposed for infinite horizon problems, such as Bajari, Benkard and Levin (2009) and Pesendorfer and Schmidt-Dengler (2010), as we must take into account the nonstationarity of agents optimal behavior due to the presence of a final period. Our method exploits Hotz and Miller s (1993) intuition that, when there is a terminating action, one can represent the expected continuation value as a function of the choice probabilities one period ahead. We build on that intuition by proposing a nonparametric estimator of the agents expectations about their continuation values, and using the estimates to recover agents preferences within a regression framework. The resulting estimator involves only linear (or nonlinear) projections and does not even require simulation. Estimation using our method is thus computationally light, easily scalable to datarich settings and can be implemented using standard statistical software such as STATA. Our estimator 2

3 is also attractive from an economic perspective, as we do not assume that agents form accurate long-term forecasts of the state transitions. Rather, we only require that the agents have correct beliefs about the state transition process between the present and one period ahead in the future. Our approach of nonparametrically estimating agents expectations and using these estimates to recover their preferences is closely related to Ahn and Manski (1993), and can be regarded conceptually as an extension of Manski (1991, 1993, and 2000), who examines the responses of economic agents to their expectations in models with uncertainty or endogenous social effects, to dynamic settings. In addition to proposing a computationally attractive semiparametric estimator, we provide novel identification results. First, we prove identification of agents time preferences. Knowing agents time preferences is critical to understanding many important economic problems. To name just a few examples, theory predicts that consumers degree of patience has a large effect on their decisions with respect to credit card use, home mortgages, and other types of debt; likewise, the degree of future discounting used by firms has major effects on their financing and investment decisions. Unfortunately, the discount factor is not identified in dynamic discrete choice models, in general (Rust, 1994; Magnac and Thesmar, 2002). Some researchers have obtained identification using experimental data (see Frederick, Loewenstein and O Donohue (2002) for a review of the literature). In the non-experimental literature, identification has relied on identification-at-infinity arguments, observing agents final-period behavior in finite horizon problems, or exclusion restrictions on variables that affect the state transition but not the per-period utility (See, for example, Hausman, 1979; Magnac and Thesmar, 2002; Yao et al., 2012; Fang and Wang, 2014; and Chung, Steenburgh, and Sudhir, 2014). In particular, Chung, Steenburgh, and Sudhir (2014) note that, when there is a finite horizon, the period utilities are identified by the static decision problem in the final period, allowing the discount factor to be identified based on the remaining temporal variation in observed behavior. Our paper advances this argument further by proving identification even if the agents final-period decisions are unobserved. The key idea is that since the finite horizon implies that the optimal choice probabilities are nonstationary, the discount factor can be identified based on the variation over time in agents choice probabilities, even if the final period is unobserved. This result does not rely on identification-at-infinity arguments or exclusion restrictions. 2 In our second identification result, we show that, under certain conditions, the actual levels of agents payoffs from taking various actions and not just the differences in utility between them are identified alongside the discount factor. Thus, there is no need to normalize the payoff from one of the actions, which the literature has demonstrated is not innocuous in dynamic settings (Bajari, Hong, and Nekipelov, 2 The finite horizon implies that the time remaining until the final period itself plays an analogous role to an exclusion restriction. 3

4 2013). To the best of our knowledge, this is a novel identification result, as the prior literature on dynamic discrete choice models has always normalized the per-period payoff to one of the actions. The logic behind this identification result is as follows. In any period before the last one, the agent has the option to choose the terminating action in the following period if he chooses a non-terminating action in the current period, but not if he chooses the terminating action. This asymmetry implies that the normalization of the payoff from the terminating action has a disproportional effect on the discounted payoffs from the various options. Therefore, both the overall level of default probability and its temporal variation are affected by the level of the payoff from the terminating action. These sources of variation allow us to identify the level of the payoff from the terminating action. We run Monte Carlo experiments in order to demonstrate our identification results and the applicability of our estimation method as well as to compare its performance with that of existing methods. In these experiments, we simulate data based on a stylized model capturing mortgage borrowers default decisions. The model has four key elements. First, it is dynamic: borrowers are forward-looking and respond to both current and expected future shocks by adjusting their current behavior. Second, borrowers choice set is discrete in each period, borrowers choose whether to default on a loan, prepay the loan, or make just the regularly scheduled monthly mortgage payment. We consider default to be a terminating action that ends the dynamic problem, with the borrower receiving a final, one-time payoff (or rather, a utility loss) and no future utility flows. Third, the borrower s decision problem has a finite horizon, reflecting the fact that mortgages have a fixed maturity commonly 30 years. Fourth, our model is a single-agent model that abstracts from potential interactions among borrowers. Because our estimation method is designed for single-agent, finite-horizon dynamic discrete choice problems with a terminating action, these features are perfectly suited for illustrating our estimation method. The Monte Carlo exercises show that our estimation method consistently recovers agents utility function and discount factor even if available data do not extend to the final period. Furthermore, the Monte Carlos demonstrate that our estimator is significantly faster computationally than existing methods because it involves only simple projections, avoids simulation, and exploits the ability to represent the expected continuation value as a function of the choice probabilities one period ahead. The results also indicate that, for the considered setting, our estimator is more robust than the two-step estimator proposed by Bajari, Benkard, and Levin (2007; BBL henceforth). We find that BBL s structural estimates are sensitive to the choice of alternative policies used in constructing the objective function as well as the choice of initial values. Moreover, the forward-simulation required in BBL necessitates extrapolating the policy functions to unobserved time periods, which we find to affect the consistency of the estimates in 4

5 the considered setting. By contrast, our estimator does not suffer from these issues. The rest of this paper proceeds as follows. In Section 2, we present our model and discuss identification of the model primitives, including the discount factor. Section 3 discusses our estimation methodology and its implementation in STATA. In Section 4, we present results from the Monte Carlo experiments and discuss the applicability and performance of our estimator. Section 5 concludes. 2 Model In this section, we set up a single-agent, finite-horizon, dynamic discrete choice model, in which one of the agent s possible actions is terminal. For clarity of exposition, we specify our model by addressing mortgage borrowers default decisions, and we prove our identification results within the context of that model. However, our identification results hold more generally and do not rely on any specific feature of the mortgage setting. In our model, each agent is a borrower that enters a mortgage contract lasting T time periods, where T corresponds to the maturity date of the mortgage and is common across all agents. 2.1 Actions At each time period t over the life of borrower i s loan, the borrower chooses an action a i,t from the finite set A = {0, 1, 2}. 3 The possible actions in A are to default (a i,t = 0), to prepay the mortgage (a i,t = 1), or to make just the regularly scheduled monthly payment, which for simplicity we refer to as paying (a i,t = 2). We assume that there is no interaction among borrowers affecting their payoffs, so our setup is a single-agent model, rather than a game. We assume that default is a terminating action: once a borrower defaults, there is no further decision to be made and no further flow of utility starting from the next period. The assumption that one of the actions is a terminating action with no future flow of utility is a key identifying assumption, as will become clear in our discussions below. 2.2 Period Utility and State Transition Each borrower observes a vector of state variables s i,t S in each period, which is also observed by the econometrician. The support S is a product space that is a subset of k-dimensional Euclidean space. We allow the subspaces of this product space to be either continuous or discrete. The state vector s i,t could include borrower i s characteristics, the current home value, monthly payments, etc. We assume that the borrower is also characterized by a time-invariant type c i C (also observed by the econometrician) 3 Note that we use t to denote the loan s age, not calendar time. Since T is assumed to be common across all agents, loans with the same t have the same number of months remaining until maturity. 5

6 and a time-dependent vector of idiosyncratic shocks associated with each action ε i,t = (ε i,0,t, ε i,1,t, ε i,2,t ) (unobserved by the econometrician). The set C is assumed to be finite. Although certain elements of s i,t may also be time-invariant, the purpose of defining a separate type space C will become apparent in Section 3, where we discuss our utility specification with random coeffi cients. Each element of ε i,t is assumed to have a continuous support on the real line. We make the following assumption regarding the marginal distributions of the random variables. ASSUMPTION 1 (i) Conditional independence of idiosyncratic payoff shocks: (s i,t ε i,t ) c i. (ii) Conditional independence over time of idiosyncratic payoff shocks: ε i,t (ε i,t 1, a i,t 1, c i ) ε i,t (a i,t 1, c i ). (iii) Exclusion restriction (c i cannot be represented as a linear combination of the elements of s i,t ): C does not belong to any proper linear subspace of S. (iv) Markov transition of state variables: s i,t follows a reversible Markov process, conditional on a i,t. In our Monte Carlo experiments we will use a conventional specification for the distribution of the idiosyncratic shocks by assuming that components of ε i,t are mutually independent, have a type I extreme value distribution, and are i.i.d. across agents and over time. This assumption is not essential, and we prove our identification results for an arbitrary continuous distribution of random shocks that satisfies Assumption 1. However, we expect that researchers who are interested in applying our method to their empirical settings are most likely to exploit the convenience of the type I extreme value distribution, so our Monte Carlo exercises below will rely on that distributional assumption. The transition of some of the state variables may be influenced by the current action taken, a i,t. We also allow the state variables potentially to follow a higher-order Markov process by including the lagged state variables from the previous periods in the vector s i,t. This structure allows for greater realism, as certain important state variables may exhibit lag dependence. We assume that the per-period utility of the borrower is separable in the idiosyncratic shock component. We can characterize the borrower s utility as: U(a i,t, s i,t ; c i ) = u(a i,t, s i,t ; c i ) + ε i,ai,t,t, for t < T, U(a i,t, s i,t ; c i ) = u T (a i,t, s i,t ; c i ) + ε i,ai,t,t, for t = T. As specified, the per-period utility has a deterministic component, u(, ; ), which is a time-invariant function of the action, state, and the borrower s type. The payoff function in the final period T can in 6

7 general differ from that of earlier periods. For example, in the context of mortgage default, the borrower obtains full ownership of the house once the mortgage is fully paid off at maturity, which we might think of as adding a lump-sum boost to the per-period utility in the final period. 2.3 Decision Rule and Value Function We consider the borrower s problem as an optimal stopping problem, and assume that the default decision is irreversible that is, the borrower cannot restart borrowing after default. Provided that the default ( stopping ) decision is irreversible, the choice of the default option is equivalent to taking a one-time payoff without future utility flows. If the borrower pays or prepays (refinances) his mortgage, he receives the corresponding per-period payoff (which can be interpreted as a combination of the utility from consuming housing services and the disutility from making payments on the mortgage), plus the expected discounted stream of future utility. We assume that the borrower s intertemporal preferences exhibit standard exponential discounting, where the parameter β is the single-period discount factor. The borrower s decision rule D t for each period t is a mapping from the vector of payoff-relevant variables into actions, D t : S C R 3 A. We denote the borrower s decision probabilities by σ t (k s i,t, c i ) = E [ 1{D t (s i,t, c i, ε i,t ) = k} s i,t, c i ] for k A. We collect σt (k s i,t, c i ) for all k and t such that σ t (s i,t, c i ) = [σ t (k = 0 s i,t, c i ), σ t (k = 1 s i,t, c i ), σ t (k = 2 s i,t, c i )] and σ = (σ 1 (s i,1, c i ),..., σ T (s i,t, c i )), and refer to σ as the policy function. The ex ante value function at period t < T is the expected discounted utility flow of a borrower who has not defaulted prior to t: V t,σ (s i,t ; c i ) = E σ,f(s) [ T τ=t ( ) ] β τ t U(a i,τ, s i,τ ; c i ) τ 1 Π 1(a i,τ 1 > 0) s i,t, c i, τ 1=1 where f(s) represents the state transitions. The term τ 1 Π 1(a i,τ 1 > 0) reflects that once a borrower τ 1=1 defaults, there is no further flow of utility starting from the next period. The choice-specific value function, denoted by V t,σ (a i,t = k, s i,t ; c i ), is the deterministic component of the borrower s discounted utility flow conditional on choosing action k in period t: V t,σ (a i,t = k, s i,t ; c i ) = u(a i,t = k, s i,t ; c i ) + βe [V t+1,σ (s i,t+1 ; c i ) s i,t, c i, a i,t = k] for t < T, V t,σ (a i,t = k, s i,t ; c i ) = u T (a i,t = k, s i,t ; c i ) for t = T. In particular, the choice-specific value of default is equal to the per-period utility of default, i.e., V t,σ (a i,t = 0, s i,t ; c i ) = u(a i,t = 0, s i,t ; c i ) for t < T, because default is a terminating action whose future value term 7

8 E [V t+1,σ (s i,t+1 ; c i ) s i,t, c i, a i,t = 0] is zero. 2.4 Optimal Policy Functions The following theorem establishes a formal existence and uniqueness result characterizing the borrower s optimal decision. Theorem 1 Under Assumption 1 there exists a unique decision rule Dt (s i,t, c i, ε i,t ) supported on A for t = 1, 2,..., T that solves the maximization problem sup V 1,σ (s i,1 ; c i ). (D 1,D 2,...,D T ) A T Proof. Our argument uses backward induction. In the final period (at mortgage maturity) the borrower faces a static optimization problem of choosing among V T (0, s i,t ; c i ) + ε i,0,t, V T (1, s i,t ; c i ) + ε i,1,t, and V T (2, s i,t ; c i ) + ε i,2,t. The optimal decision delivers the highest payoff, yielding the decision rule D T (s i,t, c i, ε i,t ) = arg max k A {V T (k, s i,t ; c i ) + ε i,k,t }. Provided that the payoff shocks are idiosyncratic and have a continuous distribution, the optimal choice probabilities are characterized by continuous functions of (V T (k, s i,t ; c i ), k A). Knowing the optimal decision rule in period T, we can obtain the choice-specific value function in period T 1 as [ V T 1 (k, s i,t 1 ; c i ) = u(k, s i,t 1 ; c i )+βe 1{DT = k } (V T (k, s i,t ; c i ) + ε i,k,t ) ] s i,t 1, c i, a i,t 1 = k. k A Provided that the T th period optimal decision has already been derived, the optimal decision problem in T 1 becomes a static choice among three alternatives. Its solution, again, trivially exists and is (almost surely) unique because the distribution of ε i,t 1 is continuous. We iterate this procedure back to t = Semiparametric Identification In this section we demonstrate that our model is identified from objects observed in the data, namely, the choice probability of each option, conditional on the current state and the borrower s observable type (P t (a i,t = k s i,t, c i )); and the transition distribution for the state variables, characterized by the conditional cdf F (s i,t s i,t 1, c i, a i,t 1 ). The model s three structural elements are: (1) the deterministic component of the per-period payoff function, u(, ; ) 4 ; (2) the time preference parameter β; and (3) the conditional distribution of the 4 We omit discussing the identification of u T (, ; ), as it is obvious that if u T u, then u T (, ; ) is identified if and only 8

9 idiosyncratic utility shocks to the borrowers, which have the type-specific joint cdf F ε ( c). We shall argue that u(, ; ) is nonparametrically identified and that the time preference parameter β is identified, for a given distribution of the idiosyncratic payoff shocks satisfying Assumption 1. We emphasize that our identification results do not rely on the extreme value assumption for the distribution of the idiosyncratic shocks. We show the model is identified by demonstrating that there exists a unique mapping from the observable distribution of the data to the structural parameters. We start with the case in which the payoff from the default option is known, before relaxing this assumption. Theorem 2 (Identification with known default utility) Suppose that the payoff from the default option is a known function u(0, ; ), and that the distribution of idiosyncratic shocks conditional on the borrower-specific heterogeneity variables c, F ε ( c), has a full support with the density strictly positive on R 3. Also, suppose that for at least two consecutive periods t and t, σ k,t ( ; ) σ k,t ( ; ) for k A. (i) If the data distribution contains information on at least two consecutive periods and the discount factor β is known, the per-period utilities u(1, s; c) and u(2, s; c) are nonparametrically identified. Moreover, if u(, ; ) = u T (, ; ) and the observed periods include the final period T, then the discount factor is also identified. (ii) If the data distribution contains information on at least three consecutive periods (which do not necessarily include the final period T ), then both the discount factor and the per-period utility functions u(1, s; c) and u(2, s; c) are identified. This theorem, proved in Appendix 1, establishes the general result that the considered model is identified (including identification of the time preference parameter) if the payoff from default is a known function. The argument requires presence of two consecutive time periods over which there is variation in the optimal decision probability, conditional on the state variables and the type. The theoretical justification for why we would expect two such periods to exist stems from the finite horizon, which makes borrowers default decision dependent on the time remaining until the mortgage maturity date. More generally, the optimal decision rules depend on time t in finite horizon models, even conditional on the state variables, satisfying the assumption of the proposition. The key feature of our identification results is that they exploit the time variation in the optimal choice probabilities that is inherent in finite horizon models. This feature differentiates our results from those in the existing literature. Unlike the prior literature on identification of time preferences (Magnac if decisions from the final period are observed. 9

10 and Thesmar, 2002; Fang and Wang, 2014), our identification of the discount factor does not require the availability of variables that affect the state transition but are excluded from the per-period utility. Rather, the presence of a finite horizon implies that the time to maturity itself plays a role analogous to that of such variables. Nor does our identification rely on the availability of observations on the final period. 5 As part (ii) of Theorem 2 states, the discount factor is identified as long as researchers observe data from at least three consecutive periods, even if those periods do not include the final period. Our identification results also differentiate us from Hotz and Miller (1993), who do not exploit the time variation in choice probabilities for identification, and whose focus is instead on the development of computationally tractable two-step estimation methods. Our results also apply to a fundamentally different set of circumstances from those considered by Kasahara and Shimotsu (2009), who examine the infinite-horizon case and whose primary focus is on the identification of unobserved heterogeneity. For cases in which the default utility is not a known function, we would in general need to normalize it. In the empirical literature on dynamic discrete optimization problems, it has been noted (e.g., see Bajari, Hong and Nekipelov, 2013) that the choice of normalization for the per-period utility associated with one of the choices is not innocuous and can have an impact on the structural parameter estimates. In Theorem 3 below, we formally demonstrate this to be the case in the finite horizon optimal stopping problem. 6 Moreover, we show that, under stronger requirements on the data, the structural model is overidentified for a given normalization, such that one can identify the payoffs from all options without the need to normalize any payoff. These two insights can be used to explore the identification of the elements of the structural model, including the payoff from default. Theorem 3 (Identification with unknown default utility) Suppose that the distribution of idiosyncratic shocks conditional on the borrower-specific heterogeneity variables c, F ε ( c), has a full support with the density strictly positive on R 3. Also, suppose that for at least two consecutive periods t and t, σ k,t ( ; ) σ k,t ( ; ) for k A. (i) If u(, ; ) u T (, ; ) or the choices of the borrowers in the final period are not observed, one cannot identify the utilities from all choices, u(0, s; c), u(1, s; c), and u(2, s; c). However, if u(0, s; c) is normalized to a fixed function, Theorem 2 (ii) applies. In this case, the choice of normalization does not affect the recovered discount factor, but it does affect the recovered differences between the per-period payoff s from payment or prepayment and the per-period payoff from default. 5 As stated in part (i) of Theorem 2, it is easy to see that the discount factor is identified if researchers have access to observations from the final two periods (under the assumption of u(, ; ) = u T (, ; )): In the final period, one can estimate parameters of the utility function since there are no intertemporal tradeoffs to deal with. Then one can easily recover the discount factor that rationalizes the difference between choices in period T and T 1. 6 We are grateful to Günter Hitsch, who encouraged us to present the formal argument supporting this statement. 10

11 (ii) Suppose that u(, ; ) = u T (, ; ) and that the choices of the borrowers in the final period T are observed along with the choices from earlier periods. If the data distribution contains information on at least three consecutive periods, then the utilities from all choices, u(0, s; c), u(1, s; c), and u(2, s; c), are identified along with the discount factor β. This theorem, proved in Appendix 2, has a clear interpretation. In the last period, the decision of the borrower is static and thus there is no option of delayed default. As a result, the final-period decision depends only on the differences between the utilities from the payment and prepayment options and the utility from the default option. However, in any period before the last, the borrower has an option of defaulting in the following period if he pays or prepays in the current period, but not if he defaults. This asymmetry implies that the normalization has a disproportional effect on the discounted payoffs from different options. Part (i) of the theorem holds because, whereas only the utility in the current period is shifted by the normalization in the case of default (as the future discounted payoff is zero), the payoffs from payment or prepayment are additionally shifted by the amount equal to the discounted expected payoff from defaulting in the next period. However, β is invariant to the normalization because the tradeoff between current payoffs and future option values is unaffected by the normalization. The asymmetric impact of the normalization on discounted payoffs from different options implies that the level of u(0, s; c) affects both the overall level of default probability as well as its temporal variation. These sources of variation allow us to identify u(0, s; c), leading to part (ii) of the theorem. Thus, observing the final-period choices allows us to identify the actual levels of the payoffs and not just the differences in level, provided that the final-period utility function is the same as in the previous periods. To the best of our knowledge, this result is novel, and none of the existing literature has explored the conditions under which there is no need to make any normalization in dynamic discrete choice models. In many realistic settings, researchers do not have data on the behavior of the economic agents in the final period, for example, due to individuals dropping out of the sample or due to sample collection ending before agents reach the final period. Furthermore, in some settings the per-period payoff function in the final period may differ from the per-period payoff function of earlier periods. In such cases, part (i) of the theorem would apply, and identifying the model would require normalizing the utility from one of the choices. To illustrate the logic behind the identification of the discount factor and default utility u 0, in the following example, we make the type I extreme value assumption for the idiosyncratic shocks. First, to consider identification of the discount factor, we normalize the default utility and in this case, the ex ante value function takes the form 11

12 ( 2 ) V t (s i,t ; c i ) = log exp (V t (a i,t = k, s i,t ; c i )) k=0 ( = log 1 σ t (0, s i,t ; c i ) ) + u(0, s i,t ; c i ), (1) where u(0, s; c) is the normalized utility from default. We can then combine the expression for the ex ante value function with the expression for the Bellman equation for each of the non-default choices to obtain the following system of equations log ( ) σt(k,s i,t;c i) σ t(0,s i,t;c i) = u(k, s i,t ; c i ) u(0, s i,t ; c i ) [ ( +βe log 1 σ t+1(0,s i,t+1;c i) ) + u(0, s i,t+1 ; c i ) s i,t, c i, a i,t = k ], k = 1, 2. (2) This system of equations can be written for each instant in time t. Expression (2) helps to illustrate the logic behind the identification of the discount factor. Because the discount factor β is the parameter attached to the expected one-period-ahead value function conditional on choosing action k in t, β can be identified if there is variation in the one-period-ahead value function that is uncorrelated with variation in the per-period utility function u(, ; ). In finite horizon models, such variation exists with respect to t, because the per-period utility function is time invariant by assumption, whereas the optimal choice probability σ depends on t. In particular, if the data contain at least three consecutive periods, the discount factor can be expressed in closed form according to the following expression: log( σt+1(k, s;ci) σ t(0, s;c i) σ β = t+1(0, s;c i) σ ) t(k, s;c i), k = 1, 2. E [log (σ t+1 (0, s i,t+1 ; c i )) s i,t = s, c i, a i,t =k] E [log (σ t+2 (0, s i,t+2 ; c i )) s i,t+1 = s, c i, a i,t+1 =k] By the assumption of Theorems 2 and 3, the denominator of this expression is not equal to zero. Thus, the discount factor is identified. The expression clearly shows that identification of the discount factor comes from time variation in optimal choice probabilities, fixing all else equal, and does not require data from the final period. Regarding the identification of u 0, recall that for period t the choice-specific value function for k = 1, 2 can be written as V t (k, s; c) = u(k, s; c) + βe [V t+1 (s ; c) a t = k, s, c]. In each period, the choice probability of option k is determined by a pair of differences V t (1, s; c) u(0, s; c) and V t (2, s; c) u(0, s; c). In other words, these pairs are recovered from the observed choice probabilities as follows: V t (k, s; c) u(0, s; c) = log ( ) σt (k, s; c), k = 1, 2. σ t (0, s; c) The ex ante value function can be decomposed into the component that is only determined by the pairs of 12

13 differences V t (1, s; c) u(0, s; c) and V t (2, s; c) u(0, s; c) and the default utility u(0, s; c). The component that is determined by the pairs of differences can be fully recovered from the choice probabilities so that we have ( V t (s; c) = log 1 σ t (0, s; c) ) + u(0, s; c). When the final period is observed, then the pairs of differences u(1, s; c) u(0, s; c) and u(2, s; c) u(0, s; c) are identified from the choice probabilities of the last period (when the problem becomes static). Now we transform the Bellman equation to the following form: V t (k, s; c) u(0, s; c) = u(k, s; c) u(0, s; c)+βe [V t+1 (s ; c) u(0, s ; c) a t = k, s, c]+βe [u(0, s ; c) a t = k, s, c] Denote 1t (k, s; c) = V t (k, s; c) u(0, s; c), 1T (k, s; c) = u(k, s; c) u(0, s; c) and 2t (s; c) = V t (s; c) u(0, s; c). From our arguments above, 1t (k, s; c), 1T (k, s; c) and 2t (s; c) are all identified from the choice probabilities (provided that the last period is observed in the data). The discount factor is identified from the differences between the choice probabilities in the consecutive periods as discussed earlier. Now we can write the transformed Bellman equation as E [u(0, s ; c) a t = k, s, c] = β 1 ( 1t (k, s; c) 1T (k, s; c) βe [ 2t+1 (s ; c) a t = k, s, c]). This expression is a conditional moment equation that identifies of the default utility. It can be viewed as a nonlinear IV problem: the unknown function on the left-hand-side, u(0, s; c), is determined by the mean independence condition between the right-hand side terms (which are identified by the observed choice probabilities) and a set of instruments consisting of the choice k, state variables s and the timeinvariant type c. In the simple case where the true default utility is constant, i.e., u(0, s; c) = u(0; c), identification comes trivially from the above equation. In the most general setting, function u(0, s; c) will be defined under the so-called completeness condition. In the parametric settings, the completeness condition reduces to the standard rank condition. If the default utility is a linear function of the state variables, then the suffi cient condition for identification requires that the autocovariance matrix E[s t+1 s t] is non-degenerate. In other words, the process of the state transition should be suffi ciently rich. 3 Econometric Methodology Our specification of borrowers per-period payoffs is a version of the random coeffi cients model, in which the distribution of coeffi cients characterizes the borrower-level heterogeneity c. For notational simplicity, 13

14 from now on we drop the index i for borrowers, except where necessary for disambiguation. The per-period utility is parametrized by the random coeffi cients, and is defined as u(a, s; c) = u(s; θ(a, c)), where a A and θ : A C Θ, with Θ denoting the parameter space. We allow the utility to be nonparametric, and the coeffi cient vector θ(a, c) may be considered the vector of coeffi cients for the sieve representation of the per-period payoff function. Such a representation of the per-period utility gives us the flexibility to choose either a parametric or a fully nonparametric specification for the utility associated with each realization of the state variables, action, and borrower type. To estimate the model, we use a plug-in semiparametric estimator. Our proposed estimator does not rely on backward induction, and provides consistent estimates even when data on the final periods are not available. Parallel to our identification argument, we nonparametrically estimate the borrowers policy functions and use them to recover the choice-specific value functions of the borrowers. We then use the latter to recover the distribution of random coeffi cients in the utility function and the time preference parameter. Below, we first characterize the general form of the estimator corresponding to an arbitrary distribution of idiosyncratic payoff shocks satisfying Assumption 1. Then, we discuss a specific implementation with idiosyncratic payoff shocks that are distributed type I extreme value, which we expect to be the most common specification choice for applied researchers. For this case, estimation reduces to evaluating several linear projections, which does not require costly computations or simulations, and can be implemented using any standard software. Estimation for more general shock distributions may necessitate using more advanced computational tools. Step 1 First, we nonparametrically estimate the conditional choice probabilities of the borrowers. Suppose the data represent a panel of i = 1,..., J loans observed in periods τ = 1,..., T indexing calendar time. The panel is unbalanced due to defaults and issuance of new loans. We use T i,τ to denote the time elapsed from the period of mortgage origination for a loan i observed in period τ. We estimate the policy functions by evaluating the conditional distribution of observed actions for each observed loan age t. It is important to recover policy functions separately for each t in order to take into account the nonstationarity of agents optimal behavior due to the presence of a final period. Our estimation procedure uses a projection onto the orthogonal series q L (s) = (q 1 (s),..., q L (s)), where L is the number 14

15 of series terms. We consider the orthogonal representation for the choice probability as log σ t(k, s; c) σ t (0, s; c) = r l (t, k, c)q l (s), for k = 1, 2, l=1 where r l (t, k, c) are coeffi cients of the series representation. This representation will provide a uniform approximation for the decision rule if the choice probability ratio is continuous in s and the state space S is a compact set, by the Weierstrass theorem. We construct our estimator by replacing the infinite sum with a finite sum for some (large) number L. The parameters are estimated by forming a quasi-likelihood: Q(r L (t, 1, c), r L (t, 2, c); c) = 1 J + 1{a i,t = 2} T J τ=1 i=1 L r l (t, 2, c)q l (s i,t ) l=1 [ 1{T i,τ = t}1{c i = c} 1{a i,t = 1} L r l (t, 1, c)q l (s i,t ) ( ( L ) ( L ] log 1 + exp r l (t, 1, c)q l (s i,t ) + exp r l (t, 2, c)q l (s i,t ))), l=1 where r L (t, k, c) = (r 1 (t, k, c),..., r L (t, k, c)). Then, we obtain the estimator as l=1 l=1 ( r L (t, 1, c), r L (t, 2, c) ) = argmax r L Q(r L (t, 1, c), r L (t, 2, c); c). (3) The estimated policy functions correspond to the fitted values based on the estimated parameters: σ t (k, s; c) = ( L 1 + exp r l (t, 1, c)q l (s) l=1 σ t (0, s; c) = 1 σ t (1, s; c) σ t (2, s; c) ( L ) exp r l (t, k, c)q l (s) l=1 ) ( L ) for k = 1, 2, + exp r l (t, 2, c)q l (s) l=1 The number of series terms is a function of the total sample size, with L as J. As we show in Appendix 4, for our asymptotic distribution results to be valid (and, thus, for the first-stage estimation error to have no impact on the convergence rate for the estimated structural parameters), it is suffi cient to find an estimator for the choice probabilities with a uniform convergence rate of at least (J) 1/4. Such estimators exist if the choice probability is a suffi ciently smooth function of the state. Unlike other multi-step estimators such as Bajari, Benkard and Levin (2007), Pesendorfer and Schmidt- Dengler (2010), and Aguirregabiria and Mira (2002), we do not need to separately estimate transition functions for the state variables. Instead of simulating future state variables using estimated state transi- 15

16 tion functions, we propose to nonparametrically estimate borrowers expectations about the one-periodahead value function, conditional on the current state, and use these estimates to recover the structural parameters within a regression framework. We discuss this approach in detail below. Step 2 In the case of an arbitrary (known) distribution of idiosyncratic shocks, we must perform a functional inversion to recover the choice-specific and ex ante value functions from the estimated policy functions. Suppose that F ε ( c) is the distribution of payoff shocks (conditional on the type of the borrower) with density f ε ( c). Consider the functions σ 0 (z 1, z 2 ; c) = f ε (ε c)dε 0 dε 1 dε 2, ε 0 z 1+ε 1, ε 0 z 2+ε 2 σ 1 (z 1, z 2 ; c) = f ε (ε c)dε 0 dε 1 dε 2, z 1+ε 1 ε 0, z 1+ε 1 z 2+ε 2 σ 2 (z 1, z 2 ; c) = f ε (ε c)dε 0 dε 1 dε 2. z 2+ε 2 ε 0, z 2+ε 2 z 1+ε 1 (4) In Lemma 1 in Appendix 1, we show that these functions are well-defined and that if the default probability is strictly between zero and one for almost all values of the state variables, the system determining the choice probabilities is everywhere invertible for z 1, z 2 R, assuming a large support for the idiosyncratic payoff shocks. We can use the above system of equations to recover the difference between the choice-specific value of each non-default choice and the utility from default (captured by z 1 and z 2 in the expression), for each period t, each value of the state variables s, and each point in the support of borrower-specific heterogeneity c. Specifically, we reexpress the choice probabilities as functions of these differences, equate them to their empirical analogues (which were recovered in the first step), and drop the expression for action 2 (because there are only two degrees of freedom). This yields σ 0,t (s; c) = σ 0 (V t (1, s; c) u(s; θ(0, c)), V t (2, s; c) u(s; θ(0, c)); c), σ 1,t (s; c) = σ 1 (V t (1, s; c) u(s; θ(0, c)), V t (2, s; c) u(s; θ(0, c)); c). The solution to the above system can be expressed as: V t (k, s; c) = u(s; θ(0, c)) + F k ( σ 0,t (s; c), σ 1,t (s; c)), k = 1, 2, where the function F k (, ) is the solution of the inversion problem (4) for the argument z k. We can 16

17 characterize the ex ante value function as V t (s; c) = u(s; θ(0, c)) + F ( σ 0,t (s; c), σ 1,t (s; c)), where the function F (, ) is determined by the solutions F 1 (, ) and F 2 (, ) and the distribution of the idiosyncratic payoff shocks. We then substitute the obtained expressions into the Bellman equation for the borrower, obtaining a system of nonlinear simultaneous equations [ E F k (σ 0,t (s t ; c), σ 1,t (s t ; c)) u(s t ; θ(k, c)) + u(s t ; θ(0, c)) βu(s t+1 ; θ(0, c)) ] βf (σ 0,t+1 (s t+1 ; c), σ 1,t+1 (s t+1 ; c)) a t = k, s t, c = 0, k = 1, 2 (5) where the current state s t, the current action, and the borrower type serve as instruments. This system can be estimated as a standard regression by substituting in the state at time t, s t, and treating F 1 and F 2 as the outcome variables, with the right-hand side containing the parameters to be estimated (i.e., the unknown utilities parametrized by θ(a, c) and the time preference parameter β). Alternatively, we can estimate this system using a nonlinear IV methodology. For estimation, it is convenient to replace this system of conditional moment equations with a system of unconditional moment equations, using the set of instruments Z t. We define the instruments as Z t = {W m (s t ; a t ; c), m M} for some set M, which are a set of functions of the current state variables, the current action, and borrower-level heterogeneity elements c. For example, these could be a finite list of orthogonal polynomials of the state variables. Denote ɛ kt = F k (σ 0,t (s t ; c), σ 1,t (s t ; c)) u(s t ; θ(k, c)) + u(s t ; θ(0, c)) βu(s t+1 ; θ(0, c)) βf (σ 0,t+1 (s t+1 ; c), σ 1,t+1 (s t+1 ; c)), k = 1, 2. (6) Define ɛ t = (ɛ 1t, ɛ 2t ) and Z t = (Z 1t, Z 2t ). Then the system of unconditional moment equations takes the familiar form E [ɛ t Z t ] = 0. Provided that the model we analyze is smooth with respect to the nonparametrically estimated components, such as choice probabilities, we can apply the results in Chen, Linton, and van Keilegom (2003) and Mammen, Rothe, and Schienle (2012) to establish the impact of the first-stage estimation error on the standard errors of the structural estimates. In Appendices 3 and 4, we will analyze the properties of 17

18 this two-step estimator for the case of a general distribution of the idiosyncratic payoff shocks. When the idiosyncratic payoff shocks have an i.i.d. type I extreme value distribution, the elements of the derived system of conditional moment equations can be expressed in closed form: F (σ 0 (s; c), σ 1 (s; c)) = log ( σ 0 (s; c) 1). And the functions F 1 ( ) and F 2 ( ) in this case take the form: F k (σ 0 (s; c), σ 1 (s; c)) = log ( σ k (s; c) / σ 0 (s; c) ), for k = 1, 2. The choice-specific and ex ante value functions in this case can be recovered directly from the estimated choice probabilities (up to the normalization of u(s; θ(0, c))), without requiring any iterations or complicated inversions. The per-period payoffs are then recovered from the second-stage plug-in estimator, using the recovered choice-specific and ex ante value functions as inputs. Thus, when the choice utilities are linearly parametrized and we assume, in addition, that the idiosyncratic payoff shocks are distributed type I extreme value, the second stage estimates can be obtained using standard least squares, which does not even require the construction of the GMM objective function. And our entire estimation procedure reduces to performing three simple regressions. 1. We nonparametrically estimate the borrower s choice probabilities σ k,t ( ; c). In STATA, this can be done by running a multinomial logit regression separately for each c and t, using, for example, splines or orthogonal polynomials of the state variables as basis functions. We perform this estimation separately for each t in order to account for the nonstationarity of the problem. Using these choice probabilities, we recover the ex ante value function up to the normalization of u(s; θ(0, c)) for each borrower in each time period, constructing the variable F i,t = F ( σ 0,t (s i,t ; c i ), σ 1,t (s i,t ; c i )) = log ( σ 0,t (s i,t ; c i ) 1). 2. We estimate the expectation of the ex ante value one period ahead, conditional on the current state (and up to the normalization), E[F (σ 0,t+1 (s i,t+1 ; c i ), σ 1,t+1 (s i,t+1 ; c i )) a i,t, s i,t, c i ], by nonparametrically regressing the variable F i,t+1 (constructed in the previous regression) on a i,t, s i,t and c i. The estimation can be performed in STATA by running a linear regression of F i,t+1 on splines or orthogonal polynomials of the state variables, separately for each combination of a i,t, c i and t. The fitted values from this regression 18

19 form the variable F k,i,t+1 = Ê[F (σ 0,t+1(s i,t+1 ; c i ), σ 1,t+1 (s i,t+1 ; c i )) a i,t = k, s i,t, c i ] for k = 1, 2, which we construct for each borrower in each time period. 3. By combining the inversion idea in Berry (1994) with the fact that we can use as a regressor the continuation values constructed via projection, we estimate the structural parameters using OLS. First we construct the outcome variables Y k,i,t = log (ˆσ k,t (s i,t ; c i ) /ˆσ 0,t (s i,t ; c i ) ) for k = 1, 2. Assuming that the per-period utility is represented by a linear index of the state variables, we also construct a vector of regressors X k,i,t = ( s i,t (k), s i,t (0), Ê[s i,t+1 (0) s i,t, a t = k], F k,i,t+1 ). The components of this vector correspond to u(s i,t, θ(k, c i )), u(s i,t, θ(0, c i )), u(s i,t+1, θ(0, c i )) and F (σ 0,t+1 (s i,t+1 ; c i ), σ 1,t+1 (s i,t+1 ; c i )) in (6), respectively. Then, estimating the coeffi cient δ k in the regression Y k,i,t = δ kx k,i,t + ɛ k,i,t (7) yields the structural parameters of interest. In fact, by construction of Y and X, we get δ k = (θ(k, c), θ(0, c), θ(0, c) β, β). In STATA, this can be done by using the built-in command for nonlinear least squares estimation, imposing the constraint that the parameters on the third term, θ(0, c) β, are the product of the parameters on the second (θ(0, c)) and fourth terms (β). We can improve inference by simultaneously estimating the equations for k = 1 and 2 as a system. Depending on the normalization we choose for u(s; θ(0, c)), the estimation might be further simplified. For instance, if we normalize u(s; θ(0, c)) to a constant, which is a common choice in empirical work and is also what we will do in Monte Carlos in the next section, we only need to run a linear regression to estimate the structural parameters. If we normalize u(s; θ(0, c)) to be a function of time-invariant state variables that only influence the default utility but not the per-period utilities of the other options, we again only need to run a linear regression, by pooling s i,t (0) and Ê[s i,t+1 (0) s i,t, a t = k] into one regressor. Note that, in the second regression (Step 2), we recover the agents expectations nonparametrically via projection of the one-period-ahead value function onto the current-period state variables, conditional on a i,t, c, and t. Our projection approach contrasts with the commonly used alternative of imposing 19