Cross Atlantic Differences in Estimating Dynamic Training Effects

Transcription

1 Cross Atlantic Differences in Estimating Dynamic Training Effects John C. Ham, University of Maryland, National University of Singapore, IFAU, IFS, IZA and IRP Per Johannson, Uppsala University, IFAU, and IZA Robert J. LaLonde, University of Chicago and IZA Xianghong Li, York University (Toronto)

2 Goal: Contrast Continental European (Timing of Events) and North American Approaches to Estimating Training Effects in Duration Models in the Following Dimensions. In North American Analysis two parameters are of interest A. Intent-to-Treat Effects (ITT) There is a treatment group randomly assigned to training with DRit 1 for all t, and a control group DRit 0 for all t. Note that this insures that DRit 1 and i are independent. We can estimate the hazards 1 ki ( t Xi ( t), ki ), 1 exp h ( t) DR X ( t) k E', U', E, U. k it i k ki

3 B. Models of Treatment on the Treated Effects (TOT) 1 ki ( t Xi ( t), ki ), 1 exp h ( t) ACT X ( t) k E', U', E, U k it i k ki where ACTit 1 if individual i has entered training by t. Unlike in A, in B, the decision to enter training may be endogenous. Here we mean that there is a hazard for entering training 1 eti ( t Xi ( t), eti ), 1 exp h ( t) DR X ( t) et it i et eti the unobservable eti is not independent of the unobservables in the labor market transition hazards ki. One can allow for the endogeneity of ACT it by jointly analyzing entering training with the labor market histories while allowing eti to be correlated with ki.

4 European studies generally don t have access to experimental data so ITT effects are not relevant for them, and thus we focus her on TOT effects. Prototype of the American Approach Eberwein, Ham and LaLonde (ReStud 1997 EHL hereafter) and extensions of it in Ham, Li, Shore- Sheppard (2013 HLSS hereafter), although HLSS do not consider training effects. Prototype of European Approach theoretical work Abbring and van den Berg (Econometrica 2003, TOE hereafter) and many applications and extensions using European data.

5 There is a lot of overlap between the approaches, but there are differences that appear to have been confused in subsequent empirical work: (i) In the TOE approach one can only enter training if they are in the initial fresh spell of non-employment; once they exit the sample before training begins, they are no longer observed. No other spell types are observed. Some subsequent studies have allowed for analysis of future spells, but training can only take place in the initial fresh unemployment spell. (ii) In EHL they can enter training any time during their (approx.) 2 year window, regardless of their current labor market state. All spells in that 2 year window are observed.

6 Summary of Differences 1. Continuous Time vs. Discrete Time. 2. Use of Time Changing Explanatory Variables 3. Idenfication based on Exclusion restrictions or Functional Form Assumptions 4. Summary Measures of the Parameter Estimates 5. What Type of Spells are Analyzed 6. How they Estimate Effect of Time Changing Explanatory Variables.

7 1. Continuous Time vs. Discrete Time. The TOE approach uses continuous time models, which is often appropriate given that unemployment spells starts and finishes often can by estimated by the day. However there will be some gaps, e.g. Christmas holidays, and identification becomes much more difficult if there is any time aggregation on when training is entered, since the hazard jumps when training is entered within a period, and one has to integrate out the time of entering training within an interval.

8 EHL use a monthly discrete time; per se we don t see that this is an important difference and in fact will work in discrete time here. 2. Use of Time Changing Explanatory Variables This, and its implications for identification, are the biggest difference in approach, which leads to a major difference in identification strategy. Natural time changing X s are labor market conditions, e.g. monthly state unemployment rates. Time changing variables under the individual s control are not included here.

9 AB rule out time changing explanatory variables, arguing that theoretical work is usually based on time constant explanatory variable. The TOE literature has followed this and used only time constant X s, perhaps because time changing X s are much more difficult to allow for when estimating a continuous time model must break up the integral each time an X changes value (in addition to when duration dependence value changes). If applications have multiple spells, they sometimes use the value of the time changing variable at the beginning of the spell.

10 Time Changing X s In Continuous Time Suppose X it changes only once at calendar time t 0 1 0, which is duration time t 1 for this individual. ( 0.is the calendar time of the start of her spell.) Then even if there is no duration dependence, no unobserved heterogeneity and the outcome is continuously measured, the contribution to the likelihood for a completed spell without time changing X s is t 0, 0 f () t ()xp{- t e ( r ) dr i i While with time changing X s, we have t 1 f ( t) ( t ) exp{- ( r ) dr ( r ) dr}. i t t 1

11 Note we will have to break up the integral for each individual at a different duration. This of course gets much more complicated with several time changing X s that change monthly and flexible duration dependence. Existing TOE programs are not written to enable such changes. EHL allow for time changing explanatory variables state unemployment rates - that change monthly (the length of a period in their discrete time model. It s trivial to modify discrete time programs to allow for time changing X s..

12 3. Identification of Training Effects. In EHL everyone starts in a left censored employment or unemployment spell. Training is supposed to be based on random assignment (RA) but there is substantial noncompliance among the treatment and controls. However, more than double the number of treatments enter training, so the RA is informative.

13 EHL have to worry about the endogeneity of training because of correlation between unobservables in the entering training hazard and the labor market hazards, and the fact that individuals nonrandomly enter fresh employment and unemployment spells after their intial left censored spell is over see also Ham and LaLonde (Ecmta, 1996). They obtain identification from RA, time changing explanatory variables, and functional form. Use of RA for identification is based on the assumption that RA affects labor market transition rates only through participation in actual training; since the model is over identified, this is a testable assumption. Independence of unobservables and training only eliminates the problem of endogeneity/selection in the first spell.

14 To see this go back to the hazards Labor market hazards: 1 ki ( t Xi ( t), ki ), 1 exp h ( t) ACT X ( t) k E', U', E, U Hazard for entering training k it i k ki 1 eti ( t Xi ( t), eti ). 1 exp h ( t) DR X ( t) et it i et eti

15 ACT it depends on all of the time changing explanatory variables and thus variation in ACT it comes from the values of the time changing explanatory variables up to this point and the RA status of the individual. Also what labor market spell someone is in is also a function of all the previous X s. Thus conditional on current X s, there is exogenous variation in ACT and the current labor market state. AB must deal with the fact that i) training is endogenous in terms of the unobservables being correlated and ii) the data are censored as soon as an individual leaves their initial unemployment spell. Thus independence of the unobservables in the training hazard is insufficient to obtain a consistent estimate of the treatment on the treated effect.

16 AB argue that in a dynamic model there are no exclusion restrictions and criticize (implicitly) EHL for using them. This of course is true in principle but implies that one can never estimate a dynamic model with endogeneity. However, their point that RA may affect labor market hazards conditional on training status is valid. AB don t have exclusion restrictions or covariance restrictions, so idenfication has to based on functional form assumptions mixed proportional hazard models and lots of variation in their time constant explanatory variables (infinite support in an explanatory variable as in Heckman-Honore competing risk model). Allowing for more general training effects will rely even more heavily on these assumptions.

17 AB also have a no anticipation assumption that is basically equivalent to EHL s assumption that RA does not affect labor market hazards conditional on training status. One can allow for more general training effects, i.e. effects of training depend on the length of time in training or whether one completes the training. This is not a big deal in EHL but identification in the TOE studies is especially tenuous, given that identification comes from functional form assumptions here.

18 Crucial issue does AB criticism of exclusion restrictions in dynamic model make sense with time changing X s trying to include the X s from each period in the hazard for duration t creates a curse of dimensionality problem. With time changing X s the only difference between TOE and the EHL approach is that EHL assume that the RA exclusion restriction is valid while AB assume no anticipation. With time changing X s, EHL s RA exclusion restriction is testable but the TOE assumption is not testable. The big picture issue here is that all of the European results are identified off the functional form assumptions, which is a very different path from North America where identification is the primary issue in analyzing training programs, and that RA can aid in identification even in nonlinear/structural models.

19 4. How Are Parameters Summarized Single Spell Measures EHL/HLS look at expected durations (after integrating out the heterogeneity) and use delta method to get standard errors since the expected duration is a continuous function of the parameter estimates. TOE look at parameter on training dummy or hazard at the average heterogeneity (no standard error given).

20 5. What Type of Spells are Analyzed. EHL look at spells in progress at the start of the sample, as well as the subsequent fresh nonemployment and employment spells. TOE studies generally look at fresh nonemployment spells, but sometimes also look at subsequent employment/nonemployment spells.

21 Look at two examples. If you sample from the labor force, you will see two types of employment histories Inidvidual A U' spell: 5 months E spell: 7 months U spell: 24 months Inidvidual B E' spell: 3 months U spell: 8 months E spell: 25 months In general people in the middle of spell will have a hazard that is different from fresh spells (e.g. in the flow sample) unless there is no unobserved heterogeneity and no duration dependence.

22 It is very hard to write the left censored spell hazards in terms of the parameters of the fresh spell hazards, so common practice is to follow Heckman and Singer (JOE 1984) and let the left censored spell hazards have different parameters than the fresh spell hazards. (Note we almost never want to use time spent in the left censored spells before the start of the sample. For example, if one calculates average duration will oversample long spells and this leads to length biased sampling. Long spells dominate those in unemployment at a moment of time.)

23 Why do left censored spells oversample long spells - Consider people who start an unemployment spell 10 periods before the start of the sample. The only ones who are in a left censored unemployment spell at the start of the sample are those with duration greater than 10. Next consider people who start an unemployment spell 9 periods before the start of the sample. The only ones who are in a left censored unemployment spell at the start of the sample are those with duration greater than 9.

24 Thus upside of this is fresh or new spells have a clearer structural interpretation than left censored spells. But if we are not after structural parameters it is straight-forward to follow Heckman and Singer and use a different hazard function for left-censored spells including a different heterogeneity term. Moreover if we are more interested in the long term unemployed using left censored unemployment spells is important since they tend to dominate left censored spells but compromise a small portion of fresh spells.

25 TOE studies often accentuate this because they often impose extra restrictions, e.g. that individuals cannot have experienced unemployment in the last three years. To see the importance of left censored spells in all spells consider the following graphs from Ham, Li and Shore-Sheppard (2013) Insert graphs from Seam bias paper here

26 Employment Spells - Proportion of Left-Censored Spells Proportion Survey Month 25

27 Non-Employment Spells - Proportion of Left-Censored Spells Proportion Survey Month

28 Also the effects are often bigger in left censored spells at least for disadvantaged single mothers in the U.S. Fresh spells may reflect a short term problem such as a separation between husband and wife.

29 6. Changes with respect to: Age (age=35) - (age=25) 12 years of schooling (s = 12) - (s < 12) Race (Black - White) Race (Hispanic - White) Number of children less than 6 years old (one - zero) Age (age=35) - (age=25) 12 years of schooling (s = 12) - (s < 12) Race (Black - White) Race (Hispanic - White) Number of children less than 6 years old (one - zero) Table 3B. The Effects of Changes in Demographic Variables - Employment and Non-employment Spells Constant Misreporting Probabilities Left-censored non-employment spells Variable Misreporting Last-Month Probabilities Dummy Model Last Month Data Left-censored employment spells Constant Misreporting Probabilities Variable Misreporting Probabilities Last-Month Dummy Model Last Month Data 7.471** 7.475** 6.253** 3.619** 5.095** 5.099** 4.328** 3.455** (1.069) (1.098) (0.875) (0.902) (0.996) (1.012) (0.892) (1.005) ** ** ** ** 7.013** 7.013** 8.813** 8.822** (1.051) (1.126) (0.977) (0.992) (1.142) (1.111) (1.026) (1.106) 2.524* 2.584** ** ** (1.299) (1.268) (1.206) (1.299) (1.182) (1.218) (1.117) (1.142) 2.708* 2.759* 2.139* * * ** (1.408) (1.449) (1.292) (1.357) (1.534) (1.479) (1.281) (1.528) 4.225** 4.225** 4.512** 5.120** * ** ** (0.906) (0.929) (0.895) (0.954) (1.029) (0.953) (0.919) (0.967) Constant Misreporting Probabilities Fresh non-employment spells Variable Misreporting Probabilities Last-Month Dummy Model Last Month Data Constant Misreporting Probabilities Fresh employment spells Variable Misreporting Probabilities Last-Month Dummy Model Last Month Data ** 1.029** 5.479** 5.195** (0.532) (0.514) (0.847) (1.261) (0.430) (0.420) (1.052) (1.439) ** ** ** ** 2.970** 2.969** 5.984** 4.737** (0.595) (0.593) (0.961) (1.497) (0.504) (0.503) (1.196) (1.671) 1.842** 1.808** 4.581** ** (0.718) (0.730) (1.252) (1.769) (0.590) (0.591) (1.381) (1.787) (0.771) (0.773) (1.224) (1.904) (0.725) (0.708) (1.670) (2.116) 1.151** 1.155** 2.096** 3.101** (0.581) (0.575) (0.963) (1.493) (0.452) (0.452) (1.102) (1.424)

30 Summarizing Parameter Estimates When Multiple Spell Types are Used Analyzing different types of spells jointly allows one to simulate the fraction of time in employment. EHL do not provide confidence intervals for these effects but HLSS do. Here, should use CI-Bootstrap of Woutersen and Ham (2013) to get confidence intervals since the resulting function is discontinuous. CI-bootstrap to obtain 1 confidence interval for function values draw from the asymptotic distribution, keep those draws in the1 confidence interval of the parameter estimates, keep all resulting function values.

31 This is different from standard AD-Bootstrap i) draw from the asymptotic distribution, calculate the function values, trim bottom and top /2 of the resulting function values. The CI-Bootstrap produces consistent confidence intervals while the AD-bootstrap generally doesn t. An efficient version of the CI-bootstrap provides estimates asymptotically equivalent to the those estimated by the delta method when the latter is appropriate, e.g. continuous function of the estimated parameters.

32 This is a general issue in simulating discontinuous functions, as in the case of constructing counter-factuals in structural modeling. HLSS use CI-bootstrap. The few TOE studies considering multiple spells use the AD-bootstrap. Can show that AD-bootstrap underestimates the size of the confidence interval, and the differences between the CI and AD bootstraps can be substantial.

33 !!!!! Table 2: 95% Confidence Intervals For the Effect of Changing Demographic Variables on the Expected Fraction of Time Spent in Employment for Different Time Horizons Estimated Expected Fraction of Time in Employment 3-year Period 6-year Period 10-year Period Estimate AD-bootstrap [0.414,0.449] [0.421,0.459] [0.431,0.470] CI-bootstrap [0.396,0.469] [0.401,0.480] [0.409,0.491] WCI-bootstrap [0.396,0.469] [0.401,0.480] [0.409,0.489] AD/CI Ratio of Lengths of Confidence Intervals AD/WCI Change on the Expected Fraction of Time Spent in Employment With Respect to: Estimated Effect Schooling: (s = 12) - (s < 12) AD-bootstrap [0.072,0.107] [0.078,0.115] [0.081,0.119] CI-bootstrap [0.053,0.125] [0.058,0.134] [0.061,0.137] WCI-bootstrap [0.053,0.126] [0.058,0.136] [0.061,0.140] Ratio of Lengths of Confidence AD/CI Intervals AD/WCI !!!!! Estimated Effect Race: Black - White AD-bootstrap [-0.050,-0.009] [-0.055,-0.011] [-0.058,-0.012] CI-bootstrap [-0.073,0.019] [-0.080,0.019] [-0.083,0.019] WCI-bootstrap [-0.073,0.019] [-0.079,0.019] [-0.083,0.019] Ratio of Lengths of Confidence AD/CI Intervals AD/WCI

34 7. Estimating the Effects of Time Changing Variables What happens when one freezes the time changing variables at their initial value during the spell? Coefficients on time changing variables appear to be seriously biased. But there is sufficient variation between spells that one can still obtain precise estimates of other parameters in a multiple spell, multiple state, model. Insert table here.

35 Table 3A. Expected Durations and the Effects of Changes in Macro and Public Policy Variables Employment and Non-employment Spells Left-censored non-employment spells Left-censored employment spells Using Time-Changing Covariates Fixing Time-Changing Covariates at the Beginning of a Spell Difference Using Time-Changing Covariates Fixing Time-Changing Covariates at the Beginning of a Spell Difference Average Expected Duration (in months) Changes with respect to: Maximum welfare benefits increasing by 10% Carrot waiver (implemented - not implemented) Stick waiver (implemented - not implemented) Minimum wage increasing by 10% Unemployment rate increasing by 25% ** ** ** ** (0.731) (0.661) (0.608) (0.631) 0.515** 0.569** (0.143) (0.219) (0.359) (0.198) * (2.892) (4.606) (2.655) (5.535) ** (3.458) (3.632) (3.514) (5.279) (1.423) (1.129) (1.043) (0.884) 1.446** (0.580) (0.505) (0.548) (0.558)

36 Fresh non-employment spells Fresh employment spells Using Time-Changing Covariates Fixing Time-Changing Covariates at the Beginning of a Spell Difference Using Time-Changing Covariates Fixing Time-Changing Covariates at the Beginning of a Spell Difference Average Expected Duration (in months) Changes with respect to: Maximum welfare benefits increasing by 10% Carrot waiver (implemented - not implemented) Stick waiver (implemented - not implemented) Minimum wage increasing by 10% Unemployment rate increasing by 25% ** ** ** ** (0.516) (0.555) (0.495) (0.579) ** ** (0.119) (0.064) (0.070) (0.110) ** ** (1.009) (0.904) (1.074) (1.178) ** (2.100) (2.536) (1.217) (1.270) ** 0.882* (0.613) (0.672) (0.538) (0.516) 0.835** ** (0.343) (0.345) (0.277) (0.275)

37 Table 3B. The Effects of Changes in Demographic Variables - Employment and Non-employment Spells Left-censored non-employment spells Left-censored employment spells Using Time-Changing Covariates Fixing Time-Changing Covariates at the Beginning of a Spell Difference Using Time-Changing Covariates Fixing Time-Changing Covariates at the Beginning of a Spell Difference Changes with respect to: Age (age=35) - (age=25) 12 years of schooling (s = 12) - (s < 12) Race (Black - White) Race (Hispanic - White) Number of children less than 6 years old (one - zero) 7.471** 7.186** ** 5.022** (1.069) (1.018) (0.996) (0.972) ** ** ** 7.456** (1.051) (1.059) (1.142) (1.142) 2.524* 2.741** (1.299) (1.247) (1.182) (1.252) 2.708* 2.774* * * (1.408) (1.421) (1.534) (1.430) 4.225** 3.534** * ** (0.906) (0.962) (1.029) (0.957)

38 Fresh non-employment spells Fresh employment spells Using Time-Changing Covariates Fixing Time-Changing Covariates at the Beginning of a Spell Difference Using Time-Changing Covariates Fixing Time-Changing Covariates at the Beginning of a Spell Difference Age (age=35) - (age=25) 12 years of schooling (s = 12) - (s < 12) Race (Black - White) Race (Hispanic - White) Number of children less than 6 years old (one - zero) ** 0.990** (0.532) (0.521) (0.430) (0.467) ** ** ** 3.242** (0.595) (0.629) (0.504) (0.533) 1.842** 2.170** (0.718) (0.728) (0.590) (0.631) (0.771) (0.809) (0.725) (0.769) 1.151** 1.501** (0.581) (0.596) (0.452) (0.486)