Alternative methods of estimating program effects in event history models

Transcription

1 Labour Economics 9 (2002) Alternative methods of estimating program effects in event history models Curtis Eberwein a, *, John C. Ham b, Robert J. LaLonde c a Center for Human Resource Research, Ohio State University, 921 Chathan Lane, Suite 100, Columbus, OH 43221, USA b Department of Economics and Center for Human Resource Research, Ohio State University, Columbus, OH, USA c Harris Graduate School of Public Studies, University of Chicago, Chicago, IL, USA and National Bureau of Economics Research, New York, NY, USA Abstract This paper first investigates the sensitivity of estimates of duration models to the specification of duration dependence. Using data from an experiment involving disadvantaged women in the U.S., we find that estimates of the parameters of hazard models are not sensitive to the way one models duration dependence as long as one uses a flexible functional form. We find that estimates of the expected duration in a state are insensitive to the way one models duration dependence if long spells are observed in the data, but that these are very sensitive to the specification when there are only relatively short spells in the data. We propose and implement alternative summary measures based on the median duration in a spell and show that these are quite robust to the specification of duration dependence. D 2002 Elsevier Science B.V. All rights reserved. JEL classification: C41; J64 Keywords: Duration models; Duration of dependence 1. Introduction Duration models are widely used in the social sciences to analyze economic and social data and to address policy questions. Labor economists have used these models to analyze the effect of unemployment compensation benefits on the lengths of spells of unemployment and part-time work (Moffitt, 1985; Ham and Rea, 1987; Meyer, 1990; McCall, 1996; * Corresponding author. Tel.: ; fax: /02/$ - see front matter D 2002 Elsevier Science B.V. All rights reserved. PII: S (02)

2 250 C. Eberwein et al. / Labour Economics 9 (2002) and the studies surveyed in Devine and Kiefer, 1991). They also have used them to analyze the length of strikes (Kennan, 1985; Gunderson and Melino, 1990), and whether out-of-the labor force and unemployment are distinct labor market states (Flinn and Heckman, 1986). Finally, many studies have used duration models to examine the impact of government training programs on the durations of subsequent employment, nonemployment, and unemployment spells (Ridder, 1986; Dolton et al., 1994; Thierry and Sollogoub, 1995; Dolton and O Neill, 1996; Ham and LaLonde, 1996; Bonnal et al., 1997; Eberwein et al., 1997; Kraus et al., 1997). A substantial amount of the foregoing research focuses on the sensitivity of estimates of the parameters of the hazard function to the specification of the duration dependence and the distribution function of the unobserved heterogeneity (Flinn and Heckman, 1982; Heckman and Singer, 1984a,b; Ridder and Verbakel, 1984; Lancaster, 1990). However, these estimates of the hazard function are often difficult to interpret and compare across studies, especially because researchers use a variety of specifications (e.g., discrete time vs. continuous time, different functional forms for the hazard function). As a result, in studies that use single-state models, researchers often focus on summary measures such as the impact of an economic or policy variable on the expected duration of a spell. In multi-state models, researchers often focus on the time spent in a state in the short and medium term, as well as in the steady state. These summary measures are appealing because they are easily interpreted by policy makers and analysts and are comparable across different modeling strategies. However, when calculating measures such as the expected duration of unemployment or the steady-state employment rate, researchers face two potential problems. First, they often must evaluate the hazard function at durations that are significantly greater than those observed in their sample to calculate an expected duration. Second, the expected duration may not exist if the hazard function goes to zero for finite durations, either because the duration dependence is explosive or because of certain draws of the unobserved heterogeneity. Our paper addresses these problems in four ways. First, we examine the sensitivity of expected duration calculations to changes in the specified form of the duration dependence. We begin by considering two widely used specifications for duration dependence: (i) a high-order polynomial in the log duration of spells (Kennan, 1985; Ham and Rea, 1987; McCall, 1996); and (ii) a step function (Moffitt, 1985; Ham and Rea, 1987; Meyer, 1990; Baker and Rea, 1998). We then consider several modifications to these two conventional specifications that may be better behaved when projecting the value of the hazard function out-of-sample. Finally, we propose an alternative specification of duration dependence that has a natural asymptote, and thus is more likely to be well behaved at long durations. Second, we examine the sensitivity of the estimated fraction of time spent in a given state to our modeling strategy. We do this for the short and medium term, as well as for the steady state. We do not expect the short and medium run fractions of time spent to be very sensitive to our specification of duration dependence or the heterogeneity distribution. By contrast, because the steady-state measures (in a two-state model) are a function of the expected durations of the respective spells, we expect these measures to be more

3 C. Eberwein et al. / Labour Economics 9 (2002) sensitive to our modeling strategy. However, we note that, although the estimated expected durations are sensitive to draws from the extreme values of the heterogeneity distribution that occur with low probability, they have less influence on our measures of the fraction of time in a state. To see this, consider the following example: suppose 99% of the population have finite employment and unemployment durations and a steady-state employment rate of The remaining 1% have a finite expected employment duration but an infinite expected unemployment duration, and therefore have a 0% steady-state employment rate. Then the expected duration in unemployment is infinite for the population as a whole, but the steady-state employment rate ( = 0.594) is essentially unaffected. Third, we address the problems encountered when using estimates of the expected duration of spells by considering estimates of the median duration of these spells. To compute the median duration, we use our estimates of the parameters of the hazard function to calculate the distribution function for the duration of a spell for each individual. From this distribution function, we calculate the median spell duration for each individual. (This measure is the point where the individual s estimated distribution function equals 0.5.) We then take the median value across the individual median durations. This measure is likely to be a more robust descriptive statistic than the mean duration since it relies much less heavily on the behavior of the hazard function at high and out-of-sample durations or on extreme values for the unobserved heterogeneity (that occur with small probability). Indeed, the median duration exists even for a defective distribution, as long as the probability a duration is finite is greater than 0.5. Fourth, we consider an estimator that allocates each individual to a specific heterogeneity group, calculates the expected duration for each individual conditional on their heterogeneity group, and then takes the median of the mean durations among individuals. The advantage of this estimator is that it allows one to calculate an expected duration for a typical individual in the case for which the mean duration does not exist (or is very large) for certain draws of the heterogeneity distribution that occur with relatively low probability. We implement this approach using the data and model for the US Job Training Partnership Act (JTPA) program described in Eberwein et al. (1997). These data contain information both on trainees and non-trainees employment and unemployment spells. In this sample, for some types of spells we observe very long durations, while for other types of spells we observe only relatively short durations. Because we expect our measures based on the latter type of spell to be much more sensitive to our modeling strategy (since they require much more out-of-sample prediction of the hazard), it is useful in this analysis to use a data set that contains both types of spells. We find that the expected durations are quite sensitive to the specification of the duration dependence when we observe only relatively short durations in our sample. Depending on the specification, the estimated expected duration for the non-trainees fresh unemployment spells ranges from about 14.2 to 63.6 months. 1 Further, this instability 1 A fresh spell is one that begins after the start of the sampling frame, while an interrupted spell is one that is in progress at the start of the sampling frame.

4 252 C. Eberwein et al. / Labour Economics 9 (2002) carries over to our estimates of the effect of being offered training on the expected duration of a fresh unemployment spell. Our estimates of this treatment effect range from a 1- month to an 8-month reduction in the expected duration of unemployment. By contrast, we find that for the interrupted unemployment spells, for which we observe long durations, our expected durations and policy effects are insensitive to our specification of the duration dependence. Our forecasts of time spent in employment in the short term (30 months) and the medium term (60 months) are insensitive to how we model the duration dependence. By contrast, the steady-state employment rates are somewhat sensitive to how we specify the duration dependence, but this is less true for the treatment effects on the steady-state employment rate. We find that for the non-trainees, the estimated steadystate employment rate ranges from 0.54 to 0.63, but the treatment effects range only from to We find that our measures of the median duration are quite stable among different specifications for all types of spells, as are the treatment effects computed from these measures. Finally, we find that the expected durations for a typical trainee and non-trainee, and the differences between them, are quite stable across specifications. These results indicate that policy evaluations based on expected durations are likely to be robust to changes in the specification of duration dependence and the distribution of the heterogeneity when the sample used for estimation contains long spells. By contrast, when the sample contains only short spells, as is the case for the fresh unemployment spells in our data set, calculating expected durations and steady-state employment rates requires significant extrapolation out-of-sample. Here we find that estimates of expected durations can be very sensitive to the specification of the hazard. Interestingly, the steady-state employment rates show somewhat less sensitivity, especially for the estimated treatment effects. Based on these results, we suggest that researchers interested in policy questions that involve the expected length of a spell use: (i) the median duration measure proposed here; (ii) truncated expected durations, for which the truncation point depends on the duration of spells observed in the data; or (iii) the expected duration for a typical individual. The implications of our results for cost benefit analysis are twofold. First, if analysts are interested in the impact of a program on the fraction of time spent in a state in the short or medium term, they can proceed in a straightforward manner. Further, the treatment effect on the steady-state employment rate appears relatively insensitive to the specification. By contrast, if analysts are interested in the expected duration of a spell, any cost benefit analysis would have to be calculated for a typical individual (as defined above) or be based on truncated expected durations. The paper proceeds as follows. In Section 2, we review two different conventional specifications of the duration dependence, consider modifications to these specifications, and propose an alternative specification. We consider estimating the fraction of time spent in a specific state as well as the expected duration of a spell. Next, we investigate median measures for duration distributions and the expected duration of a typical individual. Finally, we discuss the implications of our work for cost benefit analysis. In Section 3, we discuss the JTPA data and the econometric model. Section 4 contains the results. We conclude the paper in Section 5.

5 2. Estimation of hazard models and policy effects 2.1. Basic specification of the hazard model We consider one-state models and two-state models for simplicity, since the extensions to multi-state models are straightforward. 2 We work in discrete time. For simplicity, we use a logit specification for the hazard function, but the extension to other functional forms is again straightforward. The probability of leaving spell type j in month t of the spell is given by where k j ðt j h j Þ¼ C. Eberwein et al. / Labour Economics 9 (2002) þ expð y j ðtþþ, ð1þ y j ðtþ ¼b 1 X j ðtþþh j ðtþþh j : ð2þ In Eq. (2), X j (t) is a vector of explanatory variables and h j is an unobserved heterogeneity term which has non-zero mean and thus also plays the role of a constant term. 3 The term h j (t) in Eq. (2) captures the duration dependence in the respective hazard. One approach to modeling the duration dependence is to use the highest level polynomial in log duration that the data requires for the no heterogeneity case, i.e., continue adding higher order terms as long as the coefficients are statistically significant. 4 In this case, h j ðtþ ¼ XL k¼1 c kj logðtþ k : ð3þ Another common approach is to use a step function to model the duration dependence h j ðtþ ¼ XK k¼1 l kj 1 k ðtþ, ð4þ where 1 k (t) is an indicator function equal to 1 if t is in the interval I k, and where I 1,..., I K is a partition of the range of duration in the data. One question that arises 2 In our empirical application, we have data on employment spells and unemployment spells. 3 To streamline notation, we do not index functions or variables by an individual subscript. 4 We expect the addition of unobserved heterogeneity to lower the degree of polynomial because we would expect the duration dependence and unobserved heterogeneity to trade off, as found by Ham and Rea (1987). We allow the data to determine the number of support points in the heterogeneity distribution. We search for the smallest number of support points the data require. The recent Monte Carlo work of Baker and Melino (2000) suggests that a more conservative criterion than the likelihood ratio test may be desirable; for example, Ham et al. (1998) use the Schwartz criterion. In choosing the number of support points, it is much easier to test up than to test down, since the likelihood function is poorly behaved when one includes too many support points. Thus we follow the strategy of testing up.

6 254 C. Eberwein et al. / Labour Economics 9 (2002) is how to specify the number of intervals K. One possibility is to let K equal the number of distinct duration lengths in the sample (e.g., Meyer, 1990). This will create identification problems if one wants to estimate the heterogeneity distribution using the Heckman and Singer (1984a) approach. 5 It can also lead to the likelihood function depending on a very large number of parameters if one is estimating multiple transition rates or if there are a large number of duration lengths in the data. An alternative is to choose the intervals such that each interval contains a reasonable amount of data (e.g., five per cent of the sample, as in Ham and Rea, 1987). We follow this latter procedure since we have some long spells in our data and we also need to identify the heterogeneity distribution. Conditional on the unobserved heterogeneity, the probability that a spell of type j lasts longer than t 1 months is given by the survivor function S j ðt 1 j h j Þ ¼ j t 1 ½1 k jðs j h j ÞŠ: s¼1 ð5þ The density of a spell that lasts t months in state j is given by f j ðt j h j Þ¼k j ðt j h j ÞS j ðt 1 j h j Þ: ð6þ Finally, the probability that a spell has duration less than or equal to t is given by the distribution function F j ðt j h j Þ¼1 S j ðt j h j Þ: ð7þ 2.2. Calculating expected durations and the steady state fraction of time in a state The expected duration in state j is given by Z ED j ¼ X l H t¼1 tf j ðt j h j ÞdG j ðh j Þ, ð8þ where G j () is the distribution function for the unobserved heterogeneity term h j. Given sample estimates of the parameters, one calculates Eq. (8) for each individual and takes the sample average. 6 Note that the expected duration will not be finite if the 5 Researchers have had difficulty including dummies for each duration length and identifying the heterogeneity distribution given the Heckman and Singer (1984a) specification of heterogeneity (see Meyer, 1990; Narendranathan and Stewart, 1993). 6 One can obtain an alternative measure of the expected duration in the sample from the hazard evaluated at the mean values. We also consider this below.

7 survivor function does not tend to zero; this will occur if the duration dependence becomes explosive or a value of the heterogeneity distribution is too extreme. Alternatively, one can calculate a truncated mean Z ED j ¼ H X T* t¼1!! tf j ðt j h j Þ þ SðT* j h j ÞT* dg j ðh j Þ: Eq. (9) has the advantage that it will always be finite (for finite T*) but it still may be quite unstable if T* is large. It will also be appropriate if individuals exogenously leave the state at a certain time or age. 7 Again we calculate Eq. (9) for each individual and take the sample average. For a single-state model, policy will be based on expected durations or something analogous to them. In a multi-state model, we will also be interested in the fraction of time spent in each state. In the short-run or medium-run, this fraction can be estimated by simulation. Assuming we have two states, the steady-state fraction of time spent in State 1 is given by 8 Z SSF 1 ¼ h 1 Z h 2 C. Eberwein et al. / Labour Economics 9 (2002) ED * 1 ðh 1Þ ED * 1 ðh 1ÞþED * 2 ðh 2Þ dgðh 1,h 2 Þ: In Eq. (10), G(.,.) is the joint distribution function of h 1 and h 2 and ED j *(h j )is the expected duration in state j given h j. We calculate Eq. (10) for each individual and then take the average over all individuals. Note that this average will be finite, independent of the behavior of the duration dependence or heterogeneity, since SSF 1 will lie between zero and one for each individual. Further, extreme values of the heterogeneity distribution that occur with low probability will not have a strong influence on Eq. (10). On the other hand, the average based on Eq. (10) may be sensitive to the treatment of duration dependence Addressing potential problems in calculating policy effects Modifying the duration dependence specification Policy effects on an expected duration are found by numerically differentiating Eq. (8) or Eq. (9), i.e., by calculating the expected duration for participants in a training program and subtracting it from the expected duration for non-trainees. Alternatively, one can obtain the effect of an explanatory variable on the steadystate fraction of time spent in State 1 by numerically differentiating Eq. (10). It is ð9þ ð10þ 7 For example, in our empirical work using disadvantaged women, we assume that individuals leave the labor force at 65 years of age. 8 Here, we ignore any distinction between spells in progress at the baseline and fresh spells beginning after the baseline. The steady state fraction of time spent in State 1 is based on the expected durations for the fresh spells. See Lancaster (1990, pp ) for the relevant expression when there are more than two states; note that the argument used here also applies in that case.

8 256 C. Eberwein et al. / Labour Economics 9 (2002) clear that to carry out this type of policy evaluation in Eqs. (5) (10), one has to predict the hazard function for very high durations, and in many data sets this involves extrapolating well out of sample. Given the polynomial specification of the duration dependence (3), it is natural to simply evaluate the polynomial at the respective duration levels. However, the polynomial is chosen to fit the in-sample data and may exhibit substantial instability at out-of-sample duration levels. To avoid this problem, we modify the polynomial specification by freezing the value of h(t) for out-of-sample durations at the last duration that we observe in the data. 9 If one chooses the time dummy specification (4), a natural approach in calculating the expected duration is to let l K (the coefficient of the largest duration dummy) describe the duration dependence for all out-of-sample durations. However, this leads to the problem that a parameter which is essentially a nuisance parameter in estimation (and which may have quite a large standard error) is playing an important role in policy evaluation. To avoid this problem, we consider using an average of l K 4,..., l K to describe the out-of-sample contribution of h(t). We also consider using the average of the last five terms of h(t) for the log duration specification. An alternative to specifications (3) and (4) above is h j ðtþ ¼ XL k¼1 c jk ½ðt 1Þ=tŠ k : ð11þ Notice that the terms (t 1)/t are 0 in the initial month and tend to 1 as t tends to infinity. Thus, as t tends to infinity, we have lim h jðtþ ¼ XL t!l k¼1 c jk, ð12þ which will be finite. This specification allows us to fit the data within the sample by choosing a high enough order for the polynomial (as with the polynomial in log duration), but has the advantage that it does not tend to plus or minus infinity for large out-of-sample durations. It also has the potential advantage of allowing some extrapolation of the trend in duration dependence out-of-sample without forcing the hazards to either 0 or Calculating summary measures for a typical individual Expected duration calculations may also run into trouble if the estimated heterogeneity distribution contains extreme values of h, even if these occur with very low probability. For example, the expected duration, conditional on certain realizations of the unobserved heterogeneity, may not exist. To address this issue, we calculate the expected duration of a typical individual. To do this, we use a 9 A similar approach is used by Eberwein et al. (1997).

9 C. Eberwein et al. / Labour Economics 9 (2002) random number generator and the estimated probability distribution for the unobserved heterogeneity to allocate each individual to one of the support points h j. Next, we calculate the expected durations (8) or (9) conditional on the support point for each individual in the sample. We then take the median of the expected durations across the individuals in the sample. 10 The steady state fraction of time may also be somewhat sensitive to extreme values of the heterogeneity distribution. However, since estimates are bounded between zero and one even for extreme values of the heterogeneity distribution, we would not expect the problem to be as severe as with expected durations. To the extent that this is a problem, one can also use this approach to calculate a steadystate employment rate for each individual and then take the mean or the median of these employment rates Calculating median duration When the expected duration does not exist because the duration dependence is explosive or the heterogeneity distribution contains extreme values, Heckman and Singer (1984b) suggest using an estimator such as the median duration. Moreover, the median should also be much more robust to short sampling periods for non-defective distributions, since it depends much less heavily on out-of-sample values of the hazard function. Therefore, for each individual we calculate the median duration given the estimates of the hazard function. For a continuous distribution F(t), the median, t med, is defined such that F(t med ) = 0.5. Because we specify a discrete hazard model, we calculate an interpolated median for each individual, as suggested by Stuart and Ord (1987, vol. 1, p. 47). For example, suppose F (8 months) = 0.40 and F (9 months) = 0.55, then we calculate t med = (see Appendix A for more details.) We then arrange the individual median durations in increasing order, and take the median of these values as the median of the median durations. We then compare the difference in the median of the median duration for the treatments and controls Appropriateness of alternative policy measures for cost benefit analysis Researchers often want to use parameter estimates to conduct cost benefit analysis. Specifically, researchers want to measure the benefits for each individual in the sample and compare that to the total program costs for these individuals. For example, assume that a policy is intended to increase employment duration. If the average expected duration of employment for each individual does not exist or is unstable, an alternative approach is necessary. Unfortunately, the median of the individual medians clearly does not lend itself to calculating the average or total benefits of the program. Instead, two approaches should be investigated. First, one could calculate expected benefits for a typical individual, as discussed above. 10 This assumes that the mean (8) exists for at least 50% of the sample. 11 One could also take the sample mean of the individual median durations.

10 258 C. Eberwein et al. / Labour Economics 9 (2002) Alternatively, one could calculate the effect of the program on a truncated expected duration of employment where the truncation point corresponds to the length of employment durations observed in the data. Next consider a program that affects both employment and unemployment durations, and suppose that one wants to estimate the effect of the program on the steady state employment rate. As noted above, we would not expect the average of the steady state employment rates across individuals to be particularly sensitive to the specification of the duration dependence or outliers in the heterogeneity distribution; indeed we find this to be true below. To the extent that the steady state employment rate is sensitive, one could consider the effect of the policy on the expected employment rate of a typical individual. Alternatively, one could calculate the effect on the medium run employment rates, which will be much less sensitive to the duration dependence and heterogeneity specifications. 3. Training data and econometric model 3.1. Background We use the data and model from Eberwein et al. (1997), hereafter EHL. This paper studies the impact on disadvantaged women s unemployment and employ- ment spells of being offered classroom training under the auspices of the US Job Training Partnership Act (JTPA). 12 This effect is of substantial interest for two reasons. First, JTPA is a major program, involving expenditures of over US$4 billion a year. Second, the data we use are from an experiment in which some program applicants recommended by JTPA counselors to receive classroom training were randomly assigned to a control group, and as a result were precluded from receiving JTPA services for 18 months. The JTPA administrators assigned the remaining applicants to a treatment group, the members of which were allowed to participate in training. 13 The data analyzed in this paper come from the National JTPA Study (NJS) (Bloom et al., 1993). The study followed members of both experimental groups for 18 months after the baseline (when individuals were randomly assigned) and followed a smaller fraction for 30 months after the baseline. For our purposes, the basic experimental result from the NJS was that, 18 months after the baseline, adult women in the treatment group had an employment rate that was a statisti- 12 In our earlier paper, we also estimate the impact of actually participating in JTPA classroom training on the duration of employment and unemployment. Because we must use simultaneous equation techniques in duration models to estimate this latter impact, for simplicity and clarity we focus here on the effect of being offered training. The effect of being offered training will differ from the effect of receiving training because some individuals in the treatment group do not show up at the training center, while some individuals in the control group may obtain similar training through local non- JTPA organizations. 13 A third party supervised the random assignment of individuals into treatment and control groups.

11 cally significant 4 percentage points higher than the employment rate of the control group. In this study, we focus on the employment histories of adult women assigned to service sequences that emphasized classroom training. We limit our attention to women who were unemployed during the entire month leading up to random assignment. 14 Following EHL, we define unemployment to mean either being unemployed or being out of the labor force, as the data do not allow us to distinguish these states. Further, we estimate the econometric model for only 18 months of data, since this allows us to potentially choose between specifications of the duration dependence by using an out-of-sample goodness of fit test for employment rates at month 30. While we obtain the same general results as EHL, readers specifically interested in the actual policy effect are referred to that paper since it is based on considerably more data Econometric specification C. Eberwein et al. / Labour Economics 9 (2002) As in EHL, we ask whether the increase in employment rates for the treatments was achieved through: (i) shorter unemployment spells for the treatments; (ii) longer employment spells for the treatments; or (iii) some combination of both. In what follows, we describe spells in progress at the baseline as interrupted unemployment spells. A majority of women move from this interrupted unemployment spell to a fresh employment spell during the first 18 months after the baseline. Further, during the sample period some of these women in a fresh employment spell became unemployed and began a fresh unemployment spell. Therefore, we have three types of spells to analyze jointly. We use the logit specification of the hazard function given by Eq. (1), where the argument of the function now takes the form y j ðtþ ¼b 1j xðtþþb 2j TR þ h j ðt þ sþþh j, ð13þ for j = iu (interrupted unemployment spell), u (fresh unemployment spell), and e (fresh employment spell). In Eq. (13), x(t) is a vector of explanatory variables (some of which are time varying) and TR is a dummy variable equaling one if the individual is in the treatment group which is offered JTPA training and zero otherwise. Finally, as before, h j is an unobserved heterogeneity term and h j (t + s) captures the duration dependence in the respective hazard. For both an interrupted and fresh spell, t is the time spent in the spell since the baseline. Further, for an interrupted spell s is the number of months spent in the spell before the baseline, while 14 In the full sample from the JTPA study, 15% of the adult women assigned to classroom training had been employed during the baseline month. For simplicity, we have deleted these women. Because of the NJS s experimental design, our decision to delete such women here does not affect the integrity of the random assignment.

12 260 C. Eberwein et al. / Labour Economics 9 (2002) s = 0 for a fresh spell. 15 To model duration dependence, we consider the specifications discussed in Section 2 above. 16 Rather than discussing the general likelihood function, we present the contribution to the likelihood for the labor market history in Fig. 1. There are two states: unemployment and employment. The individual is in the midst of an unemployment spell at the beginning of the sample period. (Below we refer to such a spell as an interrupted unemployment spell.) She spends r iu additional months in this spell before beginning an employment spell. This employment spell lasts t e1 months, and is followed by an unemployment spell lasting t u1 months. Finally, the individual is employed for t e2 months when the sampling period ends. Her contribution to the likelihood is given by Z L ¼ f iu ðr iu j h iu Þf e ðt e1 j h e Þf u ðt u1 j h u ÞS e ðt e2 j h e ÞdGðh iu,h e,h u Þ, ð14þ H where dg() is the distribution function for the heterogeneity. (Examples of the likelihood for other specific employment histories are given in EHL.) We specify that the heterogeneity distribution takes the form of a one-factor loading model 17 h iu ¼ c i h þ a iu, h u ¼ c 2 h þ a u, h e ¼ c 3 h: Fig. 1. Sample Employment History. We follow Heckman and Singer (1984a) and assume that h is drawn from a discrete distribution with J support points h 1,..., h J 1, h J and associated pro- 15 EHL and Ham and LaLonde (1996) found that it was important for identification to control for the length of time that an individual had been in an interrupted spell since it began (as opposed to the length of time she had been in the spell since the baseline). We do not condition explicitly on s when writing the hazard. 16 As in EHL, we found that a fourth order polynomial was appropriate using a likelihood ratio test. 17 We leave the mean and variance of h unrestricted and instead impose the normalization c 3 =1.

13 Table 1 Testing for differences between the means and variances of controls and treatments baseline characteristics Baseline characteristic Obs Sample at 18 months Obs Sample at 30 months Means (T) T-stat (C) F-stat Means (T) T-stat (C) F-stat Used as explanatory variables Age Highest grade completed * High school dropout Non-Hispanic black * 1.14 Hispanic Never married * * 1.14 Married living with spouse Kids under Other baseline variables Never worked for pay Weeks worked last 12 months Never received AFDC payments Receiving AFDC at baseline * 1.03 Receiving food stamps * On AFDC for 2 or more years * Baseline unemployment duration Assignment month (Jan. 88 = 1) Fraction employed * The column heading Obs indicates the number of non-missing values for the given baseline characteristic; the heading t-stat indicates the value of the t-statistic for testing the null hypothesis that the controls and treatments have equal mean baseline characteristics; the column heading F-stat indicates the value of the F-statistic for testing the null hypothesis that the controls and treatments characteristics have the same variance. An * indicates that the P-value is less than C. Eberwein et al. / Labour Economics 9 (2002)

14 262 Table 2 The determinants of transitions from employment and unemployment spells by economically disadvantaged adult women in JTPA Variable Specification (1) (2) (3) (4) (5) (6) (a) Interrupted unemployment spells Treatment = (0.071) (0.071) (0.071) (0.074) (0.073) (0.073) Years of schooling (0.024) (0.024) (0.024) (0.025) (0.025) (0.025) H.S. dropout (0.078) (0.077) (0.078) (0.083) (0.080) (0.081) Kids under (0.073) (0.073) (0.074) (0.079) (0.077) (0.078) Single (0.077) (0.077) (0.077) (0.084) (0.082) (0.083) Married (0.082) (0.082) (0.082) (0.087) (0.085) 0.006(0.085) Black (0.072) (0.072) (0.072) (0.077) (0.076) (0.076) Hispanic (0.094) (0.094) (0.094) (0.100) (0.096) (0.097) Age (0.029) (0.029) (0.029) (0.030) (0.029) (0.030) Age squared/ (0.039) (0.038) (0.039) (0.040) (0.039) (0.040) Local unemployment rate (0.016) (0.016) (0.016) (0.017) (0.017) (0.017) Heterogeneity No No No Yes Yes Yes Duration specification func. log(t) (t 1)/t step func. log(t) (t 1)/t step func. Log (L) a C. Eberwein et al. / Labour Economics 9 (2002) (b) Fresh unemployment spells Treatment = (0.151) (0.151) (0.151) (0.208) (0.209) (0.210) Years of schooling (0.048) (0.048) (0.048) (0.072) (0.072) (0.072) H.S. dropout (0.169) (0.169) (0.169) (0.238) (0.246) (0.247) Kids under (0.159) (0.159) (0.160) (0.212) (0.216) (0.220) Single (0.178) (0.178) (0.178) (0.244) (0.248) (0.252) Married (0.185) (0.185) (0.185) (0.250) (0.253) (0.260)

15 Black (0.164) (0.164) (0.164) (0.215) (0.219) (0.220) Hispanic (0.233) (0.233) (0.234) (0.323) (0.326) (0.330) Age (0.068) (0.069) (0.069) (0.097) (0.098) (0.098) Age squared/ (0.092) (0.092) (0.092) (0.130) (0.130) (0.131) Local unemployment rate (0.040) (0.040) (0.040) (0.050) (0.051) (0.051) Heterogeneity No No No Yes Yes Yes Duration specification func. log(t) (t 1)/t step func. log(t) (t 1)/t step func. Log L (c) Fresh employment spells Treatment = (0.089) (0.089) (0.089) (0.093) (0.093) (0.093) Years of schooling (0.032) (0.032) (0.032) (0.034) (0.033) (0.034) H.S. dropout (0.111) (0.111) (0.111) (0.114) (0.114) (0.114) Kids under (0.099) (0.099) (0.099) (0.098) (0.097) (0.098) Single (0.106) (0.106) (0.106) (0.110) (0.109) (0.110) Married (0.116) (0.116) (0.116) (0.120) (0.119) (0.120) Black (0.098) (0.098) (0.098) (0.104) (0.103) (0.104) Hispanic (0.136) (0.136) (0.136) (0.145) (0.141) (0.146) Age (0.041) (0.041) (0.041) (0.041) (0.040) (0.041) Age squared/ (0.056) (0.056) (0.056) (0.055) (0.054) (0.055) Local unemployment rate (0.024) (0.024) (0.024) (0.025) (0.024) (0.025) Heterogeneity No No No Yes Yes Yes Duration specification func. log(t) (t 1)/t step func. log(t) (t 1)/t step func. Log L The functional form for hazards of type j is given by k(t j h j ) = 1/(1 + exp( b j X j h(t j ) h j )). The unobserved heterogeneity is drawn from a three-point distribution and is correlated among spells as described in the text. The functional forms for the duration dependence are (i) a fourth order polynomial in log duration; (ii) a fourth order polynomial in (t j 1/t j ; and (iii) a step function. a For the heterogeneity cases, likelihood values include the contributions of all three types of spells. C. Eberwein et al. / Labour Economics 9 (2002)

16 264 C. Eberwein et al. / Labour Economics 9 (2002) babilities P 1,..., P J 1. We estimate the parameters of the hazard functions, the support points and the associated probabilities by maximizing the likelihood function Alternative estimates of projected program effects 4.1. Estimates of the hazard functions We estimate the parameters of the hazard function using the first 18 months of JTPA data after the baseline. Table 1 provides the means for the treatments and controls, as well as normal statistics for the differences between the mean values of the two experimental groups characteristics. As expected with experimental data, only a small number of these differences are significantly different from zero, and this number is in line with sampling variation. As described in Section 2, we consider three specifications of h j (t) in estimation: (i) a polynomial in log duration; (ii) a polynomial in (t 1)/t; and (iii) a step function. We estimate the model using the factor structure for the heterogeneity distribution presented in Section 3. For comparison, we also esti-mate, for each duration specification, the parameters of the corresponding model in which we assume there is no unobserved heterogeneity. We present the parameter estimates for the three hazard functions in Table As shown by Table 2, the estimated coefficients associated with the explanatory variables are strikingly similar across the different specifications of the duration dependence. This finding, along with results in the literature, indicates that when estimating the parameters of the hazard function, it is important to choose a flexible specification of duration dependence, but it does not matter which flexible form one chooses for h(t). Further, this finding does not depend on whether we allow for unobserved heterogeneity, though some coefficients for the fresh unemployment hazard are affected by allowing for heterogeneity Projecting the hazard functions out of sample We now compare the estimated employment rates and program effects that are implied by our three specifications of duration dependence. In each specification, we have accounted 18 The number of mass points J is determined by the data. Choosing the number of support points involves a number of complications in terms of establishing the asymptotic distribution for the parameter estimates. First, one faces the issue that J is likely to increase with sample size, thereby creating an incidental parameter problem. Second, some parameters will be unidentified under interesting null hypotheses. For example, suppose J = 2 and one is interested in testing for no heterogeneity. Then if one tests h 1 = h 2, P is not identified under the null. Following the literature, we ignore these complications. 19 As noted above, readers primarily interested in the actual program effect of JTPA should consult EHL because the estimates presented in Appendix A use significantly less data. Many of the problems investigated here are alleviated by the fact we had 30 months of data after the baseline for some people in estimating the model in that paper.

17 C. Eberwein et al. / Labour Economics 9 (2002) Fig. 2. Hazard rates evaluated for a control with characteristics evaluated at the means in the sample. Hazard rates for out-of-sample extrapolation are evaluated using actual durations for the log duration and the (t 1)/t specifications. For the dummy variable specification, the dummy for the highest duration in the sample is used for out-of-sample extrapolation. The vertical bar gives the point at which out-of-sample extrapolation begins.

18 266 C. Eberwein et al. / Labour Economics 9 (2002) Fig. 3. Hazard rates are evaluated for a control with characteristics evaluated at the means in the sample. Hazard rates for out-of-sample extrapolation are evaluated using actual durations for the (t 1)/t specifications. Hazard rates for out-of-sample extrapolation freeze duration at the longest duration observed in the data for the log duration specification. For the dummy variable specification, the average of the last five dummies for the highest durations in the sample is used for out-of-sample extrapolation. The vertical bar gives the point at which out-ofsample extrapolation begins.

19 for unobserved heterogeneity. These projections may be sensitive to our specification of duration dependence because of the behavior of the estimated hazard function during periods that lie outside the sampling frame. Fig. 2 plots the monthly hazard rates for a woman with average characteristics for each specification. The estimated hazard for fresh unemployment based on the polynomial in log duration specification rises very sharply after month 17, and then falls just as quickly to 0 in month The estimated hazard rate for the fresh employment spells rises after month 20, reaching a rate of 0.26 by month 60. By contrast, when we use the polynomial in (t 1)/t duration specification, the out-of-sample patterns for both the estimated fresh unemployment and employment hazards are more consistent with their in-sample pattern. The pattern for the step function specification is similar to the polynomial in (t 1)/t specification, although the levels of the estimated hazards for the fresh unemployment spells are substantially smaller. We also consider two modifications of our duration specifications in light of these results. First, for the polynomial in log duration specification, the value of h(t) for durations greater than 18 months is frozen at its value at 18 months. Second, we calculate the value of h(t) for out-of-sample durations in the step function specification as the average of the last five coefficients of the step function. As shown by Fig. 3, the estimated hazard rates generated from these modified specifications appear similar to the estimated hazard rates from the (t 1)/t specification, which we devised to limit the possibility of the out-of-sample hazards taking on values that are inconsistent with the in-sample pattern Goodness of fit tests C. Eberwein et al. / Labour Economics 9 (2002) We investigate whether we can distinguish among the projected employment rates generated from these five duration specifications by using goodness-of-fit tests. We compare each projected employment rate with the actual employment rate for a subsample of women who were scheduled to be interviewed 30 months after the baseline. We present the normal statistic for each specification from the goodness-of-fit test in Table 3. See Appendix B for a detailed description of these tests. The first column of Table 3 presents the statistic for each duration specification for the JTPA treatments. As shown by this column, the results indicate that for the treatments we can satisfactorily fit the resurvey data using all five of the duration specifications. Further, in the second column of the table, we see that, for the controls, only one duration specification is rejected at the 5% level by the resurvey data. 21 To the extent that this one rejection does not simply reflect sampling variation, it may result from the 30-month resurvey data being a bad draw from the original baseline sample. To investigate this possibility, we compared the characteristics of the full sample of controls to the characteristics of the sub-sample of controls who had employment data 30 months after the baseline. We could not find any statistically 20 We control for the fact that the heterogeneity distribution changes in the sample as we move to higher duration (Kennan, 1985). 21 If we use the 10% significance level, three of the specifications are rejected for the controls. However, our test statistics are probably too large in the sense that, like Heckman and Walker (1990), we ignore the estimation error in calculating the statistic. Thus it seems more appropriate to use the 5% significance level.

20 268 C. Eberwein et al. / Labour Economics 9 (2002) Table 3 Goodness-of-fit tests for out-of sample predicted employment rates Treatments Controls Polynomial log duration 0.92 (0.36) 1.67 (0.10) Polynomial log duration (freeze duration component 0.98 (0.33) 1.76 (0.08) of fresh hazards after 18 months) Polynomial in (t 1)/t durations 0.69 (0.49) 1.58 (0.11) Dummy variables for duration 1.08 (0.28) 1.94 (0.05) Dummy variables for duration (use average of last five dummies for out of sample projections) 0.44 (0.66) 1.50 (0.13) Normal statistic for test of difference between actual and predicted employment rates. The figures in the table are the absolute values of Normal statistics for tests of differences between actual and predicted employment rates during the 30th month after the baseline. Parameters of the model are estimated using survey responses covering 18 months after the baseline for 1940 women. The tests compare actual employment rates at month 30 for the sub-sample scheduled for a 30-month follow-up interview and employment rates predicted by the model for that month. The numbers in parentheses are P-values. significant differences between the two groups of controls. The upshot is that the test statistics are not very sensitive to the specification of the duration dependence, and we cannot choose between the specifications on the basis of these statistics Alternative estimates of program effects In Table 4, we present the estimated employment rates for all treatments and controls who were interviewed at the 18th month, as well as the differences between their estimated employment rates at 24, 30, 36, 48 and 60 months after the baseline. Conceptually, we view the difference at 30 months as an estimate of the short-run program effect and the difference at 60 months as an estimate of the medium-run program effect. From the table, we see that these projected employment rates, and particularly the estimated program effects, are remarkably insensitive to the method used to estimate h(t). Estimates of the medium-run employment rates for the controls range from to Estimates of the program effects range from to Table 5 presents the average of the expected durations and the steady-state employment rates. We first consider the expected durations. Because we find some of the expected durations to be sensitive to the specification of the duration dependence, we also consider the effect of placing a lower bound on the hazard function. Accordingly, for each of our basic three specifications, we consider the effect of not letting the hazard function fall below (i) 0.01 and (ii) The expected durations for the interrupted unemployment spells in column 1 are insensitive to changing the duration specification, as are the estimated program effects in line 3 of each section of Table 5. For example, the estimated expected durations of the controls range from to months, while the treatment effects range from 2.64 to 2.79 (i.e., being offered training reduces unemployment duration by a little more than 2.5 months). The expected durations for the fresh employment spells in column 3 of Table 5 are sensitive to our specification of duration dependence, with the expected duration for the