Journal of Economic Studies. Quantile Treatment Effect and Double Robust estimators: an appraisal on the Italian job market.

Journal of Economic Studies Quantile Treatment Effect and Double Robust estimators: an appraisal on the Italian job market. Journal: Journal of Economic Studies Manuscript ID JES-0--00 Manuscript Type: Research Paper Keywords: treatment effect, quantile regression, double robust

Page of Journal of Economic Studies 0 0 0 0 Quantile Treatment Effect and Double Robust estimators: an appraisal on the Italian job market. Abstract Several approaches have been proposed to evaluate treatment effect, relying on matching methods propensity score (weighting and blocking), quantile regression, influence function, bootstrap and various combinations of the above. In particular two of these approaches are here considered to define the quantile double robust estimator: i) the inverse propensity score weights, to compare potential output of treated and untreated groups; ii) the Machado and Mata quantile decomposition approach to compute the unconditional quantiles within each group - treated and control. This allows to evaluate the impact of treatment not only on average but also in the tails of the distribution. Through i) and ii) the standard double robust estimator is extended to analyze the treatment effect in the tails. A Monte Carlo study and an empirical application for the Italian job market conclude the analysis. Keywords: treatment effect; quantile regression; propensity score; double robust.

Journal of Economic Studies Page of 0 0 0 0. Introduction Several approaches have been proposed in the literature to estimate treatment effect in observational studies. Many of those estimators focus specifically on binary treatments, considering matching methods based on propensity score (Rosenbaum and Rubin, ) to estimate the average treatment effect under various sets of assumptions (Imbens, 0). The combination of propensity score and regression model, yielding the double robust (DR) estimator (Lunceford and Davidian, 0), computes the treatment effect and compares treated and untreated groups by means of both potential output and fitted values of OLS regressions. This combination prevents misspecification since, if any of the two estimators is misspecified, the other consents to achieve consistency. The combination of the two approaches yields a correct estimator if at least one of the two is correctly specified (Neugebauer and van der Laan, 0). In what follows the DR estimator is extended to analyze the tails of the distribution comparing treated and untreated/control groups, thus defining the quantile based double robust estimator (DRQ). While DR considers only the average treatment effect, DRQ allows to measure the treatment effect along the entire outcome distribution. Such a detailed analysis allows to uncover the presence of heterogeneous impacts of the treatment along the outcome distribution. The focus is on the difference between treated and untreated groups not only at the mean but also in the tails of the distributions of the two groups, at various quantiles. The computation of the treatment effect at the quantiles, far from being a mere technicality, allows to point out variations in the impact of treatment along the outcome distributions. Indeed it is often the case that the impact in the tails sizably differs from the average treatment effect. Treatment can be more effective at the lower/higher quantiles, for instance a tax cut may induce a wider increase in consumption expenditure at lower/higher income than on average, or a different employment contract (i.e. shortfixed or long term-open ended) may have a different impact on low/medium/high earnings.

Page of Journal of Economic Studies 0 0 0 0 DRQ is implemented by replacing the OLS regression analysis with the quantile regression based unconditional distributions of treated and untreated groups and by looking at the selected quantile of the difference between them. A Monte Carlo study shows that away from average the DRQ estimator outperforms the other quantile treatment effect estimators here analyzed. The DRQ results in a real data example are more detailed and more realistic than those provided by other quantile treatment effect estimators.. Inverse propensity weight and double robust Propensity score (Rosenbaum and Rubin, ) measures the probability for the i-th observation of being treated as a function of the covariates, the probability of exposure to treatment conditional on the observed covariates. A popular method to estimate the difference between treatment and control group at the mean is to use the propensity scores to weight the outcome. Weighting the observations in each group by the inverse probability of being in that group defines the potential output (Guo & Fraser, ). The latter is a pseudo-population in which there is no confounding factor. The comparison of the potential output of treated and control yields the inverse propensity weights (IPW) estimator of the treatment effect, and this comparison is generally implemented on average, measuring the effect of a treatment as a difference between the means: the weighted averages computed within each group reflect the true averages in the two populations (Linden and Adams, ). However, the reliability of the results crucially depends upon the correct specification of the model implemented to estimate the propensity scores. Consider the outcome variable Y i, which assumes value Y i when the binary treatment variable Z i is Z i =, and value Y i0 in the control group, if Z i = 0. The propensity score P i (Z i = X, with 0<P i (Z i = X)<, is the probability of treatment given the observed matrix of covariates X Rosenbaum and Rubin () show that individuals from either group with the same propensity score are balanced, that is the distribution of X is the same regardless of exposure status. In practice,

Journal of Economic Studies Page of 0 0 0 0 the propensity score is estimated by assuming that P i (Z i = X) follows a parametric model, e.g. a logistic regression model, estimated by maximum likelihood, P (Z = )= ( ) { ( )} The probability of being in group 0 or provides the weights to compute the potential outcome. Weighting the outcome Y i by the inverse of this probability provides the potential outcome of treated and control, which is computed respectively by and by ( ) = ( ) ( ) given by the difference between potential output of the control group: n - [ () = for Zi=, ( ) ( ) for Zi =0. The IPW estimates of the average treatment effect is ( ) ( ) ( ), the potential output of treated, and, the ( ) ( ) ] () To ensure against misspecification in the IPW model, the double robust estimator includes in the above comparison the fitted values of the OLS regression. In the regression model the values Y i0 = X i0 α 0 +e i0 and Y i = X i α +e i are the observed outcomes within each group, treated and untreated. From one group to the other both covariates and coefficients may vary. Two independent regressions are implemented, one in each group, and the two vectors of coefficients are independently computed at the mean, via OLS, yielding Y = X 0 α 0 and Y = X α, the fitted values of the OLS regressions respectively for untreated and treated. To measure the treatment effect on average, the standard double robust estimator combines the IPW and the regression approach as follows n - [ ( ( ) ( ) ( ) Y ) ( ( ) ( ) + ( ) ( ) Y )] () Equation () takes the difference between two sample means, for the treated and the control group. In the treatment group, when Z i =, equation () becomes n ( ( ) ( ) ( ) Y ) Y and for the control group, with Z i =0, it is

Page of Journal of Economic Studies 0 0 0 0 n Y ( ( ) ( ) ( ) Y ). The weighted fitted values of the regression model, with weights equal to ( ) ( ) for Y and to its reciprocal ( ) ( ) for Y, adjust the potential output within each group. When both IPW and the regression model are correctly specified, equation () defines a semiparametric efficient estimator. If the IPW is correctly specified, equation () will have a smaller variance than IPW. If the regression model is correct, equation () has a larger variance than the simple regression, but it provides an insurance against misspecification of the IPW model. Consistency is granted if at least one of the two models is correctly specified.. The double robust quantile estimator The empirical approach here proposed, DRQ, combines the inverse propensity weights (IPW), which allow to compare potential output of treated and untreated groups (Guo and Fraser, ) and the Machado and Mata (0) quantile approach to compute the fitted values of the regression model within each group - treated and control. This allows to evaluate the impact of treatment not only on average but also in the tails of the distribution.. Unconditional distributions To analyze the impact of treatment beyond the mean, the fitted values of the regressions for treated and control observations in (), Y and Y representing the outcome in each group conditional on the group covariates, have to be replaced by the unconditional distributions of the fitted values of the outcome within each group, Y and Y. This is the case since, unlike the OLS analysis, where conditional and unconditional effects coincide, in the quantile framework the interpretation of the unconditional effects is slightly different from the interpretation of the conditional effects due to the definition of the quantile Alternatively the expectile estimator can be implemented to compute the regression away from the conditional mean. However quantile regressions are more robust than expectiles.

Journal of Economic Studies Page of 0 0 0 0 (Frolich and Melly, ). For instance, looking at a low quantile the conditional quantile will summarize the effect for individuals with relatively low outcome given the covariates, even if the level of the outcome is high. The unconditional quantile, on the other hand, summarizes the effect at a low outcome regardless the covariates, so that the outcome level is unquestionably low. For instance, focusing on the quantiles of a wage equation, the conditional effect of education on earnings is the given wage quantile for those having high school degree that generally differs from the wage quantile of workers without any degree. Vice versa the unconditional wage quantile coincides for both groups, for workers with and without degree, since the unconditional quantile does not depend upon the values of the covariates. Among the alternative methods proposed in the QTE literature, in what follows the Machado and Mata approach (0) is implemented. In a first step, the conditional distributions of the dependent variables are estimated by quantile regression (Koenker, 0) at many quantiles (m=0), separately in the treated and in the control group. A quantile regression (Koenker and Basset, ; Koenker, 0) is defined by an asymmetrically weighted regression having the following objective function where is the selected quantile. α + ( ) α () The above estimator computes the coefficient vector α( ), which measures the impact of the covariates on the outcome at the selected quantile. The analysis within each group yields two sets of estimated coefficients, α ( ) for the treated group and α ( ) for the control. The corresponding fitted values, Y ( ) = α ( ) and Y ( ) = α ( ), provide the conditional distribution of the outcome at a given quantile, conditioned on the covariates X of each group. As mentioned, these conditional distributions are not the appropriate terms for the double robust estimator. The In the OLS case the conditional mean, E[Y X], averages up to the unconditional mean, E[Y]. As a result, the OLS linear model for conditional means, E[Y X] = Xα, implies that E[Y] = E[X] α. When looking at a given quantile Q( ), Q( ) differs from Q(X)α, and the fitted values Xα are no longer appropriate to compute the quantile of the outcome. The unconditional distribution of Y is needed. Chernozhukov et al. () generalize the Machado and Mata procedure to provide inference on the QTE estimates.

Page of Journal of Economic Studies 0 0 0 0 unconditional distributions are needed to compute the difference between the two groups at a selected quantile. The unconditional distributions can be estimated using the Machado and Mata (0) approach. By estimating m different quantile regressions within each group, the estimated coefficients are separately computed in each group yielding m estimates of α ( ) and of α ( ). The covariates are bootstrapped within each group as well, yielding m samples of and of. By bootstrapping the covariates and the estimated coefficients within each group allows to compute the unconditional distributions of the dependent variable for treated and untreated. The unconditional distributions are computed by Y = α ( ) and Y = α ( ), and they take the place of Y and Y in equation ().. The quantile double robust estimator The standard double robust approach estimator of equation () can be now modified in order to allow the analysis of the treatment effect in the tails, beyond the average. The DRQ estimator can be defined as the selected quantile of the following difference: Q ( ( ( ) ( ) ( ) Y ) ( ( ) ( ) + ( ) ( ) Y )) () The weighted fitted values of the unconditional distributions, with weights equal to ( ) ( ) for Y and to ( ) ( ) for Y, adjust the potential output within each group. While equation () takes the sample average of the difference between treated and untreated, equation () looks at the distance between two distributions: the potential outcome adjusted by the weighted unconditional distribution of the treated, minus the potential outcome adjusted by the weighted unconditional distribution of the control group. Such a difference can be computed at various quantiles Q( ), thus measuring the treatment effect not only on average but also in the tails. By computing equation () at various quantiles, it is possible to check if the treatment effect is constant across quantiles, or if it is more effective at a given quantile than elsewhere, or even if its

Journal of Economic Studies Page of 0 0 0 0 impact is opposite in the tails and balances on average. The treatment may have an heterogeneous impact across quantiles and may be more effective in the tails than at the mean, or increase/decrease across quantiles. Just as in equation (), the combination of IPW model and quantile regression based unconditional distributions Y and Y in () ensure against misspecification of one of the two models defining the double robust estimator. With respect to the Machado and Mata approach, the inclusion of the IPW estimator ensures against misspecification in the regression model, provided the IPW is correctly specified. With respect to the standard IPW approach, the quantile regression based unconditional distributions in () insure against incorrect specification of the propensity score equation, provided the regression model is correctly specified, and in addition allows to analyze the treatment effect beyond the mean. Alternative methods to compute the quantile treatment effect have been proposed. Cattaneo () presents an estimator based on the Efficient Influence Function (EIF). The latter, at the mean, coincides with the double robust estimator. In a recent article the approach is generalized to a multi-valued QTE, in order to comprise different levels of the treatment (Cattaneo at al., ). Firpo et al. (0) present the Recentered Influence Function (RIF). The method consists in a regression of a transformation of the outcome variable on the explanatory variables. The QTE is the horizontal distance between two cumulative distribution functions, for treated and untreated, at any fixed percentile. In the following paragraphs, results from the proposed DRQ estimator will be compared with those obtained by both IPW and the EIF estimators.. Monte Carlo replicates To check the properties of the DRQ estimator at various quantiles, a Monte Carlo experiment is implemented. The model is estimated at the first, second and third quartile. The tables report the sample mean, the standard deviation and the relative bias (RB) of the estimators defined in equation

Page of Journal of Economic Studies 0 0 0 0 () and () computed in 00 iterations. Besides considering the double robust, DR, and the quantile double robust, DRQ, estimators, the IPW and the EIF estimator are also computed in the simulations. Following Emsley et al. (0), the outcome Y is generated in a sample of 00 observations as Y i = 0.Z i 0. + 0.B i + 0.A i + 0.A i B i + 0.A i B i + 0.B i + ε i () where the matrix of explanatory variables X=[A B] is defined as follows: B is drawn from a standard normal distribution and A is a binary variable assuming values 0 and. To generate the true propensity score the selected model is π i = + () (. ) (.. ) and a binary treatment is defined by comparing the probabilities predicted by () to values drawn from a uniform [0,] distribution. Z i =0 if the probability π ι is less than the uniform, and Z i = otherwise. Having defined the true model, it is possible to analyze the estimators in the following cases: i) Correct model specification Y i = α 0 + α A i + α B + α A i + α A i B i + α A i B i + α B i ii) Incorrect model specification Y = α 0 + α A i + α B + α A i + α A i B i iii) Correct logit logit(π i ) = β 0 + β A i + β B i + β A i B i iv) Incorrect logit logit(π i ) = β 0 + β A i + β B i. These four cases provide different combinations. The experiments computing the treatment effect when i) is associated with iii) consider the behavior of the estimators in the most favorable case of correct model specification for both regression and propensity score model. Selecting case ii) and iv) together yields the worst possible case of incorrect model specification for both regression model and IPW. Case i) associated with iv) allows to check the behavior of the

Journal of Economic Studies Page of 0 0 0 0 estimators in case of correct regression and incorrect logit model, while case ii) and iii) together define the experiments of correct logit and incorrect regression model. Summarizing, the options (i) to (iv) provide the following four different experiments: model ) model ) model ) model ) true exposure as in (iii) and true regression as in (i) true exposure as in (iii) and false regression as in (ii) false exposure as in (iv) and true regression as in (i) false exposure as in (iv) and false regression as in (ii) Each of the above models is computed at the first, second and third quartile and is compared with the standard DR estimator estimated at the average value, and with the EIF estimator. In addition the IPW for case (i) and (iii) is reported, both at its average value and at the selected quantile, IPWQ in the tables. Finally, to generate variability of the treatment effect on the tails, in the control group the error terms in equation, ε i, has variance smaller than in the treated group. The changing variance between groups causes heteroskedasticity in the potential output, which determines a changing treatment effect across quantiles. In Table the normal error distribution is centered on zero and has σ 0 =0.0 in the control group and in turn σ =0.; 0.; 0. in the treated, thus providing a scale shift model between the two groups. A second set of experiments considers σ 0 =0. in the control group and in turn σ =0.; 0.; in the treated. Table collects these results. The set of experiments reported in Table consider the behavior of the estimators when σ 0 =0. in the control group and in turn σ =; ; in the treated. Figure depicts the box plot of the outcome Y in both the treated and the control group as computed in one of the Monte Carlo replicates when σ 0 =0. σ =. The graph clearly shows how the larger variance of the treatment distribution causes an heterogeneous impact of the treatment across All the empirical analyses were performed using STATA software (Version., http://www.stata.com). DR, EIF and IPW/IPWQ estimators were implemented using the user written Stata module "poparms", while the Machado and Mata (0) decomposition approach was implemented using the "mmsel" user written Stata program.

Page of Journal of Economic Studies 0 0 0 0 quantiles. In the graph the horizontal solid lines depict the quartiles of the two distributions. The discrepancy between each couple of lines, signaling the distance at a given quartile between the two distributions and marked by the vertical arrows to the left, increases across quartiles. The difference between the quantiles of treated and control increases in moving to higher outcome levels. Table considers the experiments with σ 0 =0.0 in the control group and in turn σ =0.; 0.; 0. in the treated. The first row of this table reports the true treatment effect, which is 0. at the first quartile, 0. at the median, and 0. at the upper quartile. At the first quartile the DRQ estimator is the closest to the true value: very close in model () and (), and closer than all the others in model () and (). At the median the estimator that best approximates the true value is the standard DR, but DRQ results are close by. At the upper quartile DRQ is once again the closest to the true value with only one exception where it overestimates the true value in model () when σ =., that is when there is the smallest difference between treatment and control. In addition this table shows that DRQ yields the smallest standard deviations. It is possible to conclude that while the standard DR estimator performs well at the center of the distribution, in the tails the DRQ estimator improves upon the other estimators. These results are confirmed when the divergence in the variances between control and treatment group increases. In Table, where σ 0 =0. and σ =0.; 0.;, DRQ is the closest to the true treatment effect in the tails, both at the first and third quartile, while DR is the closest on average, at the center of the distribution of the difference between the two groups, treated and control. Analogously in Table, where σ 0 =0. and σ =; ;. The EIF estimator in the above sets of experiments does not provide good results. Regardless of the model under analysis, defined by any combination of true/false exposure and true/false regression, when implementing the analysis at the center, at the mean, the EIF estimator is often the farthest from the true treatment effect, with only few exceptions. In moving away from the mean, the DRQ estimator is the closest to the true value of the treatment effect.

Journal of Economic Studies Page of 0 0 0 0. Case study The previous Monte Carlo experiment seems to provide evidence on the validity of the DRQ estimator. In this paragraph we provide an empirical application to show how the method we propose can be applied to real data. The empirical analysis considers a data set for the Italian labor market released by ISFOL, an Italian research institute on labor. The ISFOL data under analysis are random sampled from the Italian population in the age range -, in year. We aim to compare per capita real labor incomes of different types of work contracts, assessing the impact of the length of the contract. So, in this example, the control population is provided by workers with long term contracts while the treated population are individuals with short term contracts. Since the outcome of interest, per capita real labor income, can be affected by other phenomena, confounding effects like the different composition of the two samples, a set of explanatory variables is included to account for the non-random nature of the comparison. A confounding effect is for instance age, since individuals with long term contracts are older, with an average age of 0 versus an average age of in the short term contract. Another difference concerns education, that is slightly higher in the short term contract: % of them have a university degree versus % of workers with long term contracts. The imbalances between individuals are explicitly controlled by including additional explanatory variables in the X matrix. In detail, the variables included in X are: age, age squared, level of education, gender, region of residence, mother and father level of education. Descriptive statistics of the variables are listed in Table. The DR estimator allows to highlight the change of the average income level, conditioned to the confounding variables. To verify the impact of contracts at the lower and higher labor income levels, the DRQ estimator is implemented. The results obtained with DR and DRQ were also compared with those obtained by the EIF estimator in Cattaneo (). Overall, real incomes (measured in euro per year) from a total of n=, individuals were analyzed, considering only those with positive income. The sample size comprises n st =,0 for

Page of Journal of Economic Studies 0 0 0 0 short term contracts equal to.% of the sample, and n lt =, for long term contracts covering.% of the sample. Figure depicts the box plot of each distribution, the per capita real labor income of long and short term contracts. The latter distribution has lower median and lower quartiles with respect to the long term distribution. The first row of Table measures the treatment effect as computed on average by DR, in the column headed Mean, and at the q=.,.0 and. quantiles, by DRQ. The second row of the table reports the EIF results. Both quantile estimators find a reduction of the real income for the short term contract. The reduction is wide at the first quartile, while it becomes milder across quantiles, as can be seen in the two top rows of this table. The standard DR yields a treatment effect of DR=- /per year. This impact in absolute terms is lower than those observed by DRQ and EIF estimates at the median, respectively DRQ.0 =- /per year and EIF.0 =-0 /per year. In particular, both treatment effect quantile estimators identify a less damaging effect across quantiles, where workers with the lowest labor income are the most damaged by the short term contracts. The DRQ estimates at the th quantile report a decrease in per capita labor income for short term contracts of DRQ. =- /per year. The negative impact moves to DRQ.0 =- /per year at the median while at the th quantile is equal to DRQ. =- /per year. The EIF estimates of the impact of short term contracts across quantiles reports a greater effect at the th quantile, EIF. =-0 /per year, while the impact at and above the median is quite constant, respectively EIF.0 =-0 /per year and EIF. =- /per year. The last two columns of the table report the interquartile differences between the median and the first quartile, 0, and between the upper quartile and the median, 0, for the two quantile estimators here implemented. DRQ presents an interquartile difference in the treatment effect that halves from 0 = /per year in the left tail of the distribution to 0 = /per year in the upper tail. The EIF interquantile difference is quite sizable in the left tail and drops from 0 =0 /per year to 0 = /per year in the upper tail.

Journal of Economic Studies Page of 0 0 0 0 Next the analysis is further detailed to consider the effect of the contracts length across the Italian regions, focusing in particular on its impact on the Italian southern economy in comparison with the North-Center area. Figure summarizes the per capita real labor income distributions for the two different types of contracts in the two regions, respectively the Center-North to the left and the South to the right hand side of the figure. In both graphs the box plot of the short term contracts is lower with respect to the one for long term contracts. In addition the first quartile of the short term contracts in the South is lower than its analogue in the North-Center regions. The middle two rows of Table report the results for the central-northern regions in a sample of size n cn =,, and.% of them, n cn,st =, are short term contracts. The bottom rows of the table collect the results of the quantile estimators in the southern regions in a sample of size n s =,, and.% of them, n s,st =, are short term contracts. The average treatment effect as computed by DR yields a wider impact of the short term contracts in the Southern regions, DR s = - /per year versus DR n =- /per year in the North-Center. The quantile treatment effect estimators allow to further detail this result. DRQ shows a less damaging impact of treatment across quantiles in both Center-North and in the southern regions with the widest negative impact in the South at the th quantile, DRQ s,. = - /per year. In both regions 0 is about half the size of 0, signalling the left skewness shown in Figure. EIF in the North-Center computes a wider negative impact at the th quantile with respect to the southern regions, EIF cn,. = - /per year versus EIF s,. = - /per year in the South. But what is surprising is the set of results at the top quartile, where EIF computes the widest of all impacts for the South, EIF s,. = - /per year versus EIF cn,. = - /per year in the North-Center. The impact of short term contracts in the South would have its most damaging impact at the upper quartile, but this result is at odds with the weakening impact of treatment estimated by EIF in the nationwide sample and with the box plots in Figure. The interquartile differences in the South are

Page of Journal of Economic Studies 0 0 0 0 small at the lower quartiles, 0 = /per year, and becomes negative and quite sizable at the upper quartiles, 0 = /per year. This would point out a right skewness which is not supported by the box plots in Figure. A comparison of the nationwide results in the top rows of Table with the analysis at a regional level of the middle and bottom rows for DRQ shows that while at the median and at the upper quartile the nationwide impact is similar to the regional impacts, at the first quartile - the lower incomes - the nationwide effect is close to the North-Center impact but undervalues the wide damaging impact in the South. In sum, the DRQ estimator shows that the impact of the short term contracts is not constant and becomes less severe across quantiles, at the higher incomes. An investigation at the regional level shows that these contracts do not have the same impact across regions, but are particularly damaging for the southern lower incomes. This result is confirmed by the average treatment effect measured by the standard DR estimator. The EIF results are less easy to interpret, since while the nationwide and the North-Center results show a weakening impact across quantiles, they show instead an increasingly damaging impact across quantiles in the South, particularly large at the upper quartile. To further analyze this issue, i.e. if the treatment has an increasing or a decreasing impact across quantiles, quantile regressions at the th, 0 th and th quantiles are implemented nationwide and individually in each of the two regions. Table presents the results for the full sample in the first section, for the North-center in the middle section and for the South in the final section of the table. Besides all the explanatory variables included in the X matrix, a dummy variable for type of contract is added, Treatment in the Table, assuming value for the short term contracts. This is a naive way to measure treatment effect. Focusing on the Treatment estimated coefficient, its estimates show a weakening impact across quantiles in all the regressions, assuming values α,. = - /per year, α,. = - /per year, α,. = - /per year in the entire sample; α,. = - The regression model alone does not control for the treatment assignment.

Journal of Economic Studies Page of 0 0 0 0 0 /per year, α,. = - /per year, α,. = - /per year in the North-Center regression and α,. = - /per year, α,. = - /per year, α,. = - /per year in the South.. Conclusions DR compares treated and untreated groups by means of both the IPW potential output and the fitted values of OLS regressions. The combination of these two estimators, propensity score and regression models, allows to prevent misspecification. If any of the two estimators is misspecified the other allows to achieve consistency as long as at least one of the two is correctly specified. In the above analysis the DR estimator has been extended to analyze the tails of the distribution comparing treated and control group. The quantile based double robust estimator (DRQ) has been defined and implemented by replacing the OLS fitted values for treated and untreated with the quantile regression estimates of the unconditional distributions of treated and untreated. The Machado and Mata (0) approach to compute the quantile regression based unconditional distributions has been selected. The computation of the treatment effect at various quantiles allows to point out discrepancies between treatment and control along the entire outcome distributions. The discrepancy in the tails may differ from the divergence between the average values. Treatment can be more effective at the lower/higher quantiles. For instance a tax cut may induce a wider increase in consumption expenditure at lower/higher income than on average; an economic crisis has usually a decreasing impact on low/median/high earnings. The DRQ estimator has been compared to the standard DR in a Monte Carlo experiment. In addition, the EIF and the IPW estimators are analyzed in the simulations. The results show that at the center, on average, the standard DR estimator outperforms all the others. However DR cannot investigate the impact of treatment in the tails, i.e. cannot evaluate the treatment effect away from the mean, at the lower and the upper tails. The simulations show that at the first and at the third

Page of Journal of Economic Studies 0 0 0 0 quartile DRQ outperforms all the other quantile treatment effect estimators and DRQ turns out to be the closest to the true treatment effect value in the tails. A case study is provided by a wage equation, where the comparison is between long term and short term contracts. DRQ shows the presence of an heterogeneous impact of short term contracts on labor income. Their impact changes depending on the income level, the outcome quantiles, and on the geographical region, showing a less damaging impact across quantiles but being particularly severe in southern workers at the lower incomes. References Cattaneo M., () Efficient semiparametric estimation of multi-valued treatment effects under ignorability, Journal of Econometrics, pg. -. Cattaneo M., Drukker D., Holland A., () Estimation of multivalued treatment effects under conditional independence. The Stata Journal, pg. 0-0. Chernozhukov V., Fernandez-Val I., Melly B., () Inference on counterfactual distributions, Econometrica, pg. -. Emsley R., Lunt M., Pickles A., Dunn G., (0) Implementing double-robust estimators of causal effects, The Stata Journal, pg.. Firpo S., Fortin N., Lemieux T., (0) Unconditional quantile regression, Econometrica, pg. -. Frolich M., Melly B., () Estimation of quantile treatment effects with Stata, The Stata Journal, pg. -. Guo S., Fraser M.W., () Propensity score analysis: Statistical methods and applications Sage, Thousand Oaks, CA. Imbens, G.W. (0). Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review. Review of Economics and Statistics, (), -. Koenker R., (0) Quantile regression, Cambridge University Press, Cambridge.

Journal of Economic Studies Page of 0 0 0 0 Koenker R., Bassett G., () Regression quantiles, Econometrica, pg. -0. A. Linden, J.L. Adams, () Using propensity score-based weighting in the evaluation of health management programme effectiveness Journal of Evaluation in Clinical Practice, (), pp.. Lunceford J., Davidian M., (0) Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study, Statistics in medicine, pg. -0. Machado J., Mata J., (0) Counterfactual decomposition of changes in wage distributions using quantile regression, Journal of Applied Econometrics, pg. -. Neugebauer, R., & van der Laan, M. (0). Why prefer double robust estimators in causal inference? Journal of Statistical Planning and Inference, ( ), 0-. Rosenbaum P., Rubin D., () The Central Role of the Propensity Score in Observational Studies for Causal Effects, Biometrika 0, pg. -.

Page of Journal of Economic Studies 0 0 0 0 Figure : Treatment and control group in one Monte Carlo replicate when σ 0=0. σ =

Journal of Economic Studies Page of 0 0 0 0 Figure : Box plots of per capita real labor income for different type of contracts nationwide Figure : Box plot of per capita real labor income in the North-Center and in the South North-Center South

Page of Journal of Economic Studies 0 0 0 0 Table : Simulations results at the.,.0 and. quantiles, control variance σ 0 =0.0 σ 0=0.0 σ =0. Mean Std.dev RB Mean Std.dev RB Mean Std.dev RB θ=0. θ=0.0 θ=0. True value 0. 0.00 0. 0.00 0. 0.00 Model IPW 0.0 0.0.0 0.0 0.0 0. 0.0 0.0 -.0 IPWQ 0.0 0.00. 0.0 0.0 0. 0.0 0.0 -. DR 0.00 0.0. 0.00 0.0 0.0 0.00 0.0 -. DRQ 0.0 0.00 0. 0.0 0.00 0. 0. 0.00-0.0 EIF 0.0 0.00. 0.0 0.0 0. 0. 0.0 -. Model DR 0.0 0.0.0 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0. 0. 0.0. 0.0 0.0 -. EIF 0.0 0.00. 0.0 0.0 0.0 0.0 0.00 -. Model IPW 0. 0.0. 0. 0.0. 0. 0.0 -. IPWQ 0.0 0.00. 0. 0.0-0.0 0.0 0.0 -. DR 0.00 0.0. 0.00 0.0 0.0 0.00 0.0 -. DRQ 0.0 0.00 0. 0.0 0.00 0. 0. 0.00-0.0 EIF 0.0 0.00. 0.0 0.0 0. 0.0 0.0 -. Model DR 0.0 0.0.00 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0. 0. 0.0. 0.0 0.0 -. EIF 0. 0.00. 0.0 0.0 0. 0.0 0.0 -. σ 0=0.0 σ =0. Mean Std.dev RB Mean Std.dev RB Mean Std.dev RB θ=0. θ=0.0 θ=0. True value 0. 0.0 0.0 0.00 0. 0.00 Model IPW 0.0 0.0. 0.0 0.0 0. 0.0 0.0 -. IPWQ 0. 0.0. 0.0 0.0. 0. 0.0 -. DR 0.0 0.0. 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0. 0. 0.0. 0. 0.0-0.0 EIF 0. 0.0. 0. 0.0.0 0. 0.0 -. Model DR 0. 0.0. 0. 0.0 0. 0. 0.0 -.0 DRQ EIF 0. 0. 0.0 0.0.. 0. 0.0 0.0 0.0.. 0. 0. 0.00 0.0 -. -. Model IPW 0. 0.0. 0. 0.0.00 0. 0.0 -.0 IPWQ 0. 0.0. 0. 0.0. 0. 0.0 -. DR 0.0 0.0. 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0. 0. 0.0. 0. 0.0-0.0 EIF 0. 0.0. 0. 0.0.0 0. 0.0 -. Model DR 0. 0.0. 0. 0.0 0.0 0. 0.0 -. DRQ 0. 0.0.0 0. 0.0. 0. 0.0 -. EIF 0. 0.0. 0. 0.0. 0. 0.0 -.

Journal of Economic Studies Page of 0 0 0 0 σ 0=0.0 σ =0. Mean Std.dev RB Mean Std.dev RB Mean Std.dev RB θ=0. θ=0.0 θ=0. True value 0. 0.0 0.0 0.0 0.0 0.0 Model IPW 0. 0.0 0. 0. 0.0 0. 0. 0.0 -. IPWQ 0. 0.0.0 0. 0.0. 0. 0.0 -. DR 0.0 0.0 0. 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0.0 0. 0.0. 0.0 0.0-0. EIF 0. 0.0.0 0. 0.0. 0. 0.0 -.0 Model DR 0. 0.0 0. 0. 0.0.0 0. 0.0 -. DRQ 0. 0.0. 0. 0.0. 0. 0.0 -. EIF 0. 0.0.0 0. 0.00. 0.0 0.0 -. Model IPW 0. 0.0. 0. 0.0. 0. 0.0 -. IPWQ 0. 0.00. 0. 0.0. 0. 0.0 -.0 DR 0.0 0.0 0. 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0. 0. 0.0. 0.0 0.0-0. EIF 0. 0.0. 0. 0.0.0 0. 0.0 -. Model DR 0. 0.0 0. 0. 0.0.0 0. 0.0 -.0 DRQ 0. 0.0. 0. 0.0. 0. 0.0 -.0 EIF 0. 0.0. 0. 0.00. 0. 0.0 -. Table : Simulations results at the.,.0 and. quantiles, control variance σ 0 =0. σ 0=0. σ =0. Mean Std.dev RB Mean Std.dev RB Mean Std.dev RB θ=0. θ=0.0 θ=0. True value 0. 0.0 0. 0.00 0. 0.00 Model IPW 0.0 0.0. 0.0 0.0 0. 0.0 0.0 -. IPWQ 0. 0.0. 0. 0.0.0 0. 0.0 -. DR 0.0 0.0. 0.0 0.0 0. 0.0 0.0 -. DRQ 0.0 0.0 0. 0.0 0.0 0. 0. 0.00-0. EIF 0. 0.0. 0. 0.0. 0. 0.0 -. Model DR 0.0 0.0.0 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0.0 0. 0.0. 0. 0.0 -. EIF 0. 0.0. 0. 0.0. 0. 0.0 -. Model IPW 0. 0.0. 0. 0.0. 0. 0.0 -. IPWQ 0.0 0.0. 0. 0.0. 0. 0.0 -. DR 0.0 0.0. 0.0 0.0 0. 0.0 0.0 -.0 DRQ EIF 0.0 0. 0.0 0.0 0.. 0.0 0. 0.0 0.0 0.. 0. 0. 0.00 0.0-0. -. Model DR 0.0 0.0. 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0.0 0. 0.0. 0. 0.0 -. EIF 0. 0.0. 0.0 0.0. 0. 0.0 -.

Page of Journal of Economic Studies 0 0 0 0 σ 0=0. σ =0. Mean Std.dev RB Mean Std.dev RB Mean Std.dev RB θ=0. θ=0.0 θ=0. True value 0. 0.0 0.0 0.0 0. 0.0 Model IPW 0. 0.0.0 0. 0.0.0 0. 0.0 -. IPWQ 0. 0.0. 0. 0.0. 0. 0.0 -. DR 0.0 0.0. 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0.0 0. 0.0. 0.0 0.0-0. EIF 0. 0.0. 0. 0.0. 0. 0.0 -. Model DR 0. 0.0. 0. 0.0. 0. 0.0 -. DRQ 0. 0.0. 0. 0.0. 0.0 0.0 -.0 EIF 0. 0.0. 0.0 0.0. 0. 0.00 -. Model IPW 0. 0.0. 0. 0.0. 0. 0.0 -. IPWQ 0. 0.0. 0. 0.0. 0.0 0.0 -. DR 0.0 0.0. 0.0 0.0 0. 0.0 0.0 -. DRQ 0. 0.0. 0. 0.0. 0.0 0.0-0. EIF 0. 0.0. 0. 0.0. 0. 0.0 -. Model DR 0. 0.0. 0. 0.0. 0. 0.0 -. DRQ 0. 0.0. 0. 0.0. 0.00 0.0 -. EIF 0. 0.0.0 0. 0.0. 0. 0.0 -. σ 0=0. σ = Mean Std.dev RB Mean Std.dev RB Mean Std.dev RB θ=0. θ=0.0 θ=0. True value -0.00 0.0 0.0 0.0. 0.0 Model IPW 0. 0.0 0. 0. 0.0.0 0. 0.0 -. IPWQ 0. 0.0. 0. 0.0.0.0 0.0 -. DR 0. 0.0 0. 0. 0.0 0. 0. 0.0 -. DRQ -0.0 0.00. 0. 0.0.. 0.0 0.0 EIF 0.0 0.0. 0. 0.0..00 0.00 -. Model DR 0. 0.00. 0. 0.00. 0. 0.00 -. DRQ 0.00 0.0.0 0. 0.0..00 0.0 -. EIF 0.0 0.0.00 0. 0.0.0.0 0.0 -. Model IPW 0. 0.0. 0. 0.0. 0. 0.0 -.0 IPWQ 0. 0.0. 0.0 0.0..0 0.0 -. DR 0. 0.0 0.0 0. 0.0 0. 0. 0.0 -. DRQ -0.0 0.00. 0. 0.0.. 0.0 0.0 EIF 0.0 0.0. 0. 0.0.0.0 0.0 -. Model DR 0. 0.00.0 0. 0.00. 0. 0.00 -. DRQ 0.00 0.0. 0. 0.0..0 0.0 -. EIF 0.0 0.0. 0. 0.0.0.0 0.0 -.

Journal of Economic Studies Page of 0 0 0 0 Table : Simulations results at the.,.0 and. quantiles, control variance σ 0 =0. σ 0=0. σ = Mean Std.dev RB Mean Std.dev RB Mean Std.dev RB θ=0. θ=0.0 θ=0. True value 0. 0.0 0. 0.0 0. 0.0 Model IPW 0. 0.0. 0. 0.0.0 0. 0.0 -. IPWQ 0.0 0.0. 0. 0.0. 0. 0.0-0. DR 0.0 0.0. 0.0 0.0. 0.0 0.0 -. DRQ 0. 0.0-0.0 0.0 0.0 0. 0.0 0.0 0. EIF 0. 0.0. 0. 0.0. 0. 0.0-0. Model DR 0. 0.00.0 0. 0.00. 0. 0.00 -. DRQ 0. 0.0. 0. 0.0. 0. 0.0. EIF 0. 0.0. 0. 0.0. 0. 0.0-0. Model IPW 0. 0.00. 0. 0.00.0 0. 0.00 -. IPWQ 0. 0.0. 0. 0.0. 0. 0.0 0. DR 0.0 0.0. 0.0 0.0. 0.0 0.0 -. DRQ 0. 0.0-0.0 0.0 0.0 0. 0. 0.00 0. EIF 0. 0.0. 0. 0.0. 0. 0.0-0. Model DR 0. 0.00.00 0. 0.00. 0. 0.00 -.0 DRQ 0. 0.0. 0. 0.0. 0. 0.0.0 EIF 0. 0.0. 0. 0.0. 0. 0.00 0. σ 0=0. σ = Mean Std.dev RB Mean Std.dev RB Mean Std.dev RB θ=0. θ=0.0 θ=0. True value -.0 0. 0.0 0.0. 0.0 Model IPW 0. 0.0. 0. 0.0. 0. 0.0 -. IPWQ -0. 0.. 0. 0.0.0. 0. -. DR 0. 0.0. 0. 0.0.0 0. 0.0 -.0 DRQ -. 0.0 -. 0. 0.0 0.. 0.0. EIF -0. 0.. 0. 0.0.. 0. -. Model DR 0. 0.0. 0. 0.0. 0. 0.0 -. DRQ -.0 0.0. 0. 0.0.. 0. -0. EIF -0. 0.. 0.0 0.0.0. 0. -. Model IPW 0. 0.0 0. 0. 0.0. 0. 0.0 -. IPWQ -0. 0.. 0.0 0.00.0. 0. -. DR 0. 0.0. 0. 0.0. 0. 0.0 -. DRQ -. 0.0 -. 0. 0.0 0.. 0.0. EIF -0. 0.. 0.0 0.0.0. 0. -. Model DR 0. 0.0.0 0. 0.0. 0. 0.0 -. DRQ -.0 0.. 0. 0.0.. 0. -0.0 EIF -0. 0.. 0. 0.0.. 0. -.

Page of Journal of Economic Studies 0 0 0 0 σ 0=0. σ = Mean Std.dev RB Mean Std.dev RB Mean Std.dev RB θ=0. θ=0.0 θ=0. True value -. 0. 0.0 0.. 0. Model IPW 0. 0. 0. 0. 0..0 0. 0. -. IPWQ -. 0.. 0. 0...0 0. -. DR 0. 0. 0. 0. 0.. 0. 0. -. DRQ -. 0. -. 0. 0. -..0 0.. EIF -. 0.. 0. 0... 0. -. Model DR 0. 0. 0. 0. 0.. 0. 0. -.00 DRQ -. 0. -. 0. 0... 0. 0.0 EIF -. 0.. 0. 0.0..0 0. -. Model IPW 0. 0. 0. 0. 0.. 0. 0. -. IPWQ -. 0..00 0. 0... 0. -. DR 0. 0. 0. 0. 0.. 0. 0. -.0 DRQ -. 0. -. 0.0 0. -.0.0 0..0 EIF -. 0.0. 0.0 0...0 0. -. Model DR 0. 0. 0. 0. 0..0 0. 0. -.0 DRQ EIF -. -. 0. 0. -.. 0. 0. 0. 0.0.... 0. 0. 0. -. Note: In bold are the values closer to the true treatment effect within each experiment, in 00 iterations. Table : Summary statistics of the variables in the case study Long term contracts Short term contracts Mean std.dev Min Max Mean std.dev Min Max Income (euros per month),.0,.0,000,0.0,00. 0 0,000 Age 0.... Age squared,.0.0,0,.0.,0 Gender ( = male) 0. n.a 0 0.0 n.a 0 Primary school 0. n.a 0 0. n.a 0 Secondary school 0. n.a 0 0. n.a 0 University 0. n.a 0 0. n.a 0 University (father) 0. n.a 0 0.0 n.a 0 University (mather) 0. n.a 0 0. n.a 0 Primary school (father) 0. n.a 0 0. n.a 0 Primary school (mother) 0. n.a 0 0. n.a 0 Secondary school (father) 0. n.a 0 0. n.a 0 Secondary school (mother) 0. n.a 0 0. n.a 0 Living region ( if South) 0. n.a 0 0.0 n.a 0

Journal of Economic Studies Page of 0 0 0 0 Table : Economic impact of contracts on per capita real labor income, double robust estimator and efficient function Mean=DR q =. q =.0 q =. 0- -0 Full sample Double robust - - - - Efficient function - -0-0 - 0 - North-Center Double robust - - - - Efficient function - - -0 - South Double robust - - - - Efficient function - - - - - Table : Quantile regression, full sample Full sample th 0 th th Coeff. p-value Coeff. p-value Coeff. p-value Constant -. 0.0. 0.0. 0.000 Treatment (=short term contract) -.00 0.000 -.0 0.000 -.0 0.000 Age. 0.000. 0.000. 0.000 Age -0.0 0. -0.0 0. 0.0 0. Gender ( = male) 0. 0.000. 0.000. 0.000 Primary school ( if yes). 0.000.0 0.000.0 0.000 Secondary school ( if yes) 0. 0.000. 0.000.0 0.000 University ( if yes). 0.000.0 0.000. 0.000 University (father) ( if yes).0 0. -. 0.0 -.0 0.0 University (mather) ( if yes). 0.. 0.00. 0.00 Primary school (father) ( if yes). 0. -. 0.0 -. 0.0 Primary school (mother) ( if yes). 0..0 0.0 0. 0.00 Secondary school (father) ( if yes). 0.0 -.0 0. -. 0. Secondary school (mother) ( if yes). 0.0. 0.00. 0.00 Region ( if South) -. 0.000 -.0 0.000 -. 0.000

Page of Journal of Economic Studies 0 0 0 0 North-Center th 0 th th Coeff. p-value Coeff. p-value Coeff. p-value Constant -.00 0.0 0. 0.000 0. 0.000 Treatment (=short term contract) -0.000 0.000 -. 0.000 -. 0.000 Age. 0.00. 0.00. 0.00 Age -0.00 0. 0.0 0. 0.0 0. Gender ( = male). 0.000. 0.000. 0.000 Primary school ( if yes). 0.000. 0.000. 0.000 Secondary school ( if yes). 0.000.0 0.000.0 0.000 University ( if yes). 0.000. 0.000.0 0.000 University (father) ( if yes). 0.0 -.0 0. -. 0.0 University (mather) ( if yes) -0. 0.. 0.. 0. Primary school (father) ( if yes). 0.0 -. 0. -. 0. Primary school (mother) ( if yes).0 0.0.0 0.0. 0. Secondary school (father) ( if yes). 0.0 0.000.000 -. 0.0 Secondary school (mother) ( if yes). 0.0. 0.00. 0. South th 0 th th Coeff. p-value Coeff. p-value Coeff. p-value Constant -.0 0.000 -. 0.0 -. 0. Treatment (=short term contract) -.0 0.000 -. 0.000 -.00 0.000 Age. 0.000. 0.000.0 0.000 Age -0. 0.000-0. 0.00-0. 0.0 Gender ( = male).0 0.000. 0.000. 0.000 Primary school ( if yes). 0.000. 0.000. 0.000 Secondary school ( if yes). 0.000. 0.000. 0.000 University ( if yes). 0.000.0 0.000. 0.000 University (father) ( if yes) -. 0. -.0 0. -.0 0.00 University (mather) ( if yes). 0.0. 0.00.0 0.00 Primary school (father) ( if yes). 0. -. 0. -. 0. Primary school (mother) ( if yes) -. 0.. 0.. 0. Secondary school (father) ( if yes). 0. -.0 0. -.0 0.0 Secondary school (mother) ( if yes). 0.. 0.0. 0.0