EVALUATION OF THE ACTIVE LABOUR MARKET PROGRAMME SEVERANCE TO JOB

Transcription

1 EVALUATION OF THE ACTIVE LABOUR MARKET PROGRAMME SEVERANCE TO JOB ALEKSANDRA NOJKOVIĆ, SUNČICA VUJIĆ AND MIHAIL ARANDARENKO Economics Faculty, University of Belgrade, Serbia Department of Management, London School of Economics, United Kingdom Paper prepared for presentation at the World Bank International Conference on Poverty and Social Inclusion in the Western Balkans WBalkans 2010 Brussels, Belgium, December 14-15, 2010 Copyright 2010 by author(s). All rights reserved. Readers may make verbatim copies of this document for non-commercial purposes by any means, provided that this copyright notice appears on all such copies. Corresponding author: Sunčica Vujić. Postal address: London School of Economics, Department of Management, New Academic Building (NAB), 54 Lincoln s Inn Fields, London, WC2A 3LJ. Tel: +44 (0) Fax: +44 (0) s.vujic@lse.ac.uk 1

2 ABSTRACT This paper studies the treatment effect of the active labour market programme, Severance to Job (STJ), implemented in Serbia from November 2006 until October The programme assisted redundant workers in finding new employment by relying on a specific feature of Serbian transition almost universal availability of rather generous severance payment packages, secured by new owners of formerly socially owned firms and/or by the government. In order to calculate the treatment effect of the programme, we employ propensity score matching (PSM) between programme participants (treatment group) and a comparable group of people not participating in the programme (control group). Empirical analysis shows significant effect of programme participation on main labour market outcomes (employment and unemployment probabilities) and no significant effect on subjective improvement in personal life (self-assessed well-being and life satisfaction). The STJ programme and the subsequent evaluation allow for a better-informed policy making in labour and employment fields in Serbia. Keywords: Labour Market Programme; Evaluation; Treatment and Control Groups; Propensity Score Matching JEL Classification: J68, H43, P27 2

3 1 INTRODUCTION This paper studies the treatment effect of the active labour market programme, Severance to Job (STJ), implemented in Serbia from November 2006 until September The STJ project aimed to preserve as far as possible, the high employment and participation rates of experienced workers, by providing re-employment opportunities for those among them who have lost their jobs in the process of privatization and restructuring. The STJ programme assisted redundant workers in finding new employment by relying on a specific feature of Serbian transition almost universal availability of rather generous severance payment packages, secured by new owners of formerly socially owned firms and/or by the government. The STJ programme aim was twofold: to broaden the range of re-employment opportunities through provision of incentives for a productive use of severance payments and to support the National Employment Service and the ministry in charge of employment policy (at first Ministry of Labour and Social Policy and afterwards, since mid-2007, the Ministry of Economy and Regional Development) in implementing and monitoring nation-wide active labour market programmes. On the basis of a comprehensive data set covering almost the entire universe of programme participants as well as a comparison group (treatment and control groups), we employ statistical tools for programme evaluation, namely, the propensity score matching (PSM), designed to calculate the treatment effect of the programme (the average treatment effect on the treated (ATT)). That is, using a suitable programme evaluation technique, we show how the analysed outcomes changed for programme participants, compared to a situation where they would not have received the treatment. The treatment effects are assessed across a range of labour market outcomes (unemployment probability, employment probability, non-activity, average net wage in 2009), as well as the range of subjective well-being outcomes (subjective evaluation of 3

4 financial situation in 2009, current subjective evaluation of financial situation as compared to the situation before job loss, chances to find a job, subjective welfare and life satisfaction) between those who participated in the STJ programme and those who did not. Empirical analysis shows significant effect of programme participation on main labour market outcomes (employment and unemployment probabilities) and no significant effect on subjective improvement in personal life (e.g., self-assessed well-being and life satisfaction). This is in contrast to Bonin and Rinne (2006), who found that the active labour market programme Beautiful Serbia 1 had stronger effects when judged based on subjective well-being rather than immediate labour market outcomes. This study contributes to the small body of literature analysing the effectiveness of labour market policies in transition economies in general and in Serbia in particular. A few examples of papers which evaluate the active labour market policy programmes in Eastern Europe, during the period which for some countries also includes accession to the European Union, are Dmitrijeva and Hazans (2007) focusing on Latvia; Rodriguez-Planas and Benus (2006) focusing on Romania; Kluve et al. (1999) focusing on Poland; Lubyova and van Ours (1998) focusing on Slovakia, and O Leary (1998) focusing on Hungary. The only other paper which provides a comprehensive net impact evaluation of an active labour market programme in Serbia and Montenegro is by Bonin and Rinne (2006). The Severance to Job programme and the subsequent net impact evaluation presented by this paper, allow for a better-informed policy making in labour and employment fields in Serbia. The rest of the paper is organised as follows. Section 2 gives an overview of Severance to Job programme, together with economic and social context of the programme and a description of the programme beneficiaries. Section 3 describes the methodology and propensity 1 The Beautiful Serbia programme operated in Serbia and Montenegro in the period and provided training and temporary work in the construction sector. 4

5 score matching estimation approach, while Section 4 presents estimated treatment effects of the programme, based on labour market and subjective well-being outcomes. Section 5 concludes. 2 ACTIVE LABOUR MARKET PROGRAMME SEVERANCE TO JOB 2.1 ECONOMIC AND SOCIAL CONTEXT OF THE PROGRAMME As the process of economic transition unfolded in the period preceding the creation of the programme, lack of employment opportunities and low levels of job creation remained the major worrisome aspects of the Serbian labour market. According to the National Employment Service (NES), registered unemployment was at the very high level of 27.4% in October 2005, and continued to grow steadily since then, reaching 28.4% in July According to the Labour Force Survey of the Republican Bureau of Statistics, the unemployment rate rose from 14.6% in 2003 to 18.5% in 2004 and then to 20.9% in Over the same period, employment and participation rates fell slightly. The restructuring of large public enterprises, which are a drain on public resources, accelerated in 2006 and This, coupled with completion of the privatization of socially owned enterprises, contributed further to the unstable political and socio-economic situation, as many people lost their jobs. According to the NES data, between 2001 and 2005, approximately 270,000 workers have lost their jobs due to redundancy. Between 2002 and 2005 severance payments amounting to around EUR 260 million were distributed to redundant workers. As Serbia entered the advanced stage of transition, additional tens of thousands of workers had become redundant. According to a survey involving representatives from all 25 branch offices of the NES, the estimated number of redundant workers in 2006 was more than 43,000. Moreover, there was another wave of redundant workers in 2007 numbering as many as 60,000. The privatization of socially owned firms took much longer than expected, and it was extended until The process of restructuring of public enterprises will last even longer. 5

6 Therefore the annual inflow of redundant workers remained significant even after This resulted mainly from post-privatization restructuring and downsizing. 2 In order to reverse the unfavourable labour market trends, it was necessary to complement the ongoing enterprise restructuring and privatization process with a pro-active employment policy with active labour market programmes (ALMPs) at its core. The existing system of public employment services, although significantly improved, offered insufficient incentives for the unemployed actively to seek work and provided only a limited range of active measures for those who are striving to improve their employability ABOUT THE PROJECT The real challenge for the NES, with its limited financial resources, was to offer a comprehensive package of services; programmes that are oriented to labour demand and linked to real workplaces, with expanded outreach to cover and carefully target beneficiaries. In this context, the Severance to Job project was an innovative active labour market programme (ALMP) tailored to address one of the most pressing problems of the Serbian labour market during the final stages of enterprise restructuring and privatization in the years and possibly beyond. It has been observed and documented that workers who received severance payments within the past downsizing programmes in Serbia tended mostly to spend their cash on additional consumption or for other non-productive purposes. Bearing in mind that unemployment compensation and social protection schemes, together with mass severance programmes, need to be accompanied and ideally replaced by effective active programmes, the 2 Privatization deals frequently protect workers from dismissal by the new owners for only one to three years. As a result, redundancies directly connected with the mass privatization programme and public sector restructuring continued until In Serbia, only 0.28% of GDP is spent on active labour market measures, which is significantly lower than in the EU where active measures account for 0.9% of GDP. Furthermore, unemployment benefits and other passive measures (excluding severance payments) account for more than 90% of the NES s total expenditures. 6

7 STJ project offered redundant workers an attractive menu of opportunities for productive investment of their severance pay, i.e. to turn what is perceived to be a typical passive measure into an active one. The STJ project was implemented by the United Nations Development Programme s (UNDP) Central Project Team (CPT) in close collaboration with the National Employment Service (NES) and the Ministry for Economy and Regional Development (MoERD), being fully incorporated into the activities of MoERD and NES. Initial funds for the Project implementation were provided by the Austrian Development Agency (ADA). In addition to ADA funding, the Government of the Republic of Serbia substantially supported the Project in the period The project included two distinctive components: (i) re-employment of redundant workers through their new employment with the new employer; and (ii) re-employment of redundant workers through their self-employment. Within the first component, re-employment with the new employer, the redundant workers were provided with the possibility to invest their severance pay or part of it: (a) to fund equipment for their new working place; (b) to acquire additional training required for their new job; and (c) to cover the costs of obligatory social security taxes. The fourth modality of the project, within the second component, included additional funding for the most vulnerable redundant workers with the aim to support their future re-employment. 2.3 PROGRAMME PARTICIPANTS Project participation included signing a contract with 24-month duration. Out of the total 1,947 participants, 416 found jobs with new employers and 1,531 used the severance pay for selfemployment. A total of 1,560 beneficiaries were interviewed face-to-face by professionals from 7

8 the Centre for Free Elections and Democracy (CeSID) in The contractual obligation had ended for 484 out of them (31%). The latter constitutes the size of our treatment group. Geographical coverage of the project was nation-wide, both for members of the work force (workers) and firms (employers). The regional distribution of beneficiaries according to the NES Branch Offices is presented in Table 1. Labour force members eligible for participation met several criteria: (i) they were made redundant by their former employers not earlier than September 2006; (ii) they received or were about to receive a severance payment; (iii) beneficiaries expressed readiness to invest their severance payment into their own re-employment. Firms eligible for participation in the project were selected based on following criteria: (i) they were privately owned; (ii) firms demonstrated that redundant workers would be hired through the project in order either to increase total employment in the firm or to replace retirees and those who have left voluntarily. In what follows, we shall present the main characteristics of the programme participants. Out of the total, 61% are male and 39% female beneficiaries. The average age of beneficiaries is 47.6 years (more than 40% of them are over 50 years of age). Most beneficiaries have completed secondary vocational school lasting 3 or 4 years (42.9% and 28.7% respectively). Families of beneficiaries are most frequently the owners of apartments or houses where they live (91.5%), and 63% of them live in the broader downtown or outskirt neighbourhoods. Up to 50.49% of interviewees describe their financial situation in 2009 as bad or very bad; 34.34% evaluate their situation as worse or much worse compared to the period prior to job loss, whereas 23.35% evaluate it as improved, to some extent (better or much better); 56% of beneficiaries express mostly satisfaction with life in general, while some 38% express mostly dissatisfaction. 8

9 Among all project beneficiaries, 90% are employed, 8% are unemployed and 2% are inactive. Among those whose contracts have ended, 73% are employed, 21% are unemployed and 6% are inactive. The average working experience of beneficiaries is 23 years, largely acquired at their last place of work (prior to beneficiary being named redundant or company bankruptcy). The largest number of interviewees previously worked in the manufacturing industry (47%), followed by trade (11.7%). A similar trend is followed during the project. When it comes to salary patterns, 44% of interviewees did not receive salaries regularly at their previous job and 37.67% had to find additional jobs. During the project, 71% of beneficiaries received their salary regularly and 23% needed additional jobs. Among project beneficiaries whose contracts have ended, 65% is still employed at the job obtained within the project. Among those whose contracts are still ongoing, 65% plan to stay on at the work they have begun during the project. The largest number of beneficiaries learned about it from NES counsellors (68%), followed by friends and colleagues (16%) and the media (12%). The characteristics of project participants within the STJ project s two distinctive components (re-employment and self-employment) are as follows: Beneficiaries in new employment. Out of 275 participants, 63 are no longer at the work they began under the project (for 50 of them the contractual obligations have expired). Main reason quoted for being fired is cessation of job (25%) or some other reason (42%). Beneficiaries in self-employment. Out of 1,174 participants, 206 stopped working at the commenced jobs, while 130 said their contractual obligations expired in the meantime; 90% continued to run their business, while 9% were unemployed after they closed down the business; 30% of new business owners employed other workers as well, but in most cases (90%) only 1 or 2. As main problems in running a business, project participants 9

10 cited problems related to the market (39%), high taxes (20%) and problems of collection of payments once the job is finished (17%). 3 METHODOLOGY In what follows, we describe the data available for STJ programme evaluation, the main methodological problems to construct the treatment and control groups in context of the programme, and the solution concepts we apply. 3.1 EVALUATION PROBLEM For a correct assessment of programme effects, it is important and necessary to compare the comparable (Heckman et al., 1999). This means that we need to compare the programme participants the so-called treatment group only to those non-participants who could have participated in the programme as well, i.e., had an equal chance to be selected for participation in the programme as those who were actually treated. A comparison group designed in this way is referred to as control group in the evaluation literature. Evaluation generally has to deal with a serious problem if the effects of participating in a specific programme should be quantified compared to that what would have been without doing so. This problem naturally arises because it is impossible to observe same individuals in two different states of nature (participation and non-participation) at the same time and place. Therefore, it is the principle task of any evaluation study to find a credible estimate for the counterfactual state of nature. There are basically two methods to estimate the counterfactual situation: randomized experiments and non-experimental (also called quasi-experimental) methods. In principle, randomized experiments provide the easiest solution to recovering the desired counterfactual. In 10

11 randomized experiments, individuals eligible for participation are randomly assigned to a treatment and control group. Since these groups differ from each other (on average) neither in observable nor in unobservable characteristics and the control group can be considered as identical to the treatment group, the average difference in outcomes between the two groups provides a simple answer to the counterfactual question. 4 While the STJ programme has not been designed as a randomized experiment, the data for the evaluation analysis were constructed to mimic an experimental situation ex-post. For each member of the treatment group, a matched partner with the same observable characteristics was drawn from the National Employment Service (NES) registers. The intention was to create a control group which would resemble the treatment group as much a possible. The individual characteristics available for this matching procedure were labour market characteristics, such as end of employment contract because of surplus of workers or firm bankruptcy. The matches were based on only very few controls, due to a lack of comprehensive or up-to-date employment registries. Thus formed control group consisted of 977 interviewed individuals. However, if additional characteristics did play a role for determining the chances to participate in the STJ programme, one could not consider the treatment and control groups as identical like in a randomized experiment. In this case, a simple comparison of mean outcomes in the two groups would not be sufficient. Moreover, the substantial differences between the number of planned and accomplished interviews in both groups could make this approach useless since the selection of the control group was based on planned rather than on accomplished interviews. 4 Often randomized experiments are politically or socially not feasible. Moreover, they are in practice not entirely free of complications: see Heckman and Smith (1995) for a discussion of the advantages and disadvantages of the randomization approach. 11

12 To assess whether programme participation can be regarded as quasi-random in our data, we need to compare the characteristics of participants and non-participants. Initially, we perform statistical tests of the hypothesis of random assignment to participation, i.e., random differences between the treatment and control group. In particular, we test statistically whether the means of important socio-demographic characteristics and labour market outcomes are significantly different between treatment and control group. If the hypothesis of random assignment is rejected, it may be misleading to compute net effects of the programme as the difference in the average outcomes between participants and non-participants. Table 2 shows the number of observations included in the treatment and control groups. Table 3 shows the t-test results 5 which test random differences between the treatment and control groups. 6 The tests indicate that the treatment and comparison groups are significantly different in the means of important characteristics. More precisely, it appears that the treatment group is substantially better positioned in the labour market than the control group. Members of the treatment group are more likely to be employed and less likely to be unemployed or inactive. They differ according to the reasons why they lost a job or whether they had an additional job next to the previously held job. Moreover, treated individuals tend to be on average younger, mostly men, are more likely to be married, have more household members, more children under 18, and less retired household members. Additionally, treated individuals tend to have elementary school only and live outside of the city centre. Further, subjective estimation of well-being is relatively worse among the members of treatment in comparison to the control group. In general, these differences are statistically significant. As these characteristics will probably positively affect employability, one would expect that a simple comparison of mean outcomes between 5 Where appropriate, we report χ 2 -test. 6 The sample size between treatment and control groups vary due to missing observations on one of the covariates. 12

13 participants and non-participants overestimates the impacts of the STJ programme on labour market outcomes. Based on these findings we conclude that the hypothesis of random differences between the treatment and comparison group can be rejected. Therefore, we have to apply a nonexperimental method accounting for the individual probabilities of programme participation, in order to construct proper control group and to calculate unbiased impacts of participation in the STJ programme. 3.2 MATCHING APPROACH Nowadays the most common technique to solve the evaluation problem when the participants and non-participants are not randomly assigned to a labour market programme is the matching approach. It mimics a randomized experiment ex post by constructing a control group that resembles the treatment group as much as possible. In particular, after matching the members of the control group, considering their observable characteristics, they have a probability to be selected for participation in the programme comparable to the members of the treatment group. We observe in our data many variables presumably influencing both the selection into the programme as well as labour market outcomes. Hence, it appears reasonable to assume that selection into the programme and labour market outcomes are independent conditional on these observables. 7 Under this assumption we apply one-to-one nearest neighbour matching with replacement. This method consists of two steps: (1) an estimation of the individual probabilities to participate in the programme or not, depending on a set of observable characteristics; (2) matching of participants and non-participants on the basis of these estimated probabilities. Oneto-one matching implies that each member of the treatment group is matched with a single 7 This is the so-called conditional independence assumption, which ensures that the matching approach indeed mimics a randomized experiment ex post. 13

14 member from the control group. Furthermore, nearest neighbour matching implies that the pairs are matched according to the minimum distance of the predicted probabilities of programme participation. Finally, matching with replacement means that the data on individuals in the control group may be used more than once, provided that they are the nearest neighbour of an individual in the treatment group. We begin with a discussion of the determinants of programme participation to be estimated in the first step. The impact of individual characteristics on the likelihood of participating in the STJ programme is estimated employing standard (probit) regressions on the treated and non-treated. The estimated coefficients reveal insights about the factors influencing the selection into the treatment. But they may also capture factors driving attrition from the survey, i.e. factors explaining differential non-response rates in the respective treatment and control groups. Our preferred specification of the regression model includes a full range of explanatory variables, which are defined in Table 3. 8 Table 5 exhibits the probit estimation results (estimated coefficients and marginal effects), 9 underlying the propensity scores for the various treatments. The results basically confirm the impression from the descriptive statistics. The estimated age and gender patterns imply that programme participation rates are lower for women and younger participants and decrease for households with more retired household members. Being married, having more household members (of all types) and more children under 18 generally increase the probability of treatment. Moreover, the probability of treatment is higher if a person does not own 8 9 We have tried several specifications of the probit model. The results did not change qualitatively. Our chosen specification appears to deliver the best overall predictions of programme participation rates. In technical terms, the reported coefficients represent so-called marginal effects. Marginal effects reveal the percentage change of the programme participation rate in response to a one percentage point change in the explanatory variable, respectively the percentage change of the programme participation rate if a dummy variable changes from value zero to value one, holding the value of all other explanatory factors constant. 14

15 a house, has low education, does work in the place of living, works in transport, public administration or health and social work sectors. On the other hand, the more members of household who are able to work, the person is less likely to participate in the STJ programme. The opposite is true if the person had a regular salary during previous job or if a person held a second job during previous main job. Considering the statistical significance of the above mentioned general effects, probit estimates suggest statistically significant effects for the following covariates. Estimated age and gender patterns imply that programme participation rates are lower for women and higher for older participants. Considering household composition, project participation decreases for households with more retired household members and with more members of household able to work. Labour market indicators suggest that the probability of treatment is higher if a person does work in the place of living and if a person had an additional job during previous job. When it comes to a particular industry sector, the probability of treatment is higher if a person works in transport, public administration or health and social work industry sectors. In sum, the probit results suggest that the STJ programme has reached its intended target group very well. This is a group of older labour force members, whose life-long employment contracts ended because they were in the group of surplus workers or due to firm bankruptcy, and those with low educational levels. However, this interpretation should be treated with some caution, given that our sample is presumably not representative of the entire population in Serbia. Furthermore, robust econometric analysis of the STJ project effects should ideally be performed six months after all participants have exited the project. This is probably a main drawback of our findings, since majority of the STJ project participants are still undergoing the treatment, hence are not present in our sample, thus making sample sizes that we work with comparatively small. 15

16 In a second step, we implement the one-to-one nearest neighbour matching principle by using the estimated parameters on display in Table 5 to predict the probability to participate in a treatment the so-called propensity score for each individual in the treatment and comparison groups. The propensity scores are used to match participants with comparable non-participants. For each treated individual, we look for the one individual among the non-participants who is the closest neighbour in terms of the predicted probability of being treated. In other words, for each pair of participant and non-participant the absolute difference in terms of the estimated propensity to participate in a certain treatment is minimized. Because the sample sizes, especially of the non-participants, are relatively small, we opt for matching with replacement. This means we allow for the possibility that different participants are matched with the same non-participants. To ensure that the matched pairs have reasonably similar probabilities to be treated, we exclude participants for whom the predicted probability to be in the programme is larger than for any individual in the comparison group. In this way we achieve so-called common support. We now illustrate the outcome of the matching procedure. Figures 1 and 2 show the distributions of the propensity scores for participants and non-participants in the STJ programme, obtained from the probit estimates. The figure depicts the number of observations in twenty intervals of width 0.05 in the possible range from 0 to 1. Obviously, the distributions differ between participants and non-participants. While most of the non-participants exhibit propensity scores close to 0, the majority of participants exhibit propensity scores of 0.5 and above. It seems that the individuals surveyed as potential controls for the evaluation exercise are not randomly selected with regard to the characteristics determining programme participation. Overall, the non-participants tend to have characteristics that make them systematically less likely 16

17 to be self-selected for participation in the STJ programme compared to individuals who received the treatment. To form a proper control group for the evaluation of programme impacts, one needs to exclude those individuals among the non-participants who appear to be too different in terms of their propensities to receive the treatment. Table 6 shows the matching quality. Among the programme participants, 8 have a higher propensity score than the individual with the highest estimated propensity score among the nonparticipants. Hence these individuals are off support and need to be excluded for the computation of the ATT. After forming the matched pairs, a suitable way to assess the matching quality is comparison of the standardized bias before matching, SB b, to the standardized bias after matching, SB a. The standardized biases are defined as where is the mean (variance) in the treated group before matching and is the analogue for the comparison group. and are the corresponding values after matching (Rosenbaum and Rubin, 1985). Following the example of Sianesi (2004) we also re-estimate the propensity score on the matched sample to compute the pseudo-r 2 before and after matching. These measures (see Table 6) suggest that the quality of our matching procedures is quite satisfactory. The standardized bias of the matched sample is markedly smaller than that of the unmatched sample. Likewise, the pseudo-r 2 after matching are fairly low and decrease substantially compared to before matching. This is what we should expect considering that after matching, there should not be any systematic differences in the distribution of covariates between participants and matched non-participants. This test of the matching quality makes us confident to 17

18 estimate meaningful treatment effects on the basis of nearest neighbour matching with replacement. If the matching approach is successful in mimicking a randomized experiment, any differences in observable characteristics between the treatment and control groups should disappear. Table 7 summarize the characteristics of the matched programme participants and nonparticipants. They indicate that the constructed treatment and control groups indeed have basically identical socio-demographic characteristics. 10 This shows that our matching approach have successfully imitated a randomized experiment, which will allow evaluating programme impacts by comparing mean outcomes between the treatment and control groups. 4 PROGRAM IMPACTS In the following we first adopt the conventional perspective on evaluation of active labour market programs and study the causal impact of STJ on labour market outcomes, namely unemployment probability, employment probability, non-activity, average net wage in In a second step, we will look at the subjective well-being variables at the core of our interest (subjective evaluation of financial situation in 2009, current subjective evaluation of financial situation as compared to the situation before job loss, chances to find a job, subjective welfare and life satisfaction). 4.1 LABOUR MARKET OUTCOMES The survey data do not trace individuals employment histories. Hence our outcome variables are based on the labour market status at the time of the interview. In particular we look at four different labour market states: (i) unemployment, (ii) employment in a regular job 10 After matching, individuals who do not own a house, those with university education level, those with more years of work experience and individuals who lived in the place of work are somewhat under/overrepresented among the participants in the complete STJ program. These exceptions are altogether negligible. 18

19 including self-employment, (iii) non-activity, and (iv) average net wage in Table 7 summarizes the estimated ATT for our four different labour market outcomes. In the following, we focus on the average treatment effects on the treated considering the probability of, for example, employment at the respective survey date. The average treatment effect on the treated (ATT) measures the average effect of the intervention on the group of individuals who participated in the programme. For example, in the present context the ATT represents the difference between the actual employment rate of participants post programme and the counterfactual employment rate of participants supposing they would not have received the treatment. Importantly, the ATT captures the causal effect of the programme on the analysed outcome. (i) Our point estimates suggest that STJ programme participation is generally associated with a higher employment probability and this effect is statistically significantly different from zero. The findings suggest that participation in the STJ programme increases the probability of being employed by about 44 percentage points, compared to a situation of not participating in the programme. (ii) Participation in the programme is generally associated with a lower unemployment probability. The findings suggest that participation in the STJ programme decreases the probability of being unemployed at the survey date by about 42.5 percentage points. This effect is also statistically significantly different from zero. (iii) Estimated effect of the programme participation decreases the probability of being non-active. However, this effect is relatively small (around 2 percentage points) and not statistically significantly different from zero. 19

20 (iv) While the samples of matched participants and non-participants are the same as before, average net wage data in 2009 are missing for a relatively large number of observations. 11 This means that, due to a poor quality and a relatively small number of observations on this covariate, the estimated programme effects on wages are even less robust than the ATT on unemployment and employment probabilities. 4.2 SUBJECTIVE WELL-BEING Even if an active labour market program does not immediately raise employment probabilities of participants, a social planner may find it beneficial if it manages to improve the individual welfare of the target group. The survey data collected in connection with the STJ programme provide us with the unique opportunity to study programme impacts also on various dimensions of life that may serve to approximate individual well-being or happiness. In some questions in this section, individuals were requested to compare their situation at the time of the interview with that before the STJ programme came into effect, and had to judge whether their situation has strongly or somewhat improved, has stayed more or less the same, or has strongly or somewhat deteriorated. In detail, the survey requested self-assessment of financial situation in 2009, subjective evaluation of current financial situation as compared to the situation before job loss, evaluation of chances to find a job, self-assessment of welfare and life satisfaction. 11 When evaluating multiple outcomes using psmatch2 command in Stata, it reduces to the minimum common number of observations with non-missing values on ALL outcomes, because otherwise the matching weights will not sum to the right number. Due to a large number of missing variables, programme effects on average net wages in 2009 are separately estimated. Estimates are generated using 88 treatment and control participants matched pairs. 20

21 In our subsequent analysis, we apply our matching approach to the subjective data. As outcome variables, we define dummy variables that take the value of one if individuals report that their personal situation has strongly or somewhat improved, and take a value of zero otherwise. In this way, the ATT measures the change in the percentage share of individuals judging their personal situations as improved because of program participation. Table 8 summarizes our findings. The general impression based on the point estimates is that programme participation has improved the personal situation with regard to all aspects of life considered, except for selfassessment of financial situation in 2009, which stayed the same. In contrast to the impact on labour market outcomes, the programme effects do not seem to be substantial, such that the estimated ATT are not statistically significant. Among the individuals participating in the programme, the rate of those reporting an improvement on current financial situation as compared to the situation before job loss is 7 percentage points larger than among non-participants, and the effect is not statistically significant. When it comes to self-assessed chances to find a job, the estimated effect is 5 percentage points larger than among non-participants, and the effect is also not statistically significant. 12 In the reminder of this section, individuals were requested to compare their situation at the time of the interview with that before the STJ programme came into effect, in the last 4-5 years, and had to judge whether their situation has somewhat ( winner ) or not at all improved ( looser ). The estimated effect of the programme on this outcome variable is 7 percentage points larger than among non-participants, and the effect is not statistically significant. Lastly, the ATT concerning 12 When evaluating multiple outcomes using psmatch2 command in Stata, it reduces to the minimum common number of observations with non-missing values on ALL outcomes, because otherwise the matching weights will not sum to the right number. Due to a large number of missing variables, programme effects on chance to find a job are separately estimated. Estimates are generated using 19 treatment and control participants matched pairs. 21

22 life satisfaction is much smaller (around 2 percentage points), and also not statistically significant. Taken together, the positive programme effects considering individuals subjective assessment of conditions of life appear to be smaller than the programme impacts when considering their objective labour market status (see Table 8). The impacts we find are positive, however the effects are small and not statistically significantly different from zero. 5 CONCLUSIONS This report evaluates the net impact of the Severance to Job programme. On the basis of a comprehensive data set covering almost the universe of programme participants as well as a comparison group of persons who ended their employment contracts because of redundancy programmes or firm bankruptcy, we employ statistical tools for programme evaluation designed to calculate the average treatment effect on the treated. The treatment effect captures the causal effect of the programme. It shows how the analysed outcome changes for programme participants, compared to a situation where they would not have received the treatment. In this study, treatment effects are assessed across a range of labour market outcomes (unemployment probability, employment probability, non-activity, average net wage in 2009), as well as the range of subjective well-being outcomes (subjective evaluation of financial situation in 2009, current subjective evaluation of financial situation as compared to the situation before job loss, chances to find a job, subjective welfare and life satisfaction) between those who participated in the STJ programme and those who did not. We show in this report significant effect of programme participation on main labour market outcomes (employment and unemployment probabilities) and no significant effect on subjective improvement in personal life (e.g., self-assessed well-being and life satisfaction). 22

23 The treatment effects estimated for the Severance to Job programme overall allow only tentative conclusions, due to the small scale of the programme and certain limitations in the accomplished survey. Firstly, our sample is presumably not representative of the entire population in Serbia. Secondly, we observe the data at the comparatively short time period after treatment. The evidence in the programme evaluation literature points to the fact that programme effects may require substantial time to fully unfold. Thirdly, robust econometric analysis of the STJ project effects should ideally be performed six months after all participants have exited the project. This is probably a main drawback of our findings, since majority of the STJ project participants are still undergoing the treatment, hence are not present in our sample, thus making sample sizes that we worked with comparatively small. Concluding, the Severance to Job programme and the subsequent evaluation allow for a better-informed policy making in labour and employment field in Serbia. 23

24 REFERENCES Bonin, H. and U. Rinne (2006). Beautiful Serbia, IZA Discussion Paper, Dmitrijeva, J. and M. Hazans (2007). A Stock-flow matching approach to evaluation of public training program in a high unemployment environment, Labour: Review of Labour Economics and Industrial Relations, 21 (3), Heckman, J.J., R.J. LaLonde and J.A. Smith (1999). The economics and econometrics of active labor market programmes, in: O. Ashenfelter and D. Card (eds.), Handbook of Labor Economics, 3, Elsevier, Amsterdam. Heckman, J.J. and J.A. Smith (1995). Assessing the case for social experiments, Journal of Economic Perspectives, 9 (2), Leuven, E. and B. Sianesi (2003). "PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing". This version Lubyova, M., and J. C. van Ours (1998). The Effects of Active Labor Market Programs on the Transition Rate from Unemployment into Regular Jobs in the Slovak Republic, Tilburg Center for Economic Research Papers, O'Leary, C. J. (1998). Evaluating the Effectiveness of Active Labor Market Programs in Hungary, Upjohn Institute Technical Report, Rodriguez-Planas, N., and J. Benus (2006). Evaluationg Active Labor Market Programs in Romania, IZA Discussion Paper, Rosenbaum, P., and D. Rubin (1983). The Central Role of the Propensity Score in Observational Studies for Causal Effects, Biometrika, 70,

25 Sianesi, B. (2004). An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s, The Review of Economics and Statistics, 86 (1), ABBREVIATIONS ADA... Austrian Development Agency ALMP... active labour market programme ATT... average treatment effect(s) on the treated CeSID... Centre for Free Elections and Democracy CPT... Central Project Team EUR... Euro MoERD... Ministry for Economy and Regional Development NES... National Employment Service STJ... Severance to Job UNDP... United Nations Development Program 25

26 TABLES AND FIGURES Table 1: Regional distribution of beneficiaries according to the NES Branch Offices Branch office New employment Self-employment Total number of beneficiaries Beograd Bor Čačak Jagodina Kikinda Kragujevac Kraljevo Kruševac Leskovac Loznica Niš Novi Pazar Novi Sad Pančevo Pirot Požarevac Prijepolje Prokuplje Šabac Smederevo Sombor Sremska Mitrovica Subotica Užice Valjevo Vranje Vršac Zaječar Zrenjanin Total number of beneficiaries 416 1,531 1,947 Source: UNDP Table 2: Definitions of treatment and control groups. Type of treatment Size of treatment group Size of control group Participation in the STJ programme 484 obs. 977 obs. Source: CeSID, own calculations 26

27 Table 3: Socio-demographic characteristics of treatment and control groups (comparison of means). Socio-demographic characteristics Treatment group Control group Significance obs. mean obs. mean t-test p-value Age * ln(age) ln(age) Gender *** Married *** # Members of household *** # Members of household able to work # Children under *** # Retired household members *** House ownership status Education: less than primary school ** Education: primary school Education: secondary/vocational Education: university level Place of living *** # Years of work experience # Years of work experience on the previous job Reason for losing previous job ** Place of living during previous job Salary on previous job Additional job during previous job *** Outcome variables Employed *** Unemployed *** Non-active *** Average net wage in ** Financial situation in 2009 (estimate) Current financial situation (estimate) *** Chances to find a job Winner or looser? *** Life satisfaction *** Source: CeSID, own calculations Notes: χ2 test 27

28 Current subjective evaluation of financial situation as compared to the situation before job loss. Difference statistically significant at the 99 (***), 95 (**), and 90 (*) percent level. 28

29 Table 4: Explanatory variables included in the preferred specification of the regression model. Gender ln(age) ln(age) 2 Married Name of variable Survey question Description What is your gender? What is your exact age? What is your marital status? 1: Male 2: Female Logarithm of age (in years) Logarithm of age (in years) squared 1: If married 0: Otherwise # Members of household Number of members of household? Number: 1-11 # Members of household able to work Number of members of household who are Number: 0-7 unemployed and able to work? # Children under 18 Number of children under 18 in household? Number: 0-5 # Retired household members Number of retired household members? Number: 0-3 House ownership status Education: less than primary school Education: primary school Education: secondary/vocational Education: university level Place of living What is your house ownership status? What is your highest educational level? What is your place of living? # Years of work experience How many years of work experience? Number: 1-54 # Years of work experience on the previous job Place of living during previous job Reason for losing previous job Industry sector of previous job Salary on previous job 1: Ownership, without credit/mortgage 2: Ownership, with credit/mortgage 3: Rental agreement 1: If without education, up to 4 years of primary school, 5 to 7 years of primary school 0: Otherwise 1: If primary school 0: Otherwise 1: If vocational/secondary special school (3 years), vocational/secondary special school (4 years), gymnasium 0: Otherwise 1: If higher school or university degree 0: Otherwise 1: City centre 2: Larger city centre 3: Suburbs 4: Rural area How many years of work experience on Number: 0-45 the job which precedes the current one? During previous job, did you work in the 1: If answer yes same place where you lived? 2: If answer no 1: Surplus of workers Why did you loose the previous job? 2: Upon agreement, with severance pay 3: Company bankruptcy, without severance pay 1: Agriculture, hunting and forestry 2: Fishing 3: Mining and quarrying 4: Manufacturing 5: Electricity, gas and water supply 6: Construction 7: Wholesale and retail trade; repair of motor vehicles, motorcycles and personal and household goods What was industry sector of previous job? 8: Hotels and restaurants 9: Transport, storage and communication 10: Financial intermediation 11: Real estate, renting and business activities 12: Public administration and defence 13: Education 14: Health and social work 15: Other community, social and personal service activities 16: Other Has salary on previous job been paid out 1: If answer yes regularly? 2: If answer no 29

30 Additional job during previous job 1: If answer yes (same job) During previous job, was it necessary to do 2: If answer yes (different job) an additional job? 3: If answer no Source: CeSID, own calculations Table 5: Probit estimation results (coefficients and marginal effects). Estimation results Significance Variable Coefficient Marginal Effect p-value Gender ** ln(age) * ln(age) * Married # Members of household # Members of household able to work *** # Children under # Retired household members *** House ownership status Education: less than primary school Education: secondary/vocational Education: university level Place of living # Years of work experience # Years of work experience on the previous job Reason for losing previous job ** Place of living during previous job * 4: Manufacturing : Electricity, gas and water supply : Construction : Wholesale and retail trade : Hotels and restaurants : Transport, storage and communication * 10: Financial intermediation : Real estate, renting and business activities : Public administration and defence ** 13: Education : Health and social work * 15: Other community, social and personal service activities : Other Salary on previous job Additional job during previous job ** 30

31 # Observations 412 Log-pseudolikelihood Pseudo R Source: CeSID, own calculations. Notes: Difference significant at the 99 (***), 95 (**), and 90 (*) percent level. Figure 1: Distribution of propensity scores and common support. Source: CeSID, own calculations Figure 2: Distribution of propensity scores and common support. 31

32 Source: CeSID, own calculations 32