Data Warehouse Monitoring in the Public Employment Service: Austria Statements and Comments

Transcription

1 Data Warehouse Monitoring in the Public Employment Service: Austria Statements and Comments Stephen Lissenburgh Employment Research Policy Studies Institute This paper comments on Data Warehouse monitoring (DWH) in the Public Employment Service in Austria and considers its applicability to the UK. The paper is divided into three parts: 1. A brief assessment of the economic circumstances, policy context and institutional background in the UK that are relevant to the policy under discussion. 2. An assessment of the potential transferability of DWH, or elements of it, to the UK. 3. A brief note on important issues and future developments that are relevant to the policy area. 1 Economic circumstances, policy context and institutional background in the UK Economic circumstances: There has been a substantial and sustained decline in the UK s unemployment rate over the last decade. After rising to 10.6 per cent in 1993, the ILO unemployment rate has fallen in each year since then and was down to 5.1 per cent in This fall in the unemployment rate has been associated with an increase in the employment rate, with the proportion of working age people in employment rising from 70.3 per cent in 1993 to 74.7 per cent in Active labour market policies (ALMPs) aimed at encouraging unemployed jobseekers into work have helped to bring about these improvements and will continue to be a key feature of a labour market strategy to improve the overall employment rate, but further increases in the employment rate are only likely to occur if progress can be made in reducing the rate of economic inactivity. The proportion of working age people who are economically inactive has shown virtually no tendency to decline over the last decade, falling only marginally from 21.3 per cent in 1993 to 21.2 per cent in Policy context: It is against this backdrop that policies have been introduced to encourage lone parents on Income Support (such as New Deal for Lone Parents) and clients claiming sickness and incapacity benefits (such as New Deal for Disabled People and Incapacity Benefit Reform Pilots) to become economically active. Such policies are likely to be an increasingly important feature of the government s strategy to improve the overall rate of employment over the next 5-10 years. Institutional background: In order to support and effectively monitor and evaluate such policies, it will be necessary for the Department for Work and Pensions (DWP) in the UK to continue and indeed enhance the process it initiated from the late 1990s of developing a sophisticated and comprehensive set of administrative databases. In this respect, it could engage constructively with the experiences of the Bundesministerium fur Wirtschaft und Arbeit (BMWA) in Austria, which through its has developed a complex and wide-ranging Data Warehouse system in recent years. 1 All figures in this paragraph are from Labour Force Survey: Summary, , Office for National Statistics. 1

2 2 Potential transferability of DWH to the UK As a prelude to assessing the potential transferability of DWH, or parts of it, to the UK, it is necessary to describe the main features of DWH and to explain how these compare with the UK s existing system of administrative databases in the PES Main features of DWH 3 DWH, introduced in 1998, combines data from two main sources: a range of PES administrative databases, which provide, among other things, individualised information on unemployment benefit spells and participation in labour market support schemes; and the insurance database of the Federation of Austrian Social Insurance Institutions (HV), which comprises all the insurance-related data of health, work accident and pension insurance schemes of the various Austrian social insurance institutions into one database. As such, it includes information on employment relationships, pensions and other insurance relationships covering roughly 98 per cent of people living in Austria. DWH is a tool which, as well as providing detailed background information about recent developments in the labour market, enables Labour Market Policy (LMP) monitoring to be carried out in three main ways: 1. Follow-up monitoring, which analyses the labour market experiences of individuals who have participated in PES-driven LMP measures. It uses data from before and after programme participation to assess the effectiveness of LMP measures. 2. Career monitoring, which analyses the development of the labour market as a whole and focuses on the flows underlying observed changes in stocks. It covers all individuals participating in a social insurance scheme in Austria. 3. Enterprise monitoring, which deals with the human resources situation of individual companies and industrial sectors as a whole. It includes all employers registered by HV in Austria. 2.2 The British equivalent of DWH: the Work and Pensions Longitudinal Study (WPLS) The development of DWH in Austria has been mirrored by similar developments in the UK since The introduction of a number of innovative ALMPs, especially the New Deals, was accompanied by the development of administrative databases to assist in the monitoring and evaluation of these programmes (Hall and Reid, 1998). The New Deal Evaluation Database (NDED) contained information on the socio-demographic characteristics of all individuals who entered one of the New Deal programmes. This included age, gender, ethnicity and region. The NDED could be linked to the Labour Market System, so that information could be matched in on the unemployment benefit claiming 4 of individuals both before (back to 1995) and after participation in a New Deal programme. This linked dataset enabled an extensive programme of scientific evaluation of the New Deals to be carried out, 2 The PES in the UK is called Jobcentre Plus. 3 The information in section 2.1 is derived from Buzek, Edlinger, Friedenthal, Hochrainer, Schmitzberger, Tauer, and Zauner (2004) and Mahringer (2004). 4 The main form of unemployment benefit in the UK is Jobseekers Allowance (JSA). 2

3 especially in the period from 1998 to 2001 (see, for example, Blundell, Costa Dias, Meghir and Van Reenen, 2001; Dorsett, 2001; Lissenburgh, 2000, 2001a, 2001b). It thus enabled work to be carried out that was similar to the follow-up monitoring made possible by DWH. As was stated in Part 1, these ALMPs were partly responsible for the increase in the UK employment rate that has occurred in recent years (Riley and Young, 2001a and 2001b) but further increases in the employment rate are only likely to occur if progress can be made in reducing the rate of economic inactivity. Policies to encourage lone parents and clients claiming sickness and incapacity benefits to become economically active have been introduced in this context. The greater emphasis on policies to assist the economically inactive into work since 2001 has been accompanied by a further development of DWP s administrative databases, with the creation of the Master Index (MI). The MI pools data on all the main categories of welfare benefit recipients 5 and all of the major ALMPs, so that analyses can be carried out on movements from one benefit to another as well as movements on and off a particular benefit. From the account above, it is clear that there has been a marked improvement in the scope and comprehensiveness of DWP administrative databases since the late 1990s and this has made possible a wide range of work in monitoring and evaluation. The greatest weakness of these databases, however, was the absence of data on employment outcomes. The Work and Pensions Longitudinal Study (WPLS) now fills this gap. The WPLS was introduced in January It combines the MI as described above with employment records from the Inland Revenue (IR). The WPLS is organised into three databases: 6 1. The benefit/programme and employment spells database holds all the benefit/programme information and all the employment information for those who are or have been DWP clients 2. The off benefit/programme into work database is a subset of database 1 and contains all leaving benefit/programme records for an individual and, where this occurs, a linked employment record. Taken together, databases 1 and 2 would be able to sustain very similar types of follow-up monitoring to that which is possible with DWH. 3. The employment only database contains all employment records for those for whom there is no DWP client record. When combined with databases 1 and 2, it would make possible career monitoring similar to that which is possible with DWH. Since its introduction earlier this year, WPLS has been used for both monitoring and evaluation purposes by DWP researchers and will be made available to external contractors where appropriate. Examples of current usage include analysis of the movements of ethnic minorities from benefits to work and the longer-term impacts of Jobcentre Plus; examining work history and its impact on programme performance; and examining long-term work patterns for those who took part in New Deal for Lone Parents. 7 5 The major change here is that Income Support and Incapacity Benefit recipients can be analysed in the same database as JSA recipients. 6 A more detailed description of WPLS can be found in Department for Work and Pensions (2004). 7 Further examples of current and planned usage of WPLS can be found at 3

4 2.3 Scope for transferability of DWH to the UK It is clear from the description in sections 2.1 and 2.2 that in Austria and the UK there has been a parallel trend towards the development of complex administrative datasets that combine information from the public employment service with information on labour market outcomes. When considering the potential for transferring DWH, or elements of it, to the UK, this parallel development acts both to make possible a transfer but also to discourage it. On the one hand, the fact the countries have been moving in broadly the same direction means the UK should be receptive to a labour market monitoring system that in some respects would represent an enhancement and logical extension of WPLS. On the other hand, WPLS is already well developed and is likely to develop further in the near future: in some respects, it would seem to have relatively little to learn from DWH and is possibly superior in certain aspects. These points are brought out in more detail below in a discussion of the potential transferability of DWH in relation to each of its three main functions. In each case, the degree to which WPLS would be enhanced by the importation of elements of DWH is considered, but also whether such a transfer is likely to be feasible given the UK s institutional and legal characteristics. Follow-up monitoring This is the area with least need for transferability because the WPLS is already able to sustain most of the analyses that could be carried out with DWH. However, there are some respects in which DWH is superior to WPLS and where the latter would be enhanced by incorporating some aspects of the former: Education: whereas the PES databases in DWH contain usable information on qualification levels at the time of programme participation, this is not the case with WPLS. Such information could be incorporated in WPLS by upgrading the information forms that are filled in by clients and their personal advisers at the beginning of programme participation. Qualifications data would be particularly useful for evaluating programmes such as New Deal for Young People, which has an important educational component, 8 along with other policies designed to implement Guideline 4 of the UK s Employment Action Plan 2003, on promoting the development of human capital and lifelong learning (Department for Work and Pensions, 2003). The HV outcome variable is more detailed than the one available through IR: it contains information on periods of unemployment, training, dependent employment, selfemployment, marginal employment, parental leave, national service, retirement and period outs of the labour force, whereas at present the IR data in WPLS only records periods of employment, although periods of unemployment and other benefit claiming can be matched in from the MI. It would be possible to address this issue to some extent with WPLS by incorporating information on periods of Statutory Maternity Pay or Maternity Allowance receipt (which would capture periods of maternity leave to some extent) but there are no plans to incorporate this information at present. 8 The Full-time Education and Training Option. 4

5 On the other hand, the WPLS has data that is in some respects superior to that in DWH, or soon will be: Pay data: as well as information on employment records, WPLS will shortly include information on the earnings of individuals during periods of employment. This will include earnings of all individuals earning above the tax threshold, including the self-employed. The HV income data, in contrast, are right censored because of the social security contribution cap. Historical data: whereas the PES data in DWH goes back to 1998, the JSA data in WPLS goes back to This is important when carrying out evaluation of ALMPs, since such historical data can be important when producing comparison groups. Career monitoring While career monitoring, extending beyond DWP clients to include the labour market as a whole, is possible with WPLS, there is much more scope for it with DWH. The main reason for this is that the HV data contains a much wider collection of socio-demographic and personal characteristics than does the IR data used in WPLS. Thus, while the IR data contains information on age, gender, postcode and some employer characteristics, the HV data additionally includes job tenure, nationality, socio-economic group and a very broad range of employer characteristics. This makes it possible to carry out sophisticated analyses of issues such as the labour market situation of older persons (Buzek, Edlinger, Friedenthal, Hochrainer, Schmitzberger, Tauer, and Zauner, 2004). The ability to carry out such analyses with WPLS would assist policy development in the UK on the issue of increasing labour supply and promoting active ageing (see Guideline 5 in UK Employment Action Plan, 2003). While there is considerable need for transferability between DWH and WPLS in relation to career monitoring, the scope for this is more limited. This is due to institutional and legal factors. The IR simply does not collect the range of socio-demographic and personal information that is available from HV, nor does it contain the same range of employer-related data. It might be possible to match in such data from registers of employers, such as the Interdepartmental Business Register (IDBR) maintained by the Office for National Statistics, but this would raise both technical and data protection issues that may prove difficult to solve. Enterprise monitoring The degree of enterprise monitoring that is possible with DWH is well beyond that which could be done with WPLS. The main reason for this again lies in the greater range of information that is available on employers from the HV as opposed to the IR data. Thus, DWH includes employer-related information on staff numbers, staff structure (in terms of gender, age and type of employment), notification of employment and termination of employment, fluctuations, and the hiring behaviour of enterprises differentiated by employers accounts. In addition, it includes industry and employer size and contains the data required to calculate growth and labour turnover figures (Mahringer, 2004). This makes it possible to carry out the kinds of detailed analyses of enterprise and employment dynamics as described in Buzek, Edlinger, Friedenthal, Hochrainer, Schmitzberger, Tauer, and Zauner (2004). The UK would benefit greatly from having administrative data that could be used for this purpose so that the need for transferability from DWH is considerable in this respect. Such data would enable 5

6 analyses of issues such as fostering entrepreneurship and promoting job creation (see Guideline 2 in UK Employment Action Plan, 2003). In addition, it would be of great assistance to DWP to know more about the types of employers that their clients find work with once leaving ALMPs. The problem remains, however, that IR does not collect much of the information used for enterprise monitoring with DWH and matching in data from an employer register such as IDBR would raise difficult technical and data protection issues. The scope for transferability, therefore, would appear quite limited. Conclusion While it is in relation to follow-up monitoring that there is least need for WPLS to incorporate information found in DWH, this is paradoxically the area in which there is most scope for such transferability. This is because the data sources used by PES in Austria and Jobcentre Plus in the UK are broadly comparable. In relation to career monitoring and enterprise monitoring, in contrast, there is a need for transferability given the greater depth and extensiveness of analyses that are possible with DWH, but less scope for it because IR does not collect much of the data provided by HV. 3 Important issues and future developments relevant to the policy Outlined below are three key issues affecting the performance, monitoring and evaluation of ALMPs that are likely to have important implications for the development and application of administrative data systems in the public employment service in the UK over the next 5-10 years: 9 Longer-term impacts of labour market programmes: until now, ALMP evaluations in the UK have rarely tracked programme entrants for longer than two years after programme entry. This is problematic given that in US research, where evaluation periods of five to ten years are commonplace, it has often been found that estimated programme impacts change appreciably more than two years after programme entry (Greenberg, Ashworth, Cebulla and Walker, 2003). An obvious problem with estimating longer-term impacts where evaluations are based on quantitative surveys is sample attrition, with the possibility of non-response bias increasing as it is difficult to achieve high response rates with clients long periods after initial contact. Administrative data is likely to have an important future role here, therefore, since it could augment survey data and make it possible to estimate longer-term impacts. Flows across programmes and benefit types: Programmes aimed at the economically inactive are likely to stimulate client flows across programmes and benefit types, as well as from out-ofwork to in-work statuses. So, for example, the IB Reform Pilots might encourage IB claimants to move onto JSA, which would accelerate their entry into New Deal 25+. The increased prevalence of this type of transition will present an important issue for future research. Labour market programme evaluations have traditionally concentrated on transitions from out-of-work to in-work statuses but will increasingly need to focus also on transitions from one type of benefit to another and on movements between different types of non-work economic status (such as from economic inactivity to unemployment). A comprehensive administrative database that makes it possible to track movements across benefits and programmes and across different types of economic status will have a very important role in evaluating the performance of DWP programmes. 9 This list is not exhaustive but space considerations preclude a more extensive discussion. 6

7 Subgroup analyses: Its Employment Action Plan 2003 commits the UK to promote the integration of and combat discrimination against people at a disadvantage in the labour market (Guideline 7). This makes it important to establish how the effects of labour market programmes vary across different groups of clients. Other important issues here include identifying what types of support are most effective for different groups; establishing whether any specific groups systematically under-perform on programmes; establishing evidence on the timing of interventions for particular groups; identifying what works best at different stages; and estimating the impact of early entry to a labour market programme. As administrative records are held on all individuals who have participated in ALMPs since 1998, they can be used to explore the experiences of precisely defined subgroups, such as particular ethnic groups, to a degree that has not been possible in UK evaluation research to date. All of these issues, and others, suggest that future developments will strongly encourage the development and expansion of WPLS and its successors in the UK. It is important, therefore, that DWP considers carefully what can be learned from PES administrative data systems developed in other EU countries, such as DWH in Austria. References Blundell, R., Costa Dias, M., Meghir, C. and Van Reenen, J. (2001) Evaluating the Employment Impact of a Mandatory Job Search Assistance Program, Institute for Fiscal Studies Working Paper, Wp01/20. Buzek, A., Edlinger, H., Friedenthal, C., Hochrainer, K., Schmitzberger, F., Tauer, B. and Zauner, M. (2004) Labour Market Monitoring based on the Data Warehouse of the Public Employment Service. Department for Work and Pensions (2003). United Kingdom Employment Action Plan Department for Work and Pensions (2004). Work and Pensions Longitudinal Study documentation: Metadata version 1.1. Dorsett, R. (2001). The New Deal for Young People: relative effectiveness of the options in reducing male unemployment. Policy Studies Institute Research Discussion Paper No. 7. Greenberg, D., K.Ashworth, A.Cebulla and R.Walker (2003). Do welfare-to-work programmes work for long? Fiscal Studies (forthcoming). Hall, J. and Reid, K. (1998) New Deal for the young unemployed: monitoring and evaluation, Labour Market Trends, November, Lissenburgh, S. (2000) New Deal for the Long-term Unemployed Pilots: Baseline Report. Employment Service Report ESR59. Lissenburgh, S. (2001a) New Deal for the Long Term Unemployed Pilots: quantitative evaluation using stage 2 survey. Employment Service Research Report ESR81. Lissenburgh, S. (2001b) New Deal for the Long-Term Unemployed: a comparison of provision in Pilot and national areas, Labour Market Trends, August,

8 Mahringer, H. (2004). Peer Review on Data Warehouse Monitoring in the Public Employment Service, Austria, 3-4 June 04. Riley, R. and Young, G. (2001a) The macroeconomic impact of the New Deal for Young People. National Institute for Economic and Social Research Discussion Paper No Riley, R. and Young, G. (2001b) Does welfare-to-work policy increase employment?: Evidence from the UK New Deal for Young People. National Institute for Economic and Social Research Discussion Paper No