Quantifying effects due to questionnaire changes using split-ballot designs

Transcription

1 Quantifying effects due to questionnaire changes using split-ballot designs Sabine Köhne-Finster Magdalena Gryga

2 Agenda Starting SITUATION Changing the questionnaire in 2011 The split-ballot DESIGN Effects of the new questions 5 First conclusions

3 Agenda Starting SITUATION Changing the questionnaire in 2011 The split-ballot DESIGN Effects of the new questions 5 First conclusions

4 NUMBER OF Employed Persons from different sources National Account Labour Force Survey Januar Februar März April Mai Juni Juli August September Oktober vember Dezember Januar Februar März April Mai Juni Juli August September Oktober vember Dezember Januar Februar März April Mai Juni Juli August September Oktober vember Dezember Januar Februar März April Mai Juni Juli August September Oktober vember Dezember } Incoherence between National Accounts and LFS in 1.000

5 NUMBER OF ONLY marginally employed persons in National Account LFS Incoherence between National Accounts and LFS Q3 Q4 Q1 Q2 Q Folie 5

6 Assumptions about the causes of measurement errors 1 Proxy interviews In proxy situation the respondent is not informed about small jobs and marginal employment of all household members Folie 6

7 Assumptions about the causes of measurement errors school 2 Main Status in Mind Pupils, students, retired persons or others keep their main status in mind and consider themselves as unemployed Folie 7

8 Assumptions about the causes of measurement errors 3 A lot of questions concerns the working situation. SHORTENING OF THE INTERVIEW t to declare an employment couldbean attempt to shorten the interview Folie 8

9 FOLLOW-UP Labour force SURVEY in Summer weeks after the regular LFS interview Short questionnaire about the working situation Starting with one question about the main status Only CATI proxy interviews allowed Folie 9

10 RESULTS OF THE FOLLOW-UP Labour force SURVEY % had changed the ILO status in the follow up survey. Most of these 8% haven t been employed in the main status. The differences between proxy and self interview haven t been significant. Empirical evidence on the main status thesis! Folie 10

11 Recommendations from project group in April 2009 Reformulate the questions concerning the ILO employment status. Implement one question about the main status in the German Labour Force Survey. Use information about main status for question formulations in regard to the different target groups Folie 11

12 Agenda Starting SITUATION Changing the questionnaire in 2011 The split-ballot experiment Effects of the new questions 5 First conclusions

13 Questions ABOUT ILO STATUS In the german LFS Have you done any paid work in the last week? Family worker? Employee Self- Employee Family worker Main Status? Pupils Students Others rmally employed? Have you any paid work (or are you family worker)? Until 2010 Have you worked last week?

14 CHANGING THE QUESTIONNAIRE Have you done any paid work in the last week? Family worker? Have you done any paid work in the last week? Family worker? Have you done any paid work in the last week? Family worker? Have you done any paid work in the last week? Family worker? Employee Self- Employee Family worker Main Status? Pupils Students Others rmally employed? rmally employed? rmally employed? rmally employed? Have you any paid work (or are you family worker)? Have you worked last week? What about time series?

15 Agenda Starting SITUATION Changing the questionnaire in 2011 The split-ballot DESIGN Effects of the new questions 5 First conclusions

16 SPLIT BALLOT DESIGN 90% of respondents get the new questions Employee Self- Employee Family worker Main Status? Pupils Students Others Have you any paid work (or are you family worker)? Have you done any paid work in the last week? Family worker? rmally employed? Have you worked last week? 10% of respondents get the old version

17 Agenda Starting SITUATION Changing the questionnaire in 2011 The split-ballot DESIGN Effects of the new questions 5 First conclusions

18 SPLIT BALLOT FIRST DATA 2011* January February March Total * without old paper questionnaire from 2010 (sample from reference week in 2011)

19 ILO-STATUS TOTAL POPULATION (15+) Employed 53,7% 52,5% NEW DESIGN OLD DESIGN Unemployed 3,6% 3,5% The sample with new questions have a higher employment rate (+ 1,2 percentage points). Inactive 42,7% 44,0% Estimated total increase employed persons in addition

20 ILO-STATUS SUBPOPULATION Pupils RETIRED* * Under 75 years Employed 15,5% 10,9% 4.6% points Employed 6,2% 5,4% 0.8% points Unemployed 1,9% 1,3% Unemployed 0,4% 0,2% Inactive 82,5% 87,8% Inactive 93,4% 94,3% REGISTERED UNEMPLOYED HOUSEKEEPING Employed 15,4% 11,0% 4.4% points Employed 17,9% 16,3% 1.6% points Unemployed 57,9% 60,4% Unemployed 3,7% 3,3% Inactive 26,7% 28,6% Inactive 78,4% 80,4% NEW DESIGN OLD DESIGN

21 Agenda Starting SITUATION Changing the questionnaire in 2011 The split-ballot DESIGN Effects of the new questions 5 First conclusions

22 FIRST CONCLUSIONS FORMULATION Principle 1 for the formulation of the questions on the labour status The questions on the labour status according to the ILO definition ( ) are in general the first questions in the individual questionnaire ( ). COMMISSION REGULATION (EC) 1897/2000 In particular, they are not preceded by questions on the main ( ) activity ( ) or on the administrative status of a registration ( ), where this would prejudice the response to the questions on the ILO labour status.

23 FIRST CONCLUSIONS FORMULATION The new question design of the German LFS is not in line with the principles for the formulation of the questions on the labour status. However, starting with a question about the main status seems to be a good practice to identify the ILO labour status. Underline the recommendation of the Task Force: R34: Review the 12 principles for the formulation of the questions on labour status ( ). Eurostat should set up a group of experts for this review.( )

24 FIRST CONCLUSIONS MONITORING To improve data quality it will always be necessary to develop question formulations. But: Methodological changes must be monitored. Underline the recommendation of the Task Force R35. Adequately plan and monitor, at national level, all changes ( ) in order to assess the statistical effect on time series. In our experience: Split ballot is a good method to quantify effects of changes.

25 THANK YOU FOR YOUR ATTENTION! Sabine Köhne-Finster Magdalena GRYGA Folie 25