EU-SILC: Impact Study on Comparability of National Implementations

Transcription

1 1 EU-SILC: Impact Study on Comparability of National Implementations No Introduction The cross-sectional EU-SILC survey of Finland is conducted together with the Finnish Income Distribution Survey, and most of the income information those surveys use is obtained from the registers. In addition the Total Statistics on Income Distribution, which covers the whole population, provides the exact parameter values for many essential income indicators. Some different definitions limit the comparability of the total statistics and the survey. The estimation of the Finnish EU-SILC is based on the effect of the sampling design and especially the calibration, which includes both many demographic variables as well as several income variables from the registers available at the time the weights are calculated. However, regarding especially the indicators of poverty (e.g. at risk poverty threshold) the studies of Eurostat 1 and Statistics Finland 2 show that the efficiency of the calibration is not at the best level. It clearly seems that most of the register totals used in calibration serve mainly the estimation of the indicators of income levels and dispersion. The level of using calibration in the EU-SILC varies from one country to another. Because of lacking sufficient register information, many countries have to use (in addition to some demographic variables) only few estimates of essential statistics in calibration. On the other hand, some countries have a lot of calibration constraints in estimation, and e.g. Denmark has included also some poverty measures into the calibration process (e.g. poverty index for the dwelling unit). The exceptional situation of Finland (having some variable values for the whole population via registers) enables a specific test situation, where different calibration strategies can be tested and evaluated considering both the bias and the standard error of some estimators. The test can be extended to some subpopulations as well. The Finnish cross-sectional SILC data 2005 of respondents includes households together with persons in them. Correspondingly, the register information for the population excluding the people in institutions consists of over 5 million persons. This technical implementation report is financed by European Commission (Grant Agreement for an Action - Agreement Number ). The construction of the testing system, main processing and writing are conducted by Pauli Ollila of Statistics Finland. Also Veli-Matti Törmälehto and Marie Reijo of Statistics Finland have contributed to this study, especially providing their valuable knowledge on the substance matter in different phases of the work. Phases of the Study The testing process includes the following phases: 1) Choosing different calibration strategies for testing (no calibration, calibrating with demographic variables and in addition 3-4 calibrations of different level [simple estimates of statistics, possibly more advanced approach, the Finnish version, the Danish version]. An essential part of this process is to plan the selection of such income variables which are applied in some EU-SILC countries. Adjusting the strategies to the Finnish data and sampling design situation. 2) Preparation of the Finnish SILC 2005 data for the tests. 3) Obtaining the necessary register information and preparing it to the tests. 4) Adjusting the existing weighting programs and the Eurostat indicator programs into the test situation. 5) Adjusting the current variance estimation programs to the test situation. 1 standard error calculations by Mr. Guillaume Osier 2 some bootstrap standard error calculations by Mr. Pauli Ollila

2 2 6) Creating the set of programs for carrying out the tests. 7) Calibrating according to the chosen strategies and calculating the estimates of indicators at the general and the subpopulation levels. 8) Evaluating the strategies by calculating the standard errors for the estimators. Carrying out some bias studies with the total statistics. 9) Comparing the sampling strategies in the framework of the Finnish EU-SILC. Analysing the results with conclusions. Weighting and Calibration in EU-SILC Countries Eurostat states in its weighting document (Verma 2007) that "the critical requirement in calibration is to ensure that the external control variables are strictly comparable to the corresponding survey variables, the distribution of which is being adjusted". Furthermore, " 'integrative' calibration is recommended. By this is meant calibration which retains uniform weights for all members of the same household ". "The household and personal level variables used for calibration can be as specified in Eurostat (2004): region (NUTS2), tenure status, household size (household type may also be useful); and gender and 5-year age group at the person level." The final cross-sectional weights are an outcome of the effect of the design (including stratification), nonresponse correction and additional adjustments (e.g. ratio adjustment / post-stratification / calibration). The impact of an auxiliary variable for weighting can appear in different parts of the weighting process depending on the member country. Nevertheless, the quality reports reveal that a common practice in almost all EU- SILC countries is to take gender and age groups, household size, and region into account in some phase of weighting, which is in line with the Eurostat statements. Also tenure status and type of municipality appear frequently in weighting criteria. In a few countries information about marital status, nationality, employment, pension, education and/or profession is used at some categorisation. The most elaborated weighting systems include income information in different forms and indicator describing poverty (Denmark). Description of Sampling in the Finnish EU-SILC Sampling Design The sampling design of the Finnish EU-SILC survey, the collection year 2005, (also parallel with the design of the Finnish Income Distribution Survey [IDS]) is a two-phase sampling design. The copy of the population register some weeks before the end of the study year included 4,185,517 non-institutional persons aged 16 years or over. The type of the frame was based on the domicile code, i.e. very exact identification of all the possible places where people can live. The first digits of this code include regional information (municipality code). Systematic sampling of persons was carried out from that frame in order to get the basis for a master sample (in 2005 exceptionally 100,000). After various checks and combinations we get 98,248 dwelling units with all their relevant members. The loss of 1,752 persons is due to the difference between the register which the selector of the master sample has and the final population register of the end of the study year. This final information (coming with the tax information to be connected to the master sample in order to create the strata, for example) is available after the master sample has been selected. At this point those who have died, moved permanently abroad or placed into an institution after the time point of the copy of the register and before the end of the year are excluded from the master sample. With this processing we correct the effect of the frame imperfection (not exactly describing the right time) in the sample. This master sample of dwelling units is used for different sampling purposes, and one of them is the Income Distribution Survey. For that the master sample is stratified by socio-economic criteria, emphasising highearners, farmers and entrepreneurs in the allocation. The sample size of the first wave is 7,500. The second wave of the IDS (5,869) is included in the set of households to be interviewed. The final definition of the

3 3 structure of the household is done during the interview. The stratum is identified for these IDS waves separately in the variable DB050. Type of Sampling and Sampling Units The Finnish sampling design includes only sampling of persons, thus there is only one stage of sampling. The stratification is constructed in the first-phase master sample, not in the population. Sampling is conducted in two phases: in the first phase persons are selected (first phase sampling unit), in the second phase the target persons together with their dwelling units are selected (second phase sampling unit). In a sense the second phase contains clustering (though constructed around the target person). However, the sampling unit can be still considered as a person (only he/she answers the personal questions). Stratification Criteria The SILC data selection follows parallelly the sampling design of the Income Distribution Survey. The IDS stratification is conducted in the first-phase master sample containing dwelling units. The strata are created by using a socio-economic categorisation based on the register information available for the members at the time of sample selection. The stratification takes the highest earning person as the categorising person, but the entrepreneur need not be the highest earning one to define the household in the class of entrepreneurs. The income class division is used to allocate the sample more to high-earners. The stratification variable is DB , containing values 1-13 for the first IDS wave and for the second IDS wave, based on the dwelling units created around the selected persons. Table 1. Stratification Criteria for the IDS IDS Wave 1 (CY 2005) IDS Wave 2 (CY 2004) Socio-economic Income Class Stratum code Socio-economic Income Class Stratum code categorisation of the household categorisation of the target person Wage earners Lowest 1 Wage earners Lowest 14 2nd lowest 2 2nd lowest 15 3rd lowest 3 3rd lowest 16 Highest 4 Highest 17 Entrepreneurs Lower 5 Entrepreneurs Lower 18 Higher 6 Higher 19 Farmers Lower 7 Farmers Lower 20 Higher 8 Higher 21 Pensioners Lower 9 Pensioners Lower 22 Higher 10 Higher 23 Others Lower 11 Others Lower 24 Higher 12 Higher 25 No tax information - 13 No tax information - 26 Sample Selection Schemes The master sample of persons (1st phase) is selected with systematic sampling from the population sorted by the domicile code. The SILC/IDS sample of the first wave with dwelling units constructed around the target persons is selected from the stratified master sample with simple random sampling without replacement within every stratum and using non-proportional allocation. The IDS second wave respondents from the previous year were selected at that time in the same way. The first wave of the EU-SILC longitudinal component selected in 2005 of size 2500 is selected randomly within strata from the first wave of the Income Distribution Survey (of size 7500) proportionally to the size of the IDS sample within strata.

4 4 Description of Weighting and Calibration in the Finnish EU-SILC Master Sample Calculated from the master sample CY 2004 (of size 50,000) we got the population figures for the person selection, e.g., where π a, person k is the inclusion probability of the selected person k in the master sample. The inclusion probabilities of the dwelling units created around the selected persons in the master sample were. The first phase weight includes the master sample information in full. π = π ak a, person kn 16+, dwelling of k Income Distribution Survey Sample The inclusion probabilities of two-phase sampling (the effect of selecting the master sample and the IDS sample) were calculated, at the second phase based on the stratification (13 strata) of the master sample and the allocation used. The over-coverage is included in calculations. For those waves we separately * calculated the inclusion probabilities π = n N, π ak = π π k = π akπ k s a, where ns n a 16+, n16 +, HH of k = N a, person k HH of k and k sa h h s a is the conditional inclusion probability at the second phase taking the stratification of the master sample into account. The sample for the new SILC wave is selected randomly within strata from the first wave of the Income Distribution Survey proportionally to the size of the IDS sample within strata. Thus the conditional inclusion probability π k s a is corrected with the term n SILC, h / nh. The base weights for the new wave were constructed as follows. / Unit Non-response Correction As the basis of calibration the unit non-response was corrected by n SILC,sample,h / n SILC,respondents,h in every stratum h (interpreted as the inverse of the response probability in every stratum). The sum of these corrected weights calculated separately in the data of accepted 16+ persons in the HHs coincides with N 16+. Calibration These weights containing a simple correction were used in calibration (the raking method) conducted with the macro CALMAR (applicable in SAS) for the accepted households (for the new SILC wave 1912). The calibration could be interpreted as integrative, i.e. both the household and the person levels were included in the process. The percentual marginal distributions and the statistics used in calibration are the following: 1) Households: province; type of municipality; HH size; sums of 15 different income variables. The first three distributions of the households were obtained from the master sample, using weights for which a primary calibration (population register: 16+ persons and persons under 16 by region; gender*age class) was conducted. The income information comes from different registers. 2) Persons: gender and age classes (0-4, 5-9,, 80-84, 85+)

5 5 Table 2. Description of the Calibration Variables Variable name Alue Ask8 Haastkur Description Region (NUTS 3 level), Capital region separated Size of dwelling unit Degree of urbanisation Mibs01-Mibs18 Men 0-4, 5-9, 10-14,, 80-84, 85- Nibs01-Nibs18 Women 0-4, 5-9, 10-14,, 80-84, 85- Trplopti Income 1: Cash or near cash employee income Saipalk Income 2: Income 1 > 0 Lelake Income 3: Pensions Tyotts Income 4: Unemployment benefits 1 Perustur Income 5: Unemployment benefits 2 Saityott Income 6: Income 4 > 0 Elintul3 Income 7: Income from self-employment Yhtytulo Income 8: Capital income 1 Maattulo Income 9: Income from agriculture Omaitul2 Income 10: Income from property and forestry 1 Muupaao2 Income 11: Other capital income Metstulo Income 12: Income from forestry 2 Myvo Income 13: Capital gains Saielake Income 14: Pensions > 0 Askorot Mortgage interests In addition, the figure 2,415,000 was used as the fixed number of households in the process. The result of this calibration was the weight that produced exactly these margins when used in the summation of these variables in the data set containing accepted observations. Calibration Models One essential reason to use calibration at least with demographic variables, is that it produces coherent estimates of these variables from the survey data, which is desirable on many occasions. The idea to correct the design weights in order to match the percent distributions contains also an assumption that the possible bias due to non-response is corrected to some extent at the same time. If the study variables are in some sense related to the demographic variables, the efficiency of the estimation (i.e. standard error) should be improved as well. The marginal statistics from registers utilised in calibration can have two forms: a category distribution with frequencies or percentages and total statistics (frequency or sum amount) for some variable in all or in a category. The households include persons, and the properties to be calibrated at that level need to be based on dichotomous variables of some properties, e.g. each age group has a 0/1 valued variable for each member of the household. These variables are then summed over the persons of the household for calibration. Thus the persons have the household weight also for non-household purposes. Finally the number of households is fixed for calibration. See Deville and Särndal (1992) for a theoretical description of the calibration techniques in survey sampling. As the basis the socio-economic stratification is taken into account in the design weights. These five calibration models are selected in order to reflect situations in the European countries. It is possible that some information about getting salary, getting unemployment benefit or getting pension (0/1 variables) or e.g. sum amount of salary or pension are available also at the population level. Then the technique of including these total statistics, at least one of them, to the calibration process, could be applied. However, this practice requires that the variable used in the register and the variable from the survey describe the same thing sufficiently enough. The calibration models to be studied are as follows:

6 6 C1 Demographic variables Region, size of household, degree of urbanisation, gender * age group (5 year categories). C2 Demographic variables and status of person Demographic variables, getting salary (0/1), getting unemployment benefit (0/1), getting pension (0/1). C3 Demographic variables and some income figures Demographic variables, amount of employee income, amount of pension. C4 Finnish version Demographic variables, employee income variables, pension variables, unemployment benefits, selfemployment benefits, capital income, agriculture income, property and forestry income, capital gains, mortgage interests. C5 Finnish version and some variables reflecting poverty Variables of Finnish calibration, disability pensions, unemployment pensions, old age pensions, general housing allowance, pensioners' housing allowance of spouses' pensions, students' housing supplement, pensioners' housing allowance of national old age, disability and unemployment pensions. In addition to these models an alternative with no calibration (NC) is studied, i.e. only the design weights with a simple non-response correction within strata. Parameters to Be Estimated The linear parameters to be estimated are the means of the variables in Table 3. These variables are strictly chosen following the requirements for the final quality report. Note that the variables HY135G, PY030G, PY070G, PY080G and PY200G do not have values in the Finnish data, and for that reason they are not studied here. Table 3. Linear Parameters to Be Estimated HY010 Total household gross income PY010G Cash or near-cash employee income HY020 Total disposable household income PY020G Non-cash employee income HY022 Total disposable household income, before social PY035G Contributions to individual private plans transfers other than old-age and survivors' benefits HY023 Total disposable household income, before social transfers including old-age and survivors' benefits PY050G Gross cash profits or losses from self -employment (incl. royalties) HY030G Imputed rent PY090G Unemployment benefits HY040G Income from rental or property or land PY100G Old-age benefits HY050G Family/children-related allowances PY110G Survivors' benefits HY060G Social exclusion payments not elsewhere classified PY120G Sickness benefits HY070G Housing allowances PY130G Disability benefits HY080G Regular inter-household cash transfers received PY140G Education-related allowances HY090G Interest, dividends, profit from capital investments in Equivalised disposable household income unincorporated businesses HY100G Interest paid on mortgages HY110G Income received by people aged under 16 HY120G Regular taxes on wealth HY130G Regular inter-household transfers paid HY140G Tax on income and social insurance contributions

7 7 The main non-linear indicators which are applicable in the cross-sectional Finnish EU-SILC data are estimated, i.e. 1) At-risk-of-poverty rate after social transfers 2) Inequality of income distribution S80/S20 income quintile share ratio 3) Relative median at-risk-of-poverty gap 4) Dispersion around the risk-of-poverty threshold (40 %) 5) Dispersion around the risk-of-poverty threshold (50 %) 6) Dispersion around the risk-of-poverty threshold (70 %) 7) At-risk-of-poverty rate before social transfers except old-age and survivors benefits 8) At-risk-of-poverty rate before transfers including old-age and survivors benefits 9) Inequality of income distribution: Gini coefficient Some subgroups are studied as well, mainly following the categorisation for indicator calculation required by Eurostat in the technical document on intermediate and final quality reports and available in the indicator programs. Testing in Practice The data sets used for testing are the cross-sectional Finnish EU-SILC files D, H, R and P for year The additional information for calibration not existing in the EU-SILC files comes from registers from Finnish Tax Administration, Social Insurance Institution of Finland and Population Register Centre of Finland. For practical reasons due to this testing the calibration is produced with a macro CLAN 3 instead of CALMAR 4 and this may cause minor differences when compared the results of this study and the SILC results for Finland. The registers are also used for calculating the marginal distributions and statistics required for calibration in general and variance estimation of the estimators of the linear parameters. The marginal distributions were constructed and converted into the structure CLAN recognises with SAS. The calibration took the sampling design into account as the basis with non-response correction within strata. The calibrated weights were checked by comparing the marginal distributions with corresponding distributions estimated at the survey level. The CLAN weights based on the Finnish calibration criteria were compared with the original weights by CALMAR, and they were in that case almost the same. Table 4. Properties of Weights Calibration model N Sum of weights Mean of weights St. dev. Coeff. of var. Min. Pct 1 Pct 5 Pct 10 Pct 25 Median Pct 75 Pct 90 Pct 95 Pct 99 Max- No calibration , Demographic variables Demographic and status Demographic and income Finnish calibration Finnish and poverty The difference between the non-response corrected design weights and calibrated weights is clear. The calibrated weights have several constraints which affect the weight structure, in this case making them more spread and providing also clearly lower and higher values than the adjusted design weight. In order to get all 3 a macro for standard error calculation of the estimators consisting of the functions of totals (including calibration), developed in Statistics Sweden 4 a macro for calibration, developed in INSEE

8 8 the calibration models through, a minimum of 0.2 was set, i.e. weights below one were allowed for testing. As the calibration theory indicates, the given number of households equals the sum of calibrated weights. The indicators are calculated with the SAS macro code provided by Eurostat for the national EU-SILC calculations. The bootstrap variance estimation method is utilised for the non-linear indicators. The principle is such that in every stratum a resample without replacement is selected from the set of respondents (Bickel and Freedman 1985, see also Sitter 1992). The size of the resample is n r,h = (16/17)*n h (rounded to an integer), where 16 is one less than the number of respondents in the smallest stratum. This operation provides an easy solution to the variance correction, which is required because the sampling fraction in the resampling design (i.e. a constant 16/17 for every stratum) is much higher than in the original design (see Ollila 2004 for more theoretical details). The overall resample size is Every resample is then calibrated under the calibration model selected for testing. With these weights, valid and right-scale estimates can be obtained for every resample as well. For the bootstrap application the code of the Eurostat indicator macros is slightly modified in order to adjust it to the changing data sets and weights from one replication to another. After calculating the estimates for the resample they are saved into a cumulative data set. At the end of the process this set of estimates is used for variance estimation. The estimator for the variance of the estimator θˆ at the second phase level is of the form ( / 75000) Vˆ( ˆ) = (1 16/17) ( ˆ θ ˆ) θ A g g= 1 A θ, where the term before summation adjusts the difference between the original second phase sampling design and resampling design in the variance formula. The standard error of the estimator is the square root of the variance. Note that one cannot correct this way within strata, because most of the indicators require percentiles, which must be calculated at the whole sample level, not in strata. The simulation tests provided in the Finnish EU-SILC intermediate quality report from the first year 2004 show that the effect of the first phase to the variance is marginal. The number of resamples A is 1000, because the process with several calibration models and a lot of indicators and their subgroups to be calculated every time a new resample is selected is laborious. On the other hand, the resampling fraction 16 / % using resampling without replacement assures a major cover of the units in every stratum, e.g. in small strata the number of different combinations of resamples from the set of respondents is not big. The variance estimation of the estimators of the linear parameters is conducted with CLAN. For the linear results obtained with non-response corrected design weighting also proc surveymeans of SAS and proc descript of Sudaan 5 in the SAS platform were used for checking. The Total Statistics on Income Distribution describe the annual income of register households and their distribution especially from the regional perspective. The statistics depict the amount of income and its formation from different income sources when taking taxation and income transfers into consideration. The main difference between the results of the Income Distribution Survey / EU-SILC and the Total Statistics on Income Distribution is that for the latter only the dwelling unit identification is available, not the structure of the household as it is defined in the survey. Törmälehto s study (2004) revealed that in Finland about 85 % of the dwelling units coincide with the households. Some indicators from the Finnish Total Statistics on Income Distribution of year 2004 (= SILC year 2005) are used here in order to take a look at the underlying bias. Results The six different weighting strategies are compared in different contexts: A software for the analysis of complex surveys by RTI.

9 9 - the estimators and their standard errors of the linear parameters at the household level (variables 1-16), at the 16+ person level (variables 17-26), and at the all-persons level (equivalised disposable income) with subgroups; - the consistency of CLAN and bootstrap standard error estimates - the estimators and their standard errors of the non-linear indicators with some subgroups - the bias study Table 5. Number of Observations, Means and Standard Errors rors for Components of Income with Different Weightings Components of income Obs. No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty mean s.e mean s.e mean s.e mean s.e mean s.e mean s.e 1 Total household gross income Total disposable household income 3 Total disposable household income, before social transfers other than old-age and survivors' benefits 4 Total disposable household income, before social transfers including old-age and survivors' benefits Imputed rent Income from rental or property or land 7 Family/children-related allowances 8 Social exclusion payments not elsewhere classified 9 Housing allowances Regular inter-household cash transfers received 11 Interest, dividends, profit from capital investments in unincorporated businesses 12 Interest paid on mortgages Income received by people aged under Regular taxes on wealth Regular inter-household transfers paid Tax on income and social insurance contributions 17 Cash or near-cash employee income Non-cash employee income Contributions to individual private plans 20 Gross cash profits or losses from self -employment (incl. royalties) 21 Unemployment benefits Old-age benefits Survivors' benefits Sickness benefits Disability benefits Education-related allowances * Households which have negative values or 0 values in the variable are counted as the households which have not received the income. Negative values of the certain gross income components in which they exist are counted in the variable HY010 on the total household gross income. The most notable results can be seen in the income variables 1 4, 11, 12, 14 and 15. Almost always the non-response corrected design weight outperforms the demographic calibration and additional status and

10 10 income information, when the standard error is considered. Although fulfilling the required demographic conditions, it seems that the basic calibration criteria of age & gender classification for the persons of the household together with the size of the household, region and type of municipality, twist the weights to an unwanted direction when considering the standard error. The effect of calibration in the five-year classes of the population targets also the children of the household, who do not earn or obtain money in practice. The positive impact of that aspect of calibration can be seen in variable 7 (family/children related allowances) where the design weight takes only the number of persons 16 years or over into account, but the calibration includes also the category information about the children. However, the intensive use of the income totals in different forms in calibration (the Finnish alternative) pays off, for these income variables mentioned earlier we have significant improvement in standard errors. In variable 12 (interest paid on mortgages) the improvement is evident: the resembling variable is included also in the calibration criteria. The results are disappointing especially for the demographic + status variables alternative. Also the demographic + two income variables alternative shows almost non-existing improvement except in variables 17 and 22, which happen to be near these two income variables from the register. The poverty variables addition to the Finnish alternative does not bring any significant help to the efficiency of estimation except variable 9, which can be found in these poverty variables as well. It is obvious that calibration with the demographic variables brings consistency to estimation especially when these groups are studied in the analysis of the data. Considering the estimates produced by six weighting strategies, it is not clear that for every variable the demographic calibration provides more accurate point estimates than the non-response corrected design weight. In some cases the point estimates of no calibration and intensive calibration are close to each other and farther from demographic calibration. Table 6. Number of Observations, Mean and Standard Errors for Equivalised Disposable Income in Household, Age and Gender Groups Equivalised disposable income Obs. No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty mean s.e mean s.e mean s.e mean s.e mean s.e mean s.e All household member household members household members household members or more Age group <25 years Age group years Age group years Age group years Age group years Age group 65- years Male Female Table 6 shows equivalised disposable income in all and in categories by household size, age group and gender. Now we are dealing with the full data including children as well. The overall results follow the same phenomenon as in the previous components of income. The key results can be found in the classification by household size. The number of observations reflects the sampling design of the Finnish EU-SILC: the persons are first selected and then the dwelling units and later households are constructed around the selected persons. The inclusion probability increases when the number of 16+ persons in the dwelling unit increases. The most effective results for calibration come from class 4 household members or more, which is the class where gender & age calibration might have the largest impact via children. Another end is class 1 household member, where the best standard error is for the no calibration alternative. There is some difference between the point estimates of simple calibration alternatives and no calibration/intensive calibration. These results were also verified by using the bootstrap variance estimation method (Table 7).

11 11 Table 7. Comparing CLAN and Bootstrap Standard Errors for Equivalised Disposable Income No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty CLAN Boot CLAN Boot CLAN Boot CLAN Boot CLAN Boot CLAN Boot Estimates Standard errors The structure of the parameter (equivalised disposable income) is not complex, and the resamples were selected with simple random sampling without replacement. Thus the standard errors should be rather close to the CLAN results, if both procedures were carried out correctly. The bootstrap standard errors seem to be slightly larger than the CLAN standard errors. This may come from the fact that the scale correction of the bootstrap variance estimate must be conducted at the overall level, not at the stratum level. This generalisation of scale correction ignores the non-proportional allocation which affects the estimation of the equivalised disposable income, when the stratification is taken into account in CLAN. The variance of an estimator of a complex parameter might not be an easy task to be estimated. The theoretical complexity is avoided here with a resampling technique. The results for the main non-linear indicators are presented in Table 8. The estimates come from the calculations with the full set of respondents, and in practice the bootstrap means of the resample estimates differ mostly one decimal maximum, usually less than that. Table 8. Main Non-linear Indicators and Their Standard Errors Indicator At-risk-of-poverty rate after social transfers Inequality of income distribution S80/S20 income quintile share ratio Relative median at-risk-ofpoverty gap Dispersion around the riskof-poverty threshold (40) Dispersion around the riskof-poverty threshold (50) Dispersion around the riskof-poverty threshold (70) At-risk-of-poverty rate before social transfers except old-age and survivors benefits At-risk-of-poverty rate before transfers including old-age and survivors benefits Inequality of income distribution: Gini coefficient Obs. No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty estim. s.e estim. s.e estim. s.e estim. s.e estim. s.e estim. s.e The point estimates don t differ much from one weighting to another. Most notable differences can be found in relative median at-risk-of-poverty gap, at-risk-of-poverty rate before social transfers except old-age and survivors benefits, and at-risk-of-poverty rate before transfers including old-age and survivors benefits. The standard errors of the intensive calibration alternatives are clearly lower with Gini coefficient and S80/S20, and correspondingly the basic calibration alternatives perform less well than the non-response

12 12 corrected design weight. On the other hand, the "at-risk" and "dispersion" indicators don't show any significant difference. The "relative median" indicator doesn't follow the usual standard error patterns. Table 9. Non-linear Indicators and Their Standard Errors for Some Subgroups Indicator No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty estim. s.e estim. s.e estim. s.e estim. s.e estim. s.e estim. s.e At-risk-of-poverty rate after social transfers Male Female At work not at work: total not at work: unemployed not at work: retired not at work: other inactive Total no dependent children person (total) adults, both < 65 years adults, at least one Other no dependent children Total dependent children Single parent, at least dependent child 2 adults, 1 dependent child adults, 2 dependent children adults, 3 dependent children Other households with dependent children Relative median at-risk-of-poverty gap Male Female Dispersion around the risk-of-poverty threshold (40) Male Female Dispersion around the risk-of-poverty threshold (50) Male Female Dispersion around the risk-of-poverty threshold (70) Male Female The subgroup results must be analysed cautiously. It is obvious that the estimates and the standard error estimates in rare groups are subject to uncertainty, e.g. class "other households with dependent children". In Table 8 the "at-risk-of-poverty-rate after social transfers" indicator included not much difference between weighting systems, and the same phenomenon can be noticed in most of the groups (excluding small rare groups). The "dispersion" standard error figures do not show notable difference either. As in Table 8, the "relative median at-risk-of-poverty gap" seems to provide results which are difficult to be interpreted.

13 13 Table 10. Comparing Indicators Based on Total Statistics on Income Distribution with Various Estimates Indicator At-risk-of-poverty rate after social transfers Inequality of income distribution S80/S20 income quintile share ratio Inequality of income distribution: Gini coefficient Register statistics No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty mean bias mean bias mean bias mean bias mean bias mean bias The planned idea to study the bias of the estimates of different weighting strategies with the Total Statistics on Income Distribution proved to be not very fruitful. Firstly, only three indicators were available from the data of the Total Statistics, because the amount of variables available from the registers is much more limited than in the Income Distribution Survey. Furthermore, the definitions behind the calculation of the indicators of the Total Statistics are different, and the calculations are based on the dwelling unit, not on the household. Even if the register figures were defined in the same way as in the survey, the differences between the weighting strategies were very modest, and there wouldn't have been strong conclusions in that situation either. Conclusions and Recommendations The results show some interesting phenomena in the context of the Finnish EU-SILC data. Although the demographic calibration (gender & age groups at the person level, household size and region at the household level) provides consistent estimates with those demographic external sources, with some estimators it seems to weaken the efficiency of estimation when compared with the non-response corrected design weights. The effect of calibration in the five-year classes of the population covers also the children of the household, who do not earn or obtain money in practice. On the other hand, the estimators concerning the age & gender categorisation (e.g. family/children related allowances) benefit also from the basic demographic calibration. Two simple demographic calibration models with adding 1) status information on persons (working, on pension, unemployed) or 2) two income variables (amount of salary, amount of pension) were tested. The idea was such that some member countries could use the totals for those variables from another sources, provided that the register and survey variables describe the same thing sufficiently. The results were disappointing. The efficiency improvement was marginal except with variables strictly dealing with the calibration variables (Cash or near-cash employee income and old-age benefit). The Finnish calibration practice with a lot of income information added to the demographic calibration seems to improve the efficiency of estimation clearly. Unfortunately, this situation with registers is a rare exception among the EU-SILC countries. The addition of some variables reflecting to poverty did not improve the Finnish calibration practice much, except for housing allowances. Concerning the complex indicators, the phenomenon described above for different weighting strategies occurred with "gini coefficient" and "S80/S20 income quintile share ratio". The other indicators were hardly affected. Note that in Finland full income information for every member of the household is obtained from registers, which is not the case for the majority of the other EU-SILC countries. Furthermore, the Finnish sampling design already includes socio-economic and high/low income categorisation, and this may cause the design

14 14 to be rather effective even before the calibration. It might happen that in another context the results can be somewhat different. One must be cautious to draw strong conclusions and recommendations based on this testing, especially when applying them in the non-register based household selecting countries, where the sampling designs differ a lot from the Finnish version. Still, the effect of demographic calibration compared with the nonresponse corrected design weights should be tested in that context as well. The results of this study do not provide evidence for recommending small scale variable additions for calibration. However, the good properties of fixing the main demographic variables with calibration are important as such, so there is no need at this point to reject the demographic calibration. References Bickel, P. J and Freedman, D. A. (1985). Asymptotic Normality and the Bootstrap in Stratified Sampling. The Annals of Statistics, 12, Deville, J.-C. and Särndal, C-E. (1992). Calibration Estimators in Survey Sampling. Journal of the American Statistical Association, Vol. 87. No. 418, Theory and Methods, Eurostat (2004). Technical Document on Intermediate and Final Quality Reports. Document EU-SILC 132/04. Ollila, P. K. (2004). A Theoretical Overview for Variance Estimation in Sampling Theory with Some New Techniques for Complex Estimators. Doctoral Thesis, University of Helsinki, Department of Statistics, The Research Reports of Statistics Finland, Helsinki. Sitter, R. R. (1992). Comparing Three Bootstrap Methods for Survey Data. The Canadian Journal of Statistics, 20, n:o 2, Törmälehto, V-M. (2004). Kotitalous haastattelusta vai rekisteristä? Unpublished research paper of Statistics Finland (in Finnish). Verma, V. (2006). EU-SILC Weighting Procedures: an Outline. Eurostat Output II.1(b).