1 EU-SILC: Impact Study on Comparability of National Implementations No 36401.2007.001-2007.192 Introduction The cross-sectional EU-SILC survey of Finland is conducted together with the Finnish Income Distribution Survey, and most of the income information those surveys use is obtained from the registers. In addition the Total Statistics on Income Distribution, which covers the whole population, provides the exact parameter values for many essential income indicators. Some different definitions limit the comparability of the total statistics and the survey. The estimation of the Finnish EU-SILC is based on the effect of the sampling design and especially the calibration, which includes both many demographic variables as well as several income variables from the registers available at the time the weights are calculated. However, regarding especially the indicators of poverty (e.g. at risk poverty threshold) the studies of Eurostat 1 and Statistics Finland 2 show that the efficiency of the calibration is not at the best level. It clearly seems that most of the register totals used in calibration serve mainly the estimation of the indicators of income levels and dispersion. The level of using calibration in the EU-SILC varies from one country to another. Because of lacking sufficient register information, many countries have to use (in addition to some demographic variables) only few estimates of essential statistics in calibration. On the other hand, some countries have a lot of calibration constraints in estimation, and e.g. Denmark has included also some poverty measures into the calibration process (e.g. poverty index for the dwelling unit). The exceptional situation of Finland (having some variable values for the whole population via registers) enables a specific test situation, where different calibration strategies can be tested and evaluated considering both the bias and the standard error of some estimators. The test can be extended to some subpopulations as well. The Finnish cross-sectional SILC data 2005 of respondents includes 11 229 households together with 29 112 persons in them. Correspondingly, the register information for the population excluding the people in institutions consists of over 5 million persons. This technical implementation report is financed by European Commission (Grant Agreement for an Action - Agreement Number - 36401.2007.002-2007.192 2007.192). The construction of the testing system, main processing and writing are conducted by Pauli Ollila of Statistics Finland. Also Veli-Matti Törmälehto and Marie Reijo of Statistics Finland have contributed to this study, especially providing their valuable knowledge on the substance matter in different phases of the work. Phases of the Study The testing process includes the following phases: 1) Choosing different calibration strategies for testing (no calibration, calibrating with demographic variables and in addition 3-4 calibrations of different level [simple estimates of statistics, possibly more advanced approach, the Finnish version, the Danish version]. An essential part of this process is to plan the selection of such income variables which are applied in some EU-SILC countries. Adjusting the strategies to the Finnish data and sampling design situation. 2) Preparation of the Finnish SILC 2005 data for the tests. 3) Obtaining the necessary register information and preparing it to the tests. 4) Adjusting the existing weighting programs and the Eurostat indicator programs into the test situation. 5) Adjusting the current variance estimation programs to the test situation. 1 standard error calculations by Mr. Guillaume Osier 2 some bootstrap standard error calculations by Mr. Pauli Ollila
2 6) Creating the set of programs for carrying out the tests. 7) Calibrating according to the chosen strategies and calculating the estimates of indicators at the general and the subpopulation levels. 8) Evaluating the strategies by calculating the standard errors for the estimators. Carrying out some bias studies with the total statistics. 9) Comparing the sampling strategies in the framework of the Finnish EU-SILC. Analysing the results with conclusions. Weighting and Calibration in EU-SILC Countries Eurostat states in its weighting document (Verma 2007) that "the critical requirement in calibration is to ensure that the external control variables are strictly comparable to the corresponding survey variables, the distribution of which is being adjusted". Furthermore, " 'integrative' calibration is recommended. By this is meant calibration which retains uniform weights for all members of the same household ". "The household and personal level variables used for calibration can be as specified in Eurostat (2004): region (NUTS2), tenure status, household size (household type may also be useful); and gender and 5-year age group at the person level." The final cross-sectional weights are an outcome of the effect of the design (including stratification), nonresponse correction and additional adjustments (e.g. ratio adjustment / post-stratification / calibration). The impact of an auxiliary variable for weighting can appear in different parts of the weighting process depending on the member country. Nevertheless, the quality reports reveal that a common practice in almost all EU- SILC countries is to take gender and age groups, household size, and region into account in some phase of weighting, which is in line with the Eurostat statements. Also tenure status and type of municipality appear frequently in weighting criteria. In a few countries information about marital status, nationality, employment, pension, education and/or profession is used at some categorisation. The most elaborated weighting systems include income information in different forms and indicator describing poverty (Denmark). Description of Sampling in the Finnish EU-SILC Sampling Design The sampling design of the Finnish EU-SILC survey, the collection year 2005, (also parallel with the design of the Finnish Income Distribution Survey [IDS]) is a two-phase sampling design. The copy of the population register some weeks before the end of the study year included 4,185,517 non-institutional persons aged 16 years or over. The type of the frame was based on the domicile code, i.e. very exact identification of all the possible places where people can live. The first digits of this code include regional information (municipality code). Systematic sampling of persons was carried out from that frame in order to get the basis for a master sample (in 2005 exceptionally 100,000). After various checks and combinations we get 98,248 dwelling units with all their relevant members. The loss of 1,752 persons is due to the difference between the register which the selector of the master sample has and the final population register of the end of the study year. This final information (coming with the tax information to be connected to the master sample in order to create the strata, for example) is available after the master sample has been selected. At this point those who have died, moved permanently abroad or placed into an institution after the time point of the copy of the register and before the end of the year are excluded from the master sample. With this processing we correct the effect of the frame imperfection (not exactly describing the right time) in the sample. This master sample of dwelling units is used for different sampling purposes, and one of them is the Income Distribution Survey. For that the master sample is stratified by socio-economic criteria, emphasising highearners, farmers and entrepreneurs in the allocation. The sample size of the first wave is 7,500. The second wave of the IDS (5,869) is included in the set of households to be interviewed. The final definition of the
3 structure of the household is done during the interview. The stratum is identified for these IDS waves separately in the variable DB050. Type of Sampling and Sampling Units The Finnish sampling design includes only sampling of persons, thus there is only one stage of sampling. The stratification is constructed in the first-phase master sample, not in the population. Sampling is conducted in two phases: in the first phase persons are selected (first phase sampling unit), in the second phase the target persons together with their dwelling units are selected (second phase sampling unit). In a sense the second phase contains clustering (though constructed around the target person). However, the sampling unit can be still considered as a person (only he/she answers the personal questions). Stratification Criteria The SILC data selection follows parallelly the sampling design of the Income Distribution Survey. The IDS stratification is conducted in the first-phase master sample containing dwelling units. The strata are created by using a socio-economic categorisation based on the register information available for the members at the time of sample selection. The stratification takes the highest earning person as the categorising person, but the entrepreneur need not be the highest earning one to define the household in the class of entrepreneurs. The income class division is used to allocate the sample more to high-earners. The stratification variable is DB050 050, containing values 1-13 for the first IDS wave and 14-26 for the second IDS wave, based on the dwelling units created around the selected persons. Table 1. Stratification Criteria for the IDS IDS Wave 1 (CY 2005) IDS Wave 2 (CY 2004) Socio-economic Income Class Stratum code Socio-economic Income Class Stratum code categorisation of the household categorisation of the target person Wage earners Lowest 1 Wage earners Lowest 14 2nd lowest 2 2nd lowest 15 3rd lowest 3 3rd lowest 16 Highest 4 Highest 17 Entrepreneurs Lower 5 Entrepreneurs Lower 18 Higher 6 Higher 19 Farmers Lower 7 Farmers Lower 20 Higher 8 Higher 21 Pensioners Lower 9 Pensioners Lower 22 Higher 10 Higher 23 Others Lower 11 Others Lower 24 Higher 12 Higher 25 No tax information - 13 No tax information - 26 Sample Selection Schemes The master sample of persons (1st phase) is selected with systematic sampling from the population sorted by the domicile code. The SILC/IDS sample of the first wave with dwelling units constructed around the target persons is selected from the stratified master sample with simple random sampling without replacement within every stratum and using non-proportional allocation. The IDS second wave respondents from the previous year were selected at that time in the same way. The first wave of the EU-SILC longitudinal component selected in 2005 of size 2500 is selected randomly within strata from the first wave of the Income Distribution Survey (of size 7500) proportionally to the size of the IDS sample within strata.
4 Description of Weighting and Calibration in the Finnish EU-SILC Master Sample Calculated from the master sample CY 2004 (of size 50,000) we got the population figures for the person selection, e.g., where π a, person k is the inclusion probability of the selected person k in the master sample. The inclusion probabilities of the dwelling units created around the selected persons in the master sample were. The first phase weight includes the master sample information in full. π = π ak a, person kn 16+, dwelling of k Income Distribution Survey Sample The inclusion probabilities of two-phase sampling (the effect of selecting the master sample and the IDS sample) were calculated, at the second phase based on the stratification (13 strata) of the master sample and the allocation used. The over-coverage is included in calculations. For those waves we separately * calculated the inclusion probabilities π = n N, π ak = π π k = π akπ k s a, where ns n a 16+, n16 +, HH of k = N a, person k HH of k and k sa h h s a is the conditional inclusion probability at the second phase taking the stratification of the master sample into account. The sample for the new SILC wave is selected randomly within strata from the first wave of the Income Distribution Survey proportionally to the size of the IDS sample within strata. Thus the conditional inclusion probability π k s a is corrected with the term n SILC, h / nh. The base weights for the new wave were constructed as follows. / Unit Non-response Correction As the basis of calibration the unit non-response was corrected by n SILC,sample,h / n SILC,respondents,h in every stratum h (interpreted as the inverse of the response probability in every stratum). The sum of these corrected weights calculated separately in the data of accepted 16+ persons in the HHs coincides with N 16+. Calibration These weights containing a simple correction were used in calibration (the raking method) conducted with the macro CALMAR (applicable in SAS) for the accepted households (for the new SILC wave 1912). The calibration could be interpreted as integrative, i.e. both the household and the person levels were included in the process. The percentual marginal distributions and the statistics used in calibration are the following: 1) Households: province; type of municipality; HH size; sums of 15 different income variables. The first three distributions of the households were obtained from the master sample, using weights for which a primary calibration (population register: 16+ persons and persons under 16 by region; gender*age class) was conducted. The income information comes from different registers. 2) Persons: gender and age classes (0-4, 5-9,, 80-84, 85+)
5 Table 2. Description of the Calibration Variables Variable name Alue Ask8 Haastkur Description Region (NUTS 3 level), Capital region separated Size of dwelling unit Degree of urbanisation Mibs01-Mibs18 Men 0-4, 5-9, 10-14,, 80-84, 85- Nibs01-Nibs18 Women 0-4, 5-9, 10-14,, 80-84, 85- Trplopti Income 1: Cash or near cash employee income Saipalk Income 2: Income 1 > 0 Lelake Income 3: Pensions Tyotts Income 4: Unemployment benefits 1 Perustur Income 5: Unemployment benefits 2 Saityott Income 6: Income 4 > 0 Elintul3 Income 7: Income from self-employment Yhtytulo Income 8: Capital income 1 Maattulo Income 9: Income from agriculture Omaitul2 Income 10: Income from property and forestry 1 Muupaao2 Income 11: Other capital income Metstulo Income 12: Income from forestry 2 Myvo Income 13: Capital gains Saielake Income 14: Pensions > 0 Askorot Mortgage interests In addition, the figure 2,415,000 was used as the fixed number of households in the process. The result of this calibration was the weight that produced exactly these margins when used in the summation of these variables in the data set containing accepted observations. Calibration Models One essential reason to use calibration at least with demographic variables, is that it produces coherent estimates of these variables from the survey data, which is desirable on many occasions. The idea to correct the design weights in order to match the percent distributions contains also an assumption that the possible bias due to non-response is corrected to some extent at the same time. If the study variables are in some sense related to the demographic variables, the efficiency of the estimation (i.e. standard error) should be improved as well. The marginal statistics from registers utilised in calibration can have two forms: a category distribution with frequencies or percentages and total statistics (frequency or sum amount) for some variable in all or in a category. The households include persons, and the properties to be calibrated at that level need to be based on dichotomous variables of some properties, e.g. each age group has a 0/1 valued variable for each member of the household. These variables are then summed over the persons of the household for calibration. Thus the persons have the household weight also for non-household purposes. Finally the number of households is fixed for calibration. See Deville and Särndal (1992) for a theoretical description of the calibration techniques in survey sampling. As the basis the socio-economic stratification is taken into account in the design weights. These five calibration models are selected in order to reflect situations in the European countries. It is possible that some information about getting salary, getting unemployment benefit or getting pension (0/1 variables) or e.g. sum amount of salary or pension are available also at the population level. Then the technique of including these total statistics, at least one of them, to the calibration process, could be applied. However, this practice requires that the variable used in the register and the variable from the survey describe the same thing sufficiently enough. The calibration models to be studied are as follows:
6 C1 Demographic variables Region, size of household, degree of urbanisation, gender * age group (5 year categories). C2 Demographic variables and status of person Demographic variables, getting salary (0/1), getting unemployment benefit (0/1), getting pension (0/1). C3 Demographic variables and some income figures Demographic variables, amount of employee income, amount of pension. C4 Finnish version Demographic variables, employee income variables, pension variables, unemployment benefits, selfemployment benefits, capital income, agriculture income, property and forestry income, capital gains, mortgage interests. C5 Finnish version and some variables reflecting poverty Variables of Finnish calibration, disability pensions, unemployment pensions, old age pensions, general housing allowance, pensioners' housing allowance of spouses' pensions, students' housing supplement, pensioners' housing allowance of national old age, disability and unemployment pensions. In addition to these models an alternative with no calibration (NC) is studied, i.e. only the design weights with a simple non-response correction within strata. Parameters to Be Estimated The linear parameters to be estimated are the means of the variables in Table 3. These variables are strictly chosen following the requirements for the final quality report. Note that the variables HY135G, PY030G, PY070G, PY080G and PY200G do not have values in the Finnish data, and for that reason they are not studied here. Table 3. Linear Parameters to Be Estimated HY010 Total household gross income PY010G Cash or near-cash employee income HY020 Total disposable household income PY020G Non-cash employee income HY022 Total disposable household income, before social PY035G Contributions to individual private plans transfers other than old-age and survivors' benefits HY023 Total disposable household income, before social transfers including old-age and survivors' benefits PY050G Gross cash profits or losses from self -employment (incl. royalties) HY030G Imputed rent PY090G Unemployment benefits HY040G Income from rental or property or land PY100G Old-age benefits HY050G Family/children-related allowances PY110G Survivors' benefits HY060G Social exclusion payments not elsewhere classified PY120G Sickness benefits HY070G Housing allowances PY130G Disability benefits HY080G Regular inter-household cash transfers received PY140G Education-related allowances HY090G Interest, dividends, profit from capital investments in Equivalised disposable household income unincorporated businesses HY100G Interest paid on mortgages HY110G Income received by people aged under 16 HY120G Regular taxes on wealth HY130G Regular inter-household transfers paid HY140G Tax on income and social insurance contributions
7 The main non-linear indicators which are applicable in the cross-sectional Finnish EU-SILC data are estimated, i.e. 1) At-risk-of-poverty rate after social transfers 2) Inequality of income distribution S80/S20 income quintile share ratio 3) Relative median at-risk-of-poverty gap 4) Dispersion around the risk-of-poverty threshold (40 %) 5) Dispersion around the risk-of-poverty threshold (50 %) 6) Dispersion around the risk-of-poverty threshold (70 %) 7) At-risk-of-poverty rate before social transfers except old-age and survivors benefits 8) At-risk-of-poverty rate before transfers including old-age and survivors benefits 9) Inequality of income distribution: Gini coefficient Some subgroups are studied as well, mainly following the categorisation for indicator calculation required by Eurostat in the technical document on intermediate and final quality reports and available in the indicator programs. Testing in Practice The data sets used for testing are the cross-sectional Finnish EU-SILC files D, H, R and P for year 2005. The additional information for calibration not existing in the EU-SILC files comes from registers from Finnish Tax Administration, Social Insurance Institution of Finland and Population Register Centre of Finland. For practical reasons due to this testing the calibration is produced with a macro CLAN 3 instead of CALMAR 4 and this may cause minor differences when compared the results of this study and the SILC results for Finland. The registers are also used for calculating the marginal distributions and statistics required for calibration in general and variance estimation of the estimators of the linear parameters. The marginal distributions were constructed and converted into the structure CLAN recognises with SAS. The calibration took the sampling design into account as the basis with non-response correction within strata. The calibrated weights were checked by comparing the marginal distributions with corresponding distributions estimated at the survey level. The CLAN weights based on the Finnish calibration criteria were compared with the original weights by CALMAR, and they were in that case almost the same. Table 4. Properties of Weights Calibration model N Sum of weights Mean of weights St. dev. Coeff. of var. Min. Pct 1 Pct 5 Pct 10 Pct 25 Median Pct 75 Pct 90 Pct 95 Pct 99 Max- No calibration 11229 2338777 208.28 163.65 78.57 13.55 22.91 31.63 39,03 73.03 179.43 277.24 490.85 556.06 739.68 739.68 Demographic 11229 2415000 215.07 183.97 85.54 0.27 9.51 30.22 41.73 69.55 173.08 286.96 486.88 612.44 820.78 1394.5 variables Demographic 11229 2415000 215.07 183.12 85.14 0.27 9.93 30.17 40.87 67.58 174.56 285.11 492.02 612.29 817.40 1248.5 and status Demographic 11229 2415000 215.07 185.77 86.38 0.27 9.26 30.15 41.51 69.05 171.02 286.99 485.50 610.35 840.90 1383.4 and income Finnish 11229 2415000 215.07 182.87 85.03 0.27 9.63 30.40 41.57 69.23 174.35 285.02 486.45 610.44 819.42 1331.3 calibration Finnish and poverty 11229 2415000 215.07 183.57 85.36 0.27 10.00 31.43 42.38 69.22 172.64 286.69 486.94 613.22 819.18 1341.6 The difference between the non-response corrected design weights and calibrated weights is clear. The calibrated weights have several constraints which affect the weight structure, in this case making them more spread and providing also clearly lower and higher values than the adjusted design weight. In order to get all 3 a macro for standard error calculation of the estimators consisting of the functions of totals (including calibration), developed in Statistics Sweden 4 a macro for calibration, developed in INSEE
8 the calibration models through, a minimum of 0.2 was set, i.e. weights below one were allowed for testing. As the calibration theory indicates, the given number of households equals the sum of calibrated weights. The indicators are calculated with the SAS macro code provided by Eurostat for the national EU-SILC calculations. The bootstrap variance estimation method is utilised for the non-linear indicators. The principle is such that in every stratum a resample without replacement is selected from the set of respondents (Bickel and Freedman 1985, see also Sitter 1992). The size of the resample is n r,h = (16/17)*n h (rounded to an integer), where 16 is one less than the number of respondents in the smallest stratum. This operation provides an easy solution to the variance correction, which is required because the sampling fraction in the resampling design (i.e. a constant 16/17 for every stratum) is much higher than in the original design (see Ollila 2004 for more theoretical details). The overall resample size is 10569. Every resample is then calibrated under the calibration model selected for testing. With these weights, valid and right-scale estimates can be obtained for every resample as well. For the bootstrap application the code of the Eurostat indicator macros is slightly modified in order to adjust it to the changing data sets and weights from one replication to another. After calculating the estimates for the resample they are saved into a cumulative data set. At the end of the process this set of estimates is used for variance estimation. The estimator for the variance of the estimator θˆ at the second phase level is of the form (1 11229/ 75000) Vˆ( ˆ) = (1 16/17) ( ˆ θ ˆ) θ A g g= 1 A θ, where the term before summation adjusts the difference between the original second phase sampling design and resampling design in the variance formula. The standard error of the estimator is the square root of the variance. Note that one cannot correct this way within strata, because most of the indicators require percentiles, which must be calculated at the whole sample level, not in strata. The simulation tests provided in the Finnish EU-SILC intermediate quality report from the first year 2004 show that the effect of the first phase to the variance is marginal. The number of resamples A is 1000, because the process with several calibration models and a lot of indicators and their subgroups to be calculated every time a new resample is selected is laborious. On the other hand, the resampling fraction 16 / 17 94 % using resampling without replacement assures a major cover of the units in every stratum, e.g. in small strata the number of different combinations of resamples from the set of respondents is not big. The variance estimation of the estimators of the linear parameters is conducted with CLAN. For the linear results obtained with non-response corrected design weighting also proc surveymeans of SAS and proc descript of Sudaan 5 in the SAS platform were used for checking. The Total Statistics on Income Distribution describe the annual income of register households and their distribution especially from the regional perspective. The statistics depict the amount of income and its formation from different income sources when taking taxation and income transfers into consideration. The main difference between the results of the Income Distribution Survey / EU-SILC and the Total Statistics on Income Distribution is that for the latter only the dwelling unit identification is available, not the structure of the household as it is defined in the survey. Törmälehto s study (2004) revealed that in Finland about 85 % of the dwelling units coincide with the households. Some indicators from the Finnish Total Statistics on Income Distribution of year 2004 (= SILC year 2005) are used here in order to take a look at the underlying bias. Results The six different weighting strategies are compared in different contexts: 1 2 5 A software for the analysis of complex surveys by RTI.
9 - the estimators and their standard errors of the linear parameters at the household level (variables 1-16), at the 16+ person level (variables 17-26), and at the all-persons level (equivalised disposable income) with subgroups; - the consistency of CLAN and bootstrap standard error estimates - the estimators and their standard errors of the non-linear indicators with some subgroups - the bias study Table 5. Number of Observations, Means and Standard Errors rors for Components of Income with Different Weightings Components of income Obs. No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty mean s.e mean s.e mean s.e mean s.e mean s.e mean s.e 1 Total household gross income 11229 38932 414 40279 536 40004 523 39529 526 39309 80 39322 79 2 Total disposable household 11229 28470 281 29330 361 29164 353 20927 359 28780 53 28790 53 income 3 Total disposable household 11229 24458 280 25309 365 25066 351 24835 357 24689 57 24663 56 income, before social transfers other than old-age and survivors' benefits 4 Total disposable household income, before social transfers including old-age and survivors' benefits 11229 19949 275 20953 363 20714 349 20522 356 20384 50 20377 50 5 Imputed rent 11229 3213 30 3254 28 3239 28 3218 27 3207 26 3195 25 6 Income from rental or property 11229 320 18 330 19 328 19 327 19 349 14 348 13 or land 7 Family/children-related 11229 1059 26 1030 15 1033 15 1035 15 1037 15 1035 15 allowances 8 Social exclusion payments not 11229 155 8 157 9 164 9 163 9 168 9 169 8 elsewhere classified 9 Housing allowances 11229 345 10 336 10 343 10 346 10 351 10 364 0.4 10 Regular inter-household cash 11229 139 8 124 7 124 7 126 7 126 7 126 7 transfers received 11 Interest, dividends, profit from 11229 2500 356 2755 500 2725 493 2738 507 2592 53 2597 54 capital investments in unincorporated businesses 12 Interest paid on mortgages 11229 507 11 517 10 512 10 506 10 492 2 492 2 13 Income received by people 11229 46 6 45 6 45 6 45 6 45 5 43 5 aged under 16 14 Regular taxes on wealth 11229 124 13 130 15 129 15 128 15 125 8 125 8 15 Regular inter-household transfers paid 11229 194 9 202 9 200 9 197 9 197 9 196 9 16 Tax on income and social 11229 10143 129 10618 167 10511 162 10277 157 10208 37 10212 37 insurance contributions 17 Cash or near-cash employee income 22961 13770 107 14160 138 13948 130 13701 107 13696 107 13696 107 18 Non-cash employee income 22961 101 5 107 6 105 6 98 6 99 6 100 6 19 Contributions to individual 22961 137 5 142 6 141 6 137 6 136 6 137 6 private plans 20 Gross cash profits or losses 22961 1268 43 1291 63 1318 64 1336 61 1295 37 1295 37 from self -employment (incl. royalties) 21 Unemployment benefits 22961 848 22 842 31 868 28 865 31 846 24 846 24 22 Old-age benefits 22961 3214 46 3043 62 3038 62 3000 53 2993 51 2980 52 23 Survivors' benefits 22961 97 9 95 11 94 11 93 11 92 11 90 11 24 Sickness benefits 22961 96 6 97 9 98 9 100 9 100 9 99 9 25 Disability benefits 22961 736 27 733 36 748 34 738 34 748 33 764 33 26 Education-related allowances 22961 128 6 130 8 129 8 132 8 131 7 130 6 * Households which have negative values or 0 values in the variable are counted as the households which have not received the income. Negative values of the certain gross income components in which they exist are counted in the variable HY010 on the total household gross income. The most notable results can be seen in the income variables 1 4, 11, 12, 14 and 15. Almost always the non-response corrected design weight outperforms the demographic calibration and additional status and
10 income information, when the standard error is considered. Although fulfilling the required demographic conditions, it seems that the basic calibration criteria of age & gender classification for the persons of the household together with the size of the household, region and type of municipality, twist the weights to an unwanted direction when considering the standard error. The effect of calibration in the five-year classes of the population targets also the children of the household, who do not earn or obtain money in practice. The positive impact of that aspect of calibration can be seen in variable 7 (family/children related allowances) where the design weight takes only the number of persons 16 years or over into account, but the calibration includes also the category information about the children. However, the intensive use of the income totals in different forms in calibration (the Finnish alternative) pays off, for these income variables mentioned earlier we have significant improvement in standard errors. In variable 12 (interest paid on mortgages) the improvement is evident: the resembling variable is included also in the calibration criteria. The results are disappointing especially for the demographic + status variables alternative. Also the demographic + two income variables alternative shows almost non-existing improvement except in variables 17 and 22, which happen to be near these two income variables from the register. The poverty variables addition to the Finnish alternative does not bring any significant help to the efficiency of estimation except variable 9, which can be found in these poverty variables as well. It is obvious that calibration with the demographic variables brings consistency to estimation especially when these groups are studied in the analysis of the data. Considering the estimates produced by six weighting strategies, it is not clear that for every variable the demographic calibration provides more accurate point estimates than the non-response corrected design weight. In some cases the point estimates of no calibration and intensive calibration are close to each other and farther from demographic calibration. Table 6. Number of Observations, Mean and Standard Errors for Equivalised Disposable Income in Household, Age and Gender Groups Equivalised disposable income Obs. No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty mean s.e mean s.e mean s.e mean s.e mean s.e mean s.e All 29112 19374 163 19750 198 19639 194 19469 196 19390 56 19401 56 1 household member 2390 15451 403 15791 814 15724 806 15641 804 15440 578 15412 581 2 household members 8414 21241 376 21767 467 21627 458 21454 468 21268 330 21292 329 3 household members 5808 20942 410 21409 439 21261 428 21102 435 21193 433 21199 433 4 household members or more 12500 18877 176 19108 119 19020 118 18802 120 18805 123 18805 123 Age group <25 years 9720 17652 129 18078 239 17988 237 17803 236 17787 236 17787 237 Age group 25-34 years 2812 20325 938 21050 1108 20923 1089 20875 1127 20497 746 20497 744 Age group 35-44 years 3861 20461 190 20692 445 20584 442 20396 437 20358 437 20358 436 Age group 45-54 years 4941 21916 271 22052 426 21920 422 21689 419 21505 396 21505 395 Age group 55-64 years 4465 22774 577 23116 1066 22910 1053 22690 1061 22704 721 22704 724 Age group 65- years 3313 15972 181 16000 345 15961 341 15838 308 15854 289 15854 294 Male 14739 19861 210 20230 388 20112 383 19929 384 19809 264 19809 264 Female 14373 18921 157 19291 271 19187 265 19029 269 18989 224 18989 224 Table 6 shows equivalised disposable income in all and in categories by household size, age group and gender. Now we are dealing with the full data including children as well. The overall results follow the same phenomenon as in the previous components of income. The key results can be found in the classification by household size. The number of observations reflects the sampling design of the Finnish EU-SILC: the persons are first selected and then the dwelling units and later households are constructed around the selected persons. The inclusion probability increases when the number of 16+ persons in the dwelling unit increases. The most effective results for calibration come from class 4 household members or more, which is the class where gender & age calibration might have the largest impact via children. Another end is class 1 household member, where the best standard error is for the no calibration alternative. There is some difference between the point estimates of simple calibration alternatives and no calibration/intensive calibration. These results were also verified by using the bootstrap variance estimation method (Table 7).
11 Table 7. Comparing CLAN and Bootstrap Standard Errors for Equivalised Disposable Income No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty CLAN Boot CLAN Boot CLAN Boot CLAN Boot CLAN Boot CLAN Boot Estimates 19374 19372 19750 19749 19639 19638 19469 19468 19390 19388 19401 19399 Standard errors 163 164 198 220 194 216 196 223 56 60 56 60 The structure of the parameter (equivalised disposable income) is not complex, and the resamples were selected with simple random sampling without replacement. Thus the standard errors should be rather close to the CLAN results, if both procedures were carried out correctly. The bootstrap standard errors seem to be slightly larger than the CLAN standard errors. This may come from the fact that the scale correction of the bootstrap variance estimate must be conducted at the overall level, not at the stratum level. This generalisation of scale correction ignores the non-proportional allocation which affects the estimation of the equivalised disposable income, when the stratification is taken into account in CLAN. The variance of an estimator of a complex parameter might not be an easy task to be estimated. The theoretical complexity is avoided here with a resampling technique. The results for the main non-linear indicators are presented in Table 8. The estimates come from the calculations with the full set of respondents, and in practice the bootstrap means of the resample estimates differ mostly one decimal maximum, usually less than that. Table 8. Main Non-linear Indicators and Their Standard Errors Indicator At-risk-of-poverty rate after social transfers Inequality of income distribution S80/S20 income quintile share ratio Relative median at-risk-ofpoverty gap Dispersion around the riskof-poverty threshold (40) Dispersion around the riskof-poverty threshold (50) Dispersion around the riskof-poverty threshold (70) At-risk-of-poverty rate before social transfers except old-age and survivors benefits At-risk-of-poverty rate before transfers including old-age and survivors benefits Inequality of income distribution: Gini coefficient Obs. No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty estim. s.e estim. s.e estim. s.e estim. s.e estim. s.e estim. s.e 29112 11.65 0.52 11.67 0.51 11.80 0.56 11.60 0.51 11.69 0.51 11.78 0.52 29112 3.63 0.08 3.64 0.11 3.65 0.11 3.60 0.11 3.59 0.04 3.61 0.04 29112 13.70 0.60 13.63 0.45 13.45 0.51 13.43 0.67 13.45 0.84 13.40 0.79 29112 2.11 0.17 2.04 0.16 2.05 0.17 2.05 0.17 2.09 0.17 2.11 0.17 29112 5.04 0.28 4.93 0.28 5.01 0.28 4.95 0.27 5.00 0.27 5.01 0.28 29112 20.43 0.48 20.48 0.47 20.51 0.46 20.54 0.48 20.52 0.50 20.70 0.50 29112 27.73 0.42 27.44 0.39 27.88 0.38 27.93 0.38 28.07 0.38 28.32 0.39 29112 40.98 0.38 39.73 0.39 40.26 0.37 40.31 0.37 40.46 0.35 40.47 0.35 29112 25.90 0.58 26.03 0.80 26.08 0.79 25.82 0.83 25.71 0.22 25.81 0.22 The point estimates don t differ much from one weighting to another. Most notable differences can be found in relative median at-risk-of-poverty gap, at-risk-of-poverty rate before social transfers except old-age and survivors benefits, and at-risk-of-poverty rate before transfers including old-age and survivors benefits. The standard errors of the intensive calibration alternatives are clearly lower with Gini coefficient and S80/S20, and correspondingly the basic calibration alternatives perform less well than the non-response
12 corrected design weight. On the other hand, the "at-risk" and "dispersion" indicators don't show any significant difference. The "relative median" indicator doesn't follow the usual standard error patterns. Table 9. Non-linear Indicators and Their Standard Errors for Some Subgroups Indicator No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty estim. s.e estim. s.e estim. s.e estim. s.e estim. s.e estim. s.e At-risk-of-poverty rate after social transfers Male 10.64 0.52 10.50 0.62 10.67 0.64 10.52 0.62 10.65 0.61 10.70 0.62 Female 12.60 0.74 12.79 0.72 12.89 0.77 12.64 0.72 12.70 0.72 12.81 0.73 0 15 10.00 1.17 9.98 1.13 10.35 1.22 10.13 1.14 10.29 1.23 10.23 1.20 16+ 12.08 0.69 12.08 0.74 12.15 0.72 11.95 0.68 12.03 0.68 12.15 0.69 16 64 10.87 0.67 10.69 0.73 10.80 0.75 10.62 0.67 10.74 0.67 10.74 0.68 65 + 17.37 2.38 18.58 2.64 18.40 2.58 18.19 2.51 18.05 2.64 18.74 2.86 At work 3.65 0.22 3.74 0.22 3.80 0.25 3.77 0.23 3.83 0.23 3.80 0.23 not at work: total 21.12 0.67 21.49 0.70 21.39 0.65 21.07 0.63 21.14 0.68 21.39 0.68 not at work: unemployed 34.90 1.63 35.50 1.96 35.77 1.77 34.92 1.77 35.65 1.81 35.56 1.86 not at work: retired 16.29 0.87 17.56 0.80 17.14 0.83 16.89 0.84 16.78 0.86 17.32 0.87 not at work: other inactive 25.24 1.05 23.13 1.23 23.29 1.08 23.02 1.02 23.23 1.04 23.11 1.03 Total no dependent children 14.05 1.06 14.06 1.09 14.00 1.03 13.85 1.03 13.91 1.07 14.10 1.04 1 person (total) 29.49 2.29 29.86 2.48 29.66 2.32 29.16 2.35 29.20 2.44 29.46 2.32 2 adults, both < 65 years 6.59 1.12 6.54 1.26 6.58 1.18 6.67 1.17 6.78 1.18 6.79 1.21 2 adults, at least one 65+ 7.79 1.71 8.36 1.77 8.28 1.77 8.26 1.80 8.18 1.80 8.73 2.15 Other no dependent children 4.50 2.79 4.22 2.32 4.41 2.33 4.43 2.39 4.48 2.43 4.46 2.56 Total dependent children 8.67 0.80 8.67 0.74 8.97 0.76 8.79 0.77 8.95 0.78 8.90 0.79 Single parent, at least 1 17.73 3.35 18.36 3.63 18.83 3.83 18.10 3.55 18.68 3.64 18.65 3.75 dependent child 2 adults, 1 dependent child 7.08 1.66 7.06 1.63 7.33 1.63 7.27 1.66 7.44 1.69 7.47 1.65 2 adults, 2 dependent children 5.60 1.38 5.98 1.44 6.22 1.44 5.91 1.42 6.05 1.34 6.00 1.41 2 adults, 3 dependent children 11.37 2.07 11.84 1.90 12.21 1.97 12.17 1.96 12.24 2.06 12.09 2.05 Other households with dependent children 6.36 3.50 6.43 2.89 6.52 3.02 6.60 3.20 6.59 3.15 6.63 3.27 Relative median at-risk-of-poverty gap Male 14.55 1.10 14.29 0.82 14.37 1.29 14.50 1.03 14.58 1.02 14.39 0.99 Female 13.21 0.90 13.19 0.44 12.91 0.46 12.82 0.64 12.79 0.69 12.82 0.71 0 15 9.98 2.27 10.26 1.99 10.06 1.98 10.25 2.11 10.35 2.04 10.29 2.8 16+ 14.73 0.71 14.17 0.50 14.15 0.83 14.26 0.79 14.33 0.86 14.23 0.84 Dispersion around the risk-of-poverty threshold (40) Male 2.36 0.21 2.27 0.19 2.30 0.19 2.31 0.18 2.35 0.19 2.34 0.19 Female 1.84 0.15 1.76 0.14 1.78 0.15 1.78 0.16 1.79 0.16 1.78 0.16 Dispersion around the risk-of-poverty threshold (50) Male 4.87 0.26 4.74 0.27 4.82 0.27 4.75 0.26 4.82 0.27 4.82 0.27 Female 5.14 0.30 5.12 0.29 5.11 0.28 5.07 0.30 5.08 0.29 5.12 0.29 Dispersion around the risk-of-poverty threshold (70) Male 18.87 0.46 18.74 0.46 18.89 0.45 18.85 0.48 18.95 0.50 19.08 0.49 Female 21.92 0.51 21.96 0.48 22.12 0.49 21.98 0.50 22.05 0.50 22.20 0.51 The subgroup results must be analysed cautiously. It is obvious that the estimates and the standard error estimates in rare groups are subject to uncertainty, e.g. class "other households with dependent children". In Table 8 the "at-risk-of-poverty-rate after social transfers" indicator included not much difference between weighting systems, and the same phenomenon can be noticed in most of the groups (excluding small rare groups). The "dispersion" standard error figures do not show notable difference either. As in Table 8, the "relative median at-risk-of-poverty gap" seems to provide results which are difficult to be interpreted.
13 Table 10. Comparing Indicators Based on Total Statistics on Income Distribution with Various Estimates Indicator At-risk-of-poverty rate after social transfers Inequality of income distribution S80/S20 income quintile share ratio Inequality of income distribution: Gini coefficient Register statistics No calibration Demographic Demographic + status variables Demographic + income Finnish Finnish + poverty mean bias mean bias mean bias mean bias mean bias mean bias 13.7 11.7-2.0 11.7-2.0 11.8-1.9 11.7-2.0 11.7-2.0 11.8-1.9 2.1 3.6 1.5 3.6 1.5 3.6 1.5 3.6 1.5 3.6 1.5 3.6 1.5 28.3 25.9-2.4 26.1-2.2 26.1-2.2 25.8-2.5 25.7-2.6 25.8-2.5 The planned idea to study the bias of the estimates of different weighting strategies with the Total Statistics on Income Distribution proved to be not very fruitful. Firstly, only three indicators were available from the data of the Total Statistics, because the amount of variables available from the registers is much more limited than in the Income Distribution Survey. Furthermore, the definitions behind the calculation of the indicators of the Total Statistics are different, and the calculations are based on the dwelling unit, not on the household. Even if the register figures were defined in the same way as in the survey, the differences between the weighting strategies were very modest, and there wouldn't have been strong conclusions in that situation either. Conclusions and Recommendations The results show some interesting phenomena in the context of the Finnish EU-SILC data. Although the demographic calibration (gender & age groups at the person level, household size and region at the household level) provides consistent estimates with those demographic external sources, with some estimators it seems to weaken the efficiency of estimation when compared with the non-response corrected design weights. The effect of calibration in the five-year classes of the population covers also the children of the household, who do not earn or obtain money in practice. On the other hand, the estimators concerning the age & gender categorisation (e.g. family/children related allowances) benefit also from the basic demographic calibration. Two simple demographic calibration models with adding 1) status information on persons (working, on pension, unemployed) or 2) two income variables (amount of salary, amount of pension) were tested. The idea was such that some member countries could use the totals for those variables from another sources, provided that the register and survey variables describe the same thing sufficiently. The results were disappointing. The efficiency improvement was marginal except with variables strictly dealing with the calibration variables (Cash or near-cash employee income and old-age benefit). The Finnish calibration practice with a lot of income information added to the demographic calibration seems to improve the efficiency of estimation clearly. Unfortunately, this situation with registers is a rare exception among the EU-SILC countries. The addition of some variables reflecting to poverty did not improve the Finnish calibration practice much, except for housing allowances. Concerning the complex indicators, the phenomenon described above for different weighting strategies occurred with "gini coefficient" and "S80/S20 income quintile share ratio". The other indicators were hardly affected. Note that in Finland full income information for every member of the household is obtained from registers, which is not the case for the majority of the other EU-SILC countries. Furthermore, the Finnish sampling design already includes socio-economic and high/low income categorisation, and this may cause the design
14 to be rather effective even before the calibration. It might happen that in another context the results can be somewhat different. One must be cautious to draw strong conclusions and recommendations based on this testing, especially when applying them in the non-register based household selecting countries, where the sampling designs differ a lot from the Finnish version. Still, the effect of demographic calibration compared with the nonresponse corrected design weights should be tested in that context as well. The results of this study do not provide evidence for recommending small scale variable additions for calibration. However, the good properties of fixing the main demographic variables with calibration are important as such, so there is no need at this point to reject the demographic calibration. References Bickel, P. J and Freedman, D. A. (1985). Asymptotic Normality and the Bootstrap in Stratified Sampling. The Annals of Statistics, 12, 470-482. Deville, J.-C. and Särndal, C-E. (1992). Calibration Estimators in Survey Sampling. Journal of the American Statistical Association, Vol. 87. No. 418, Theory and Methods, 376-382. Eurostat (2004). Technical Document on Intermediate and Final Quality Reports. Document EU-SILC 132/04. Ollila, P. K. (2004). A Theoretical Overview for Variance Estimation in Sampling Theory with Some New Techniques for Complex Estimators. Doctoral Thesis, University of Helsinki, Department of Statistics, The Research Reports of Statistics Finland, Helsinki. Sitter, R. R. (1992). Comparing Three Bootstrap Methods for Survey Data. The Canadian Journal of Statistics, 20, n:o 2, 135-154. Törmälehto, V-M. (2004). Kotitalous haastattelusta vai rekisteristä? Unpublished research paper of Statistics Finland (in Finnish). Verma, V. (2006). EU-SILC Weighting Procedures: an Outline. Eurostat Output II.1(b).