Precision Requirements in SASU

Transcription

1 Precision Requirements in SASU Martins Liberts January 20, Precision Requirements in the Regulation The precision requirements are defined in the article 52 of the regulation [3]: 2. The sample size to be achieved, calculated on the assumption of simple random sampling shall be a minimum of: a persons in Member States with a population aged 16 and over which is higher than 10 million; b persons in Member States with a population aged 16 and over which is higher than 5 million and lower than 10 million; c persons in Member States with a population aged 16 and over which is higher than 1.5 million and lower than 5 million; d persons in Member States with a population aged 16 and over which is higher than 0.5 million and lower than 1.5 million; e persons in Member States with a population aged 16 and over which is lower than 0.5 million. The sample size defined is so called effective sample size. The precision requirements are defined by the effective sample size in this case. This is one of the approach how precision requirements could be defined the similar approach is used for EU-SILC. The approach is not recommended usually because it defines the precision requirements in ambiguous way. Recommended approach is to define precision requirements by variance measures coefficient of variation or standard error [2]. Several conclusions can be made from the precision requirements defined: Firstly the effective sample size is expressed as number of persons. However it relates both to persons estimates and households estimates households estimates can be achieved also from sample of persons. Secondly the precision requirements are set only for a variance. A bias is not considered. The precision requirements are set for the estimates of the main indicators defined at the point 1 of the annex I of the regulation [3]: 1. Prevalence rate of car theft, 2. Prevalence rate of theft from cars, 1

2 3. Prevalence rate of motorcycle theft, 4. Prevalence rate of bicycle theft, 5. Prevalence rate of burglary in main home, 6. Prevalence rate of robbery, 7. Prevalence rate of theft of personal property, 8. Prevalence rate of consumer fraud, 9. Prevalence rate of card / on-line banking abuse, 10. Prevalence rate of bribery backhanders. The first five prevalence rates are household related and the last five prevalence rates are persons related. Let c be a given type of crime. The first five indicators are defined by the formula 1 and last five indicators are defined by the formula 2. P H c = M c P P c = N c 2 N P H c and P P c denotes the prevalence rate of households and persons respectively; M c and N c denotes number of households or persons having fallen victim to the crime c respectively; M and N denotes total number of households and persons in the target population respectively in formulas 1 and 2. The precision requirements are set to all other estimates of indicators indirectly. We can expect good enough precision for many estimates if the precision requirements will be satisfied for the estimates of the main indicators. The interpretation of the precision requirements defined by an effective sample size are given by the definition 1. Definition 1 All ten main indicators defined at the point 1 of the annex I of the regulation [3] has to be estimate with variance not higher than theoretical variance of the same indicator estimated using simple random sampling SRS of persons, π estimator known as Horvitz-Thompson estimator, in case where non-sampling errors do not exist perfect situation with full response and error free sampling frame denoted by FR = Full Response and with sample size of persons equal to the effective sample size. 1.1 The Proportions of persons Assume the indicator of interest is the proportion of persons θ = P P. The variance of PˆP under SRS of persons, π estimator, in case of FR and with sample size equal to the effective sample size can be computed as: M 1 ˆ P P SRS P, π, F R, = 1 f S2 3 2

3 where S 2 = N N 1 P P 1 P P and f = N. We will call the variance 3 as the benchmark variance. P ˆ P can be approximately computed as: ˆ P P SRS P, π, F R, P P 1 P P 4 if N is large enough and N. The variance estimator of 4 is PˆP 1 ˆ P P ˆ P ˆ P SRS P, π, F R, 5 Precision requirements for the proportion of persons will be satisfied if ˆ P ˆ P P ˆ P1 P ˆ P for the five main P ˆ P relating to persons, where ˆ P ˆ P is a variance of Pˆ P under the current sampling design, estimated by the current estimator, with a current response pattern and using initial sampling size n I. 1.2 The Proportions of households Assume the indicator of interest is the proportion of households θ = P H. The benchmark variance is denoted by P ˆ H SRS P, π, F R,. This is a theoretical variance of PˆH using simple random sampling SRS of persons, π estimator, in case where non-sampling errors do not exist and with sample size of persons equal to the effective sample size. The P ˆ H SRS P, π, F R, can not be expressed in such a simple form as 4 unfortunately. This is because the SRS of persons do not lead to SRS of households. Precision requirements for the proportion of households will be satisfied if ˆ P ˆ H ˆ P ˆ H SRS P, π, F R, for the five main relating to households, where ˆ P ˆ H is a variance of ˆ P H ˆ P H under the current sampling design, estimated by the current estimator, with a current response pattern and using initial sampling size n I. 1.3 The Concludions 1. The precision requirements are set for the estimates of the ten main indicitors defined at the point 1 of the annex I of the regulation [3]; 2. The evaluation of the estimates of persons is trivial. We have to be careful with the evaluation of the estimates of households because the estimation of theoretical variance of household proportion under simple random sample of persons is not trivial. 3. The sampling of persons the sample of households is derived from the sample of persons seems to be the optimal choice as sampling design. It comes from the fact that benchmark variances are based on sampling of persons. 3

4 4. The sampling of household the sample of persons is derived from the sample of households would be good choice for the estimates of households indicators but it would require much bigger sample to fulfil the precision requirements for persons indicators. It comes from 6 and 7. P ˆ H SRS H, π, F R, P ˆ H SRS P, π, F R, P ˆ P SRS H, π, F R, P ˆ P SRS P, π, F R, Desigffect The effective sample size usually is connected with a design effect. We will introduce a design effect here. It will be used latter. Different definitions and terminology are used for a design effect. Design effect is defined only from theoretical point of view usually. Non-sampling errors as non-response or over-coverage are not considered regarding a design effect usually. The non-sampling errors are unavoidable in practice. A design effect has to be redefined incorporating non-sampling errors so it could be used in practical situations. A design effect is defined in 8. ˆθ P, E, R, n Deff = 8 ˆθ SRSP, π, F R, n where P is the current sampling design used, E is the current estimator used for ˆθ, R is the current response pattern, n is a sample size, SRS is a simple random sampling, π is π estimator and F R is full response pattern non-existence of non-sampling errors. Please note the usage of SRS P in the denominator this is specific for SASU because the effective sample size is defined by the number of persons. I will refer to ˆθ SRSP, π, F R, n as the benchmark variance. Def f can be disaggregated into three components: I will define: Deff = deff 1 deff 2 deff 3 9 deff 1 as the effect of sampling design. Usually stratification correlating with study variable will decrease deff 1, clusterization related to study variable will increase deff 1, etc. deff 2 as the effect of estimator. Estimators using auxiliary information ratio, GREG or calibration related to study variable will decrease deff 2. deff 3 as the effect of non-sampling errors. Non-sampling errors nonresponse, over-coverage, etc. will increase deff 3. All three effects show the effect of a sampling design, an estimator and non-sampling errors regarding the benchmark variance. Def f is multiplicative overall effect influencing the benchmark variance. 4

5 From 8 follows ˆθ P, E, R, n = ˆθ SRSP, π, F R, n Def f 10 The precision requirements can be reformulated as ˆθ SRSP, π, F R, n I Deff ˆθ SRSP, π, F R, 11 where n I is the initial sample size and is the effective sample size. Assume the indicator of interest is proportion of persons θ = p. 11 can be written as p 1 p p 1 p Deff 12 n I n I Deff 13 n I deff 1 deff 2 deff 3 14 Fulfillment of 14 will result with the fulfillment of precision requirements defined. 3 The Estimation of the Initial Sample Size The initial sample size necessary to achieve the precision requirements for the estimate of one indicator can be estimated as: ˆn I = deff ˆ 1 deff ˆ 2 deff ˆ 3 15 Please note that the estimation of deff ˆ 1, deff ˆ 2 and deff ˆ 3 is complicated especially in planing stage of the survey. deff 1 depends from various sampling design components. deff 1 can be disaggregated into more components: deff 1 = deff 1str deff 1cl 16 where deff 1str is stratification effect and deff 1cl is clusterization effect. Usually deff 1str 1. I do not see any practical method how deff 1str could be estimated. One aproach would be to set deff 1str = 1 to be on a safe side. deff 1cl could be estimated as ˆ deff 1cl = 1 + b 1 ˆρ 17 where b is the mean number of sample units per cluster and ˆρ is the estimated intra-household correlation coefficient [4, p. 5]. deff 2 depends on relation between study variable and auxiliary information used in estimator. Stronger relation will decrease the value of deff 2. deff 3 depends on number of respondents from the initial sample. deff 3 could be estimated as: ˆ deff 3 = 1 1 OC ˆ r 1 NR ˆ 18 r 5

6 where OC ˆ r is an expected unweighted over-coverage rate [1, p. 5] and NR ˆ r is an expected unweighted non-response rate [1, p. 8]. It is clear that deff 3 will always be greater than 1, because non-sampling errors are unavoidable. The initial sample size can not be the same as the effective sample size even if somebody would choose to use SRS and π estimator. The initial sample must be increased in this case because of deff 3. It is obvious that Deff mainly deff 1 and deff 2 strongly depends on the indicator in question. To satisfy the precision requirements the maximum value of Deff has to be estimated over the ten main indicators and the size of initial sample size can be computed as: ˆn I = max Deff ˆθj, j = where max Deff ˆθj maximum design effect selected from the estimates of the ten main indicators. 4 The Evaluation of the Precision The achieved precision has to be evaluated after the survey has been done. There are two options how to do it. The first option is to state that precision requirements are fulfilled if ˆθ ˆθ SRSP, π, F R, 20 for all main ˆθ. If θ is proportion of persons 20 can be rewritten as: ˆp 1 ˆp ˆp 21 If θ is proportion of households ˆθ SRSP, π, F R, has to be estimated. The second option is to compute the achieved effective sample size. The achieved effective sample size n AE can be computed as: or n I n AE = 22 Deff ˆθ n AE = n I max Deff ˆθj, j = Please note that 22 is computed for one indicator but 23 is computed for the ten main indicators. Precision requirements will be fulfilled if n AE. 6

7 References [1] Eurostat 2010 ESS Guidelines for the Implementation of the ESS Quality and Performance Indicators [2] Eurostat, ESS Handbook on ariance Estimation and Precision Requirements for household surveys DRAFT [3] Draft Regulation of the European Parliament and the Council ouropean statistics on safety from crime version available on CIRCA [4] Guillaume Osier, Desigffect Estimation 7