ANALYSIS OF DESIGN EFFECTS AND VARIANCE COMPONENTS IN MULTI -STAGE SAMPLE SURVEYS

Transcription

1 1. INTRODUCTION ANALYSIS OF DESIGN EFFECTS AND VARIANCE COMPONENTS IN MULTI -STAGE SAMPLE SURVEYS R. Platek and G.B. Gray, Statitic Canada a) General Survey ample technique have been in ue for many year, and among probability ample, there are many type. They vary from imple random ampling to a complex multi -tage tratified ample deign and it i the latter type which i the ubject of thi paper. b) Need for Multi -Stage Sample Deign A particular urvey deign i determined by a variety of factor uch a objective of the urvey, method of enumeration, availability of cenu data, reliability of etimate required, and frequency of urvey. The availability of fund uually play an important, practical part in determining a urvey deign. Any ample deign, whether it ue peronal interview olely, or in conjunction with telephone or mail, require clutering of the ample to ome degree. Even mail urvey with peronal follow -up of non -repondent may require clutering to a certain extent. Although clutering uually reduce the urvey cot, the ampling variance of etimate of mot tatitic i increaed. In order to introduce clutering in a urvey deign, a certain amount of information about the population i required. Cenu data may be ued, for example, to delineate trata and primary ampling unit, while more up -todate information on area within primary ampling unit may be ued to delineate ub -ampling unit in ubequent tage. The periodicity or frequency of a particular urvey affect the type of urvey deign and the degree of clutering. For example, a continuou urvey uually require a permanent organization and permanent interviewer who mut be trained and controlled. c) Need for Component of Variance For the ole purpoe of obtaining meaure of reliability of etimate, uually confined to etimate of ampling variability, it i elfom neceary to plit up the variance into component and to etimate each component. In a multi -tage deign however, when total variance are tudied, quetion frequently arie a to the contribution of variou tage to the total variance. Examination of the relative magnitude of variance component throw ome light on the alternative ample deign through change in the ize of the unit, or change in the allocation of ample unit that might be a- dopted to decreae the ampling variance for a given budget, or decreae the cot for a given ampling variance. Clearly, the anwer to either of thee require a cot analyi in conjunction with a variance analyi and we are aware that by providing variance analyi without cot analyi the tudy i incomplete. However, the magnitude of the component at certain tage may determine the deirability or neceity of iolating cot component at the ame tage. In ome intance, a variance component may be unacceptably high regardle of the correponding cot component. Total ampling variance and deign effect [7] enable u to evaluate the gain or loe a a reult of tratification, clutering and ratio etimation compared with imple random ampling and imple etimation. However, in order to analyze the deign effect more fully, component of variance are required. For example, low deign effect may mean that clutering play no part in increaing the ample variance o that component of variance other than thoe between ultimate unit may be quite low, while high deign effect may indicate an extenive clutering effect which would lean toward high component of variance in at leat one other tage beide the between ultimate unit component. Alo, the ize of the ampling unit at a given tage of ampling uually affect the variance component, ince the population variance of which the variance component i a function, almot alway increae with the ize of a unit fater than by a linear relationhip with the ize. The reaon for the fater than linear increae i a poitive intracla correlation [5]. Deign effect by tage of ampling may alo be calculated (See Defini- tion in Section 4b). 2. VARIANCE COMPONENTS AND ANALYSIS Variance component were undertaken for about 40 characteritic in June, 1973, for the Canadian Labour Force Survey, uing Yate -Grundy etimation formula [9], and further developed by Gray [3] and [4]. More detail on the ample deign of the Canadian Labour Force Survey may be obtained from [2] and [8], and the detail related to variance component etimation may be obtained from [4]. The tudy of variance component baed on the reult from the LFS lead to two baic type'of analyi, (a) a tudy of the variance component a percentage of the total variance and their relationhip to deign effect [7], and (b) a tudy of a variance function in term of average weight and number of unit at variou tage and the population variance for which individual deign effect may be obtained. 3. DESIGN EFFECTS AND CATEGORY OF CHARACTERISTICS On the above broad definition of the type of characteritic a well a the oberved deign effect, the 19 characteritic were grouped into three categorie, maintaining a much a poible the common category between urban and rural area. 6 73

2 There i a rather high correlation of.860 between the deign effect of the characteritic for urban and rural area and without either "Finance and Inurance" or "Non-Agriculture Employed", the correlation increae to.907. The deign effect in the urban and rural area have alo been ranked in order to how more clearly the correlation between the two et of obervation. In general, the clutering effect a een by the magnitude of the deign effect i more ignificant in the rural area than in the urban area, the average deign effect being and in the urban and rural area repectively. Although obervation pertaining to category I are lacking in thi tudy, monthly variance etimate of other categorie uing the Keyfitz formula [6] have revealed a imilar phenomenon of low deign effect among characteritic pertaining to pecific age -ex categorie. a) General 4. ANALYSIS OF COMPONENTS OF VARIANCE Each tage of ampling reult in ome contribution to the total ampling variance. The magnitude of the component vary from tage to tage and by characteritic within each tage of ampling. Thee have been calculated uing method developed in [4] for many Labour Force Survey characteritic in mall urban and rural area acro Canada where four tage ampling wa applied. The reult are given in Table 2 for Urban and Rural. The percentage of the component of variance are tudied in relation to the average weight and ize of ampling unit at the variou tage of election. (b) Obervation from Table 2 Dealing With Variance Component i) The average weight W (i.e. the total number of rth tage uni within all ampled (r -1)th tage unit divided by the ampled number of unit within all ampled (r -1)th tage unit, "0" being taken a the tratum level) for each tage of ampling. ii) The average ize of rth tage unit. and averaged over the characteritic ofreach category. iii) The average percentage contribution for each component of variance 100 Vr /V by tage of ampling. iv) The average deign effect Fr, defined by r _ /( W - 1) Ppq, and with ome further =1 - approximation may be hown to equal Ppq. v) The average meaure of homogeneity, -l/ referring to the meaure of imilarity in characteritic for any pair of houehold within rth tage unit of a tratum a compared with any pair of houehold within the tratum, averaged over all trata acro Canada. The item above are tabulated for Canada urban and Canada rural a of June, i the average population variance ber:r-1 tween rth tage unit within (r-1)th tage unit (not hown in thee table) averaged over all (r-1)th tage unit. All average mentioned a- bove are etimate baed on the ample. In a particular (r -1)th tage unit it 2 2 N a = E P. r -1 r -r-1-1 r I (X. /P. -r -1 -r 1r r -1 - X. )2; where N i the -r -1-1 number of rth tage unit in it i Pi r -r-1 the relative ize of rth tage Unit ir, and Xi i the characteritic value of unit i i -1 r ri-r-1 in i uch that E X. = X. -r-1 i i i i r -r -r r In the above, = h (tratum); i1 = (PSU), i2 = K (egment in type of area j); i3 = c (cluter) and i4 c) Analyi of Table 2 houehold. The percentage contribution to the total variance, deign effect by tage, and meaure of homogeneity all how a ditinct pattern by the three categorie of characteritic in both urban and rural area. Turning firt to the percentage contribution, we find that the total variance for category I characteritic i almot entirely contributed by the between houehold component (80% in the urban area, and 59% in the rural area). For category II characteritic, the between houehold component of variance drop to 68% in the urban area and 47% in the rural area with light change to moderate increae in the other component. For category III characteritic, the between houehold component drop to 44% and 30% in the urban and rural area repectively, but the between PSU component increae to about 1/3 in both type of area. The high between houehold component of variance i reflected in low deign effect and low meaure of homogeneity at all tage. However, looking at individual characteritic for the moment rather than the average, one notice that high component of variance at a given tage are reflected in high deign effect and high meaure of homogeneity. Negative component of variance etimate and negative deign effect may be interpreted a etimate of unknown value that are poitive but cloe to zero. If meaure of homogeneity are calculated, deign effect may be conidered redundant. However, meaure of homogeneity are difficult to calculate and apply in the formulae for variance function and population variance mut be calculated individually (not averaged over characteritic), ince they are a function of the ize of 674

3 the characteritic total or mean. Deign effect are ueful both for individual characteritic and for et of characteritic to arrive at etimated population variance in the component of variance function. 5. VARIANCE COMPONENTS AND DESIGN EFFECTS IN ALLOCATION STUDIES For a given deign (i,e. trata, unit with their ize unchanged), the population variance remain unchanged and the variance function ( #7 Appendix) will be a function of the average weight for ampling at the variou tage only. Or, algebraically, the rth tage component of variance i given by: r -1 r -1 - Vr = [ W (Wr - 1)/ W (Wr - 1) Vr =1 =1 where " denote the preent value of the parameter. For example, and preent ampling fraction of PSU V1 = preent between PSU component of variance. For a 4 -tage ample deign, V = E Vr, and by r =1 ubtituting the currant percentage variance component for different weight, one can readily ee whether the altered weight would increae or decreae the variance according a V i greater or le than 100. Table 3 provide the variance that would occur under different ample allocation of weight change while maintaining the ame overall weight (product of the 4 weight kept the ame). Fifteen different value of 4 4 are obtained for 100 E V / E V r =1 r r =1 r each of the three categorie uing different component weight uch =1 that 4 4 H, = W The weight are varied motly by halving and doubling the number of elected unit per unit of the next lower tage for two of four tage (indicated by.5* and.2*). One pecial cae of a cenu of primary ampling unit with a more cattered ample within i alo conidered. In Table 3, the variance a compared with the preent are obtained for Canada Urban and Canada Rural. The total cot of the urvey will not be the ame for all of the allocation preented in Table 6, ince the houehold may be on an average more pread out or more clutered than at preent, but it erve to illutrate one method of optimization on the aumption that the total urvey cot may not differ appreciably if the total ample ize i kept contant, whatever the allocation by tage. Under the urvey contraint of a fixed overall ample ize and fixed trata and delineated ampling unit, it turn out that in only one of the 15 cae examined in Table 3 will the total variance be reduced for all 3 categorie of characteritic, although triking reduction may occur in any one of the categorie. If every PSU i taken (i.e. each PSU a tratum), then the variance of category III variable would be ubtantially reduced, but 1/2 of the egment and 1/2 of the cluter and the appropriate proportion of houehold within cluter to maintain the ame overall ampling ratio (ee note at bottom of Table 3) would be required. There i a relatively mall choice of allocation to examine. Wr cannot exceed Pr /Pr, for we would take fewer than one rth tage unit per (r-1)th tage unit. Alo Wr cannot be le than one, for we would be taking more unit in the ample than there exit in the population. It would be poible to take twice a many rural egment per PSU a we now do, but not twice a many urban egment, ince in mot PSU we already take every urban egment. For thi reaon, the cae of 2* for egment were omitted from Table 3. The change in the variance from the preent under different allocation given in Table 3 (no change in the ize or delineation of the unit at the variou tage) i given by: 4 r-1 - = E (a - 1)/ r=1 =1 S r r r-1 _ R Wr-1) -1] Vr =1 where denote the preent weight (invere ampling ratio) for ampling rth, tage unit, Wr denote the altered weight according to the allocation trategy a of Table 3 (commonly ar.5 or.2), and Vr denote the preent vari- ance between rth tage unit, expreed a a preent total variance. By ubtituting the appropriate value of a-1 in the above formula, imple relationhip between V and V may be derived for each allocation trategy and the condition between Vr ' neceary to enure a de- creae in the variance readily derived. The implified relationhip may help explain peculiar reult uch a allocation #4 and #5, where it turn out that V4 < 4.1 V3 in order to enure a decreae in V from V in the urban area. Concluion from Table 3 i) Depite the exitence of take -all cluter, the between houehold component of variance account for a ignificant portion of the total variance and in fact dominate the total variance for category I variable (e.g., 80% of the total variance in Canada urban). Conequently, for fixed total ample ize, change in the allocation will have a mall effect on the variance for category variable. ii) If we elect only one PSU intead of 2 a in allocation #6 and 7, the variance of category II and III variable would increae ubtantially, in ome cae over 675

4 50 %. While the variance i ubtantially increaed, the field cot might be only marginally reduced ince the workload in each PSU i doubled and we might require two interviewer a we would in the cae of 2 elected PSU. If, however, the interviewer' workload could be doubled without jeopardizing the quality of the interview, then there would be ome reduction in the cot with fewer interviewer to train and lower travel cot. iii) If we elect 3 or 4 PSU a in the cae of allocation 8 to 12 and 13 to 15 repectively, there are only mall decreae or increae for Category II Urban but a greater tendency for reduction for Category II Rural. For Category III Urban, the tendency lean toward reduced variance up to 20% (exception being allocation 11 and 12) but for Category III Rural, the reduction vary from a negligible 3% in the cae of allocation 11 and 12 to about 30 %. The interview cot would likely increae ubtantially either through increaed travel between more PSU which would reult in larger area to cover or through an increae in the number of interviewer in maller area and maller workload for each, thu increaing the training and interviewer control cot. iv) If, in the extreme cae, we elect every PSU in each tratum, the reduction in the variance for Category II Urban and Category III variable are large (e.g., 45% reduction for Category III in the rural area). The effect on Category I variable and Category II in the urban area i minimal (le than 8 %). If we had delineated trata the ize of PSU, there would perhap have been a larger reduction than indicated in thi paper. For the philoophy behind the delineated trata i to make them a ditinct a poible while PSU hould be a much alike a poible. v) Depite the rather evere contraint of a fixed overall ample ize with varying weight by tage but with delineated trata and unit unchanged, many intereting and important reult were revealed. The data of Table 2 and 4, and the variance function in term of weight could be further utilized to determine the variance for the requirement of reducing or increaing the total ample ize, examining variou trategie of reduction or increae of the number of elected unit at the different tage of election. 6. VARIANCES OF CHARACTERISTICS AND SIZES OF UNITS Jut a in Section 5, we examined the change in the variance a the allocation of the ample by tage (maintaining or fixed overall ample ize and the ame delineated unit and trata), we can examine the change in the variance a the average ize of the unit are altered (maintaining the ame trata and ampling rate by tage). In a imilar manner employed in Section 5, the average ize of the unit are halved or doubled in uch a manner that a lower tage unit would never be maller than a higher tage unit. Uing #2 in the methodology, V wa obtained for different average ize of for r = 1, 2, and 3 while Po and P4 are fixed. The reult are preented in Table 4. Analyi of Table 4 It can be een that only marginal improvement are oberved for any trategy of altered ize of unit and they are largely confined to the cae of changed egment and cluter ize. When the PSU ize i doubled, however, the rural etimate are hown to poe an increae in variance up to almot 20% for Category I variable and almot 40% for Category III variable. In the cae of Category III variable with high meaure of homogeneity for mot ize of unit, any increae in the average ize of unit will tend to increae rather than decreae the variance even while maintaining the fixed overall ample ize. From the oberved large between PSU variance of rural Category III variable, one would expect ome ubtantial reduction in the variance of thee variable if maller PSU were delineated and twice a many of them were elected. The interview cot for maller PSU, however, may be greater on a per houehold bai becaue of the extra pread of the ample or the neceity to hire extra interviewer to avoid the travelling between PSU. General Concluion The analyi of the component of variance pertained trictly to the current LFS deign. However, the method can be eaily adopted to any multi -tage ample deign imilar to the above, uch a the revied LFS deign. Only one urvey' data wa ued in thi article and it appear that the allocation of the ample by tage and the ize of delineated unit are near optimum value for the given ample ize. However, the cot function for change in the weight and /or ize of unit wa not taken into account o that one cannot draw fully certain concluion about the optimum propertie of the ample deign. To monitor the ample deign for it efficiency in term of allocation and ize of unit continuouly in a continuing urvey where growth and feature are contantly changing it i recommended that component of variance be obtained four time per year acro Canada with pecial run for maller area where problem are occurring. REFERENCES [1] Fellegi, I.P., "Sampling With Varying Probabilitie Without Replacement: Rotating and Non -Rotating Sample ", Journal of the American Statitical Aociation, 58 (1963), [2] Fellegi, I.P., Gray, G.B., and Platek, R., "The New Deign of the Canadian Labour Force Survey ", Journal of the American Statitical Aociation, 62 (1967), [3] Gray, G.B., "Variance Component and Variance 676

5 Function ", Proceeding of the Canadian Conference in Applied Statitic, Statitic '71 Canada, 1971, [4] Gray, G.B., "Component of Variance in Multi - Stage Stratified Sample ", Houehold Survey Development Staff Survey Methodology, Vol. 1, No. 1, (June 1975), [5] Hanen, M.H., Hurwitz, W.N., and Madow, W.G., "Sample Survey Method and Theory ", Vol. 1 and 2, New York: John Wiley & Son (1953). [6] Keyfitz, N., "Etimate of Sampling Variance Where Two Unit Are Selected From Each Stratum", Journal of the American Statitical Aociation, 52 (1957), [7] Kih, L., "Survey Sampling ", New York: John Wiley & Son (1965), [8] Sampling and Survey Reearch Staff, Dominion Bureau of Statitic (now Houehold Survey Development Staff, Statitic Canada): Methodology, Canadian Labour Force Survey ", (1965). [9] Yate, F. and Grundy, P.M., "Selection Without Replacement From Within Strata With Probability Proportional To Size ", Journal of the Royal Statitical Society, Serie B, 15 (1953), Methodology For The Study A mentioned earlier, the variance component were etimated by the Yate - Grundy formula [9], alo quoted in Fellegi [1] and the approximate variance function in term of weight and population variance, developed in more detail by Gray [3] and [4] i given by: V (P Wr 1)ar:r -1/ (1 - r -1 /Pr) = rth tage component of variance, where p = total population pr = average ize of rth tage unit W = average weight for th tage unit within ( -1)th tage unit wa defined in the text. If the trata and ampling unit at all tage remain fixed a in Table 3, then r -1 V a II W '(W -1). Although the variance etir =1 r mate were obtained uing the appropriate individual and joint election probabilitie, the variance function wa developed on the aumption that the average f.p.c. rfp = -1/(Nr:r -1-1), where i the average number of Nr:r -1 (1) Table 1: Average Deign Effect by Category (no. of characteritic in bracket *) for Canada Urban and Canada Rural Category Canada Urban Canada Rural (2).989 (2) (9) (10) (8) (7) * with only one exception, the category of the characteritic wa the ame in urban and rural area Table 2: rth tage unit per (r -1)th tage unit. To tudy the change in the variance a the ize of the unit are changed, but maintaining fixed houehold and trata ize, a in Table 4, the population variance function wa derived below {[1+(N -1) ] r:r -1 4:r 4:o 4:r 4:r - 1[1-1- 1)754r -10 (2) where N :r = average no. of th tage unit per rth tage unit. Etimate of :r were obtained from June, 1973 data and interpolated by the reciprocal of the average ize of unit a different ize of unit at variou tage were conidered. Average weight (wt.), ize of unit; percent component of variance %, deign effect F, and meaure of homogeneity by tage of ampling for Canada by type of area and by Category of variable (June, 1973). Type of Area, Stage of Sampling Aver. Wt. Aver. Size Urban, , , Rural, , , % Category I Category II F Category III % F

6 Table 3: Variance/Preent Variance uing different allocation of weight while maintaining current overall weight Sample Allocation Category I Category II Category III Alloc. PSU/ Seg/ Clu/ Hhld/ No. Str PSU Seg Clu Urban Rural Urban Rural Urban Rural 1) *(2) * * * ) *.5* 2* * ) *.5* * 2* ) * *.5* 2* ) * * 2*.5* ).5 *(1) * 2* * ).5* * * 2* ) 1.5 *(3) 2/3* * * ) 1.5* * 2/3* * ) 1.5* * * 2/3* ) 1.5*.5* 4/3* * ) 1.5*.5* * 4/3* ) 2. *(4).5* * * ) 2.* *.5* * ) 2.* * *.5* ) all.5*.5* (a) Number of PSU elected per tratum in bracket (a): W4 = 6.78(U); 3.69(R) *: Mean current ampling ratio of elected unit within unit at the next lower tage on an average. all: Mean that there i no ampling of unit at a particular tage, i.e. all unit are elected. Table 4: Variance /Preent Variance for different ize of unit by tage of ampling (trata unchanged) and the ame weight applied at each tage a currently Size of Unit Strategy Compared to Current Category I Category II Category III Number Seg Urban Rural Urban Rural Urban Rural 1 1.*(2) 1.* 1.* *.5* 1.* * 1.*.5* * 2.* 1.* * 1.* 2.* *.5*.5* *.5* 2.* * 2.*.5* * 2.* 2.* *(1).5* 1.* * 1.*.5* * 2.* l.* * 1.* 2.* *.5*.5* *.5* 2.* * 2.*.5* * 2.* 2.* A * mean A time a large a current average ize. No. of elected PSU per tratum in bracket 678