TOPIC 7 ANALYSING WEIGHTED DATA

Size: px

Start display at page:

Download "TOPIC 7 ANALYSING WEIGHTED DATA"

Melvin Davidson
5 years ago
Views:

1 TOPIC 7 ANALYSING WEIGHTED DATA You do t have to eat the whole ox to kow that the meat s tough. Samuel Johso Itroducto dfferet aalyss for sample data Up utl ow, all of the aalyss techques have oly dealt wth producg the mea ad varace from the raw data. But what do we do f we have weghted data? Ths chapter wll dscuss frstly what a weght s ad the look at how we ca calculate summary statstcs from weghted data. Fally, we wll look at the Stadard Error ad dscuss how t dffers from the Stadard Devato. What are weghts? the sample represets the populato We have already see Topc 1 that ofte t s better to take a sample of the populato rather tha gog to the etre populato. Remember that a sample s a small umber of statstcal uts selected to represet the etre populato. After we have completed the survey, we are ofte requred to make fereces about the populato usg the sample. For example, let us say that we have just completed a sample survey askg about how may shells of Kava you drak last week. We could easly apply the techques the frst 6 topcs of ths book to produce the mea, meda ad varace of ths sample. However, how could we estmate the total amout of moey spet o Kava Vauatu or the total umber of people that drak Kava? estmates for the populato use weghts To produce estmates about the etre populato we allocate each perso the sample a weght. A weght dcates how may people the populato ths perso represets. For example, f our Kava Survey wet to 1,500 people ad there are 150,000 people the populato the each perso the survey would receve a weght of 150,000 / 1,500 =100. So each perso the sample would be represetg 100 people the populato. weghts deped o sample selecto I the Kava Survey, each perso receved the same weght, however ths s ot always the case. The weght of each statstcal ut wll deped o the way that the uts were selected ad how may uts dd ot respod. However, the followg thgs should always be true: Data Aalyss Course Topc 7-145

2 3 rules for weghts 1 f you add up all of the weghts they wll equal the umber of uts the populato; 2 the weght of a ut dcates how may uts the populato t s represetg; ad 3 weghts should always be larger tha 1. If you are a survey the you must be represetg at least oe perso...yourself! weghts do ot have to be whole umbers Although t souds strage, weghts do ot eed to be a whole umber. That s, you ca have a weght of 4.53 or Although t may seem strage that a perso ca be represetg people the populato, t s just our best guess of how may people wth smlar characterstcs are the populato. Usg weghts aalyss sample values estmate the populato Oe mportat dfferece whe we are aalysg weghted data s that we do t kow the exact value for the populato. For example, whe we are workg out the mea value for a cesus, we kow every value the populato, so the result wll be exactly rght. However, whe we take a sample we are ot gog to everyoe the populato, just a few people or statstcal uts who we thk represet the populato. Sometmes the sample selected wll provde a good or accurate represetato of the populato, ad at other tmes t mght ot be so good. It s mportat to select the sample such a way as to best represet the populato. TIP Whe we calculate the mea, meda ad varace usg weghted data, we say that we are estmatg the populato value, that s, we are estmatg the populato mea, meda or varace. at best a estmate So f we are workg o cesus data, the we ca say that we are calculatg the mea, but f we are usg a sample to estmate the mea of a populato the we are estmatg the mea. sample error s the measure of the sample s represetato of the populato These sample estmates of the populato values are subject to samplg error. Ths s error because we have chose a sample, ad ot the whole populato. If we have chose the sample correctly, our estmates of the populato values wll be very close to, or eve the same as, the populato values. I these stuatos the samplg error s very small. Remember that populato estmates are also affected by o-samplg errors whch are very dffcult to calcuate. assume here that the weghts have bee defed for you Whe you are workg wth weghted data, the weghts wll be provded to you wth the data. Weghts are determed by a umber of factors, such as how the sample was selected, respose rates ad other crtera usually specfed by a mathematcal statstca. Do ot worry about how to calculate the Topc Secretarat of the Pacfc Commuty

3 weghts, t s ofte complex ad out of the scope of ths course. Estmatg the mea formula For weghted data the formulae for estmatg the mea s: Estmate of the populato mea from the sample = w x 1 w 1 where w s the weght for each ut x I s the value for the varable example For example, f we had the followg results from a survey: Table 7.1 Estmatg the populato mea from sample data Value (x ) Weght (w ) Weght Value (w x ) Total Source: Illustratve data oly. mea estmate The our estmate of the populato mea would be: = w x = 60 w 1 = 5.25 Estmatg the varace of the populato estmate varace For weghted data the formula for estmatg the varace of the populato looks complcated, because the weghts are tegrated to the calculato: Data Aalyss Course Topc 7-147

4 Varace estmate = w x 1 2 ( w x ) 1 ( w ) ( w ) example We ca use the same sample survey data as before to estmate the varace for the populato. Of course, t s much easer to make the estmates f you use a computer to perform the calculatos usg formulas Table 7.2 Estmatg the populato varace from sample data Value (x ) Weght (w ) Weght Value (w x ) Weght Value squared (w x 2 ) Total ,865 Source: Illustratve data oly. varace estmate The our estmate of the populato varace would be: Varace estmate = 1 w x 2 ( 1 ( ( w ) 1 w x ) 1 1 w ) 2 = (315) 1, = = 3.58 Creatg frequecy dstrbutos sum the weghts We ca also create frequecy dstrbutos of the populato usg weghts. It s very smlar to creatg the frequecy dstrbutos that we created Topc 3. The ma dfferece s that our frequecy s ow the sum of the weghts the class terval. example For example, let us say that we have the followg data: Table 7.3 Creatg frequecy dstrbutos from sample data Varable (x ) Weght (w ) Topc Secretarat of the Pacfc Commuty

5 Populato Frequecy Aalysg weghted data Source: Illustratve data oly. class tervals Let us use the followg classes: 2 3, 4 5, 6 7, 8 9 frequecy The our frequeces for our populato would be: Table 7.4 Creatg frequecy dstrbutos from sample data Class Weghts Class (w ) Frequecy (w ) Source: Illustratve data oly. Charts Oce you have the frequecy dstrbuto, you chart the data the same way you chart uweghted data, usg the same gudeles for quattatve ad qualtatve data Populato dstrbuto for weghted data Class 8 9 Populato proportos estmate populato proportos Ofte we wat to kow the umber of people the populato wth a certa characterstc. As for may of the measures demostrated ths chapter, you use the sum of the weghts rather tha the frequecy of the value or varable. Data Aalyss Course Topc 7-149

6 For example, we may wat to estmate the umber of people the populato who drk Kava. The followg formula wll estmate the populato proporto from the sample: formula Populato proporto estmate (p) = w (wth characterstc) w (all uts) So you take the sum of the weghts of the varables wth the characterstc ad dvde t by the sum of all the weghts the sample. example So let us say that we had the followg sample of people, ad we asked them f they drak Kava last ght: Table 7.5 Sample of people ad f they drak Kava last ght Perso Weght Drak Kava Joh 10 Yes Smo 9 Yes Howard 7 No Jack 8 Yes Theto 10 No Jea Mark 9 No Meryle 4 Yes Source: Illustratve data oly. workg Our estmate of the proporto of people the populato who drak Kava last ght would be: Populato proporto estmate of kava drkers (p) = w (wth characterstc) w (all uts) = ( ) ( ) 31 = 57 = result So we ca say that we estmate that 54.38% of people the populato drak kava last ght. Stadard errors sample data estmates the populato Wheever a sample survey s coducted, a addtoal dmeso of error s troduced due to samplg oly a subset of the populato. The theory relatg to ths type of error s well developed ad the ablty to calculate the error troduced by ths collecto methodology s oe of the attractve features of sample surveys. The usual quatty calculated to measure the accuracy of a sample estmate s the stadard error of the estmate. The stadard error of a estmate eables us to make certa probablty statemets about the estmate, f certa codtos are met (these codtos are ot costrctve as log as the sample Topc Secretarat of the Pacfc Commuty

7 szes are ot very small). We ca say that we are 95% certa that the true value of what we are tryg to measure wll be wth two stadard errors of our sample estmate. how accurate s the sample data? The Stadard Error should ot be cofused wth the Stadard Devato. Stadard Error: measure of how accurate a estmate s from a sample survey. Stadard Devato: measure of the spread of the values the populato. The Stadard Devato s a compoet of the Stadard Error, such that the greater the Stadard Devato, the greater the Stadard Error. The larger our Stadard Error s, the worse the estmate s. samplg techques avalable There are may dfferet sample selecto techques avalable, ad qute ofte the preferred opto s a combato of umerous techques. Some commo techques appled by survey statstcas clude: ) Smple Radom Samplg: s the most smple form of samplg, ad volves assgg a radom umber to the uts the populato of terest ad usg ths radom umber to select the sample. Usually results a well spread out sample. ) ) v) Systematc Samplg: s also a smple form of samplg ad volves lstg the uts the populato of terest (ofte ordered by a partcular varable/s), ad the rug a skp through the lst to select the sample. Also results a well spread out sample. Stratfed Samplg: volves splttg the uts the populato of terest to sub-populatos (strata), ad selectg separate samples wth each stratum. Has the beeft of beg able to cotrol sample szes for sub-populatos. Probablty Proportoal to Sze (PPS) Samplg: for ths techque, the sze of a ut determes the lkelhood of that ut beg selected. That s, f vllages are beg selected for the survey, ad the sze measure beg adopted s the umber of households each vllage, the a vllage wth twce as may households as aother vllage, wll have twce as much chace of beg selected. v) Cluster Samplg: volves selectg clusters of uts stead of uts spread out radomly. Has the beeft of cost savg due to the reduced travel cost. Ufortuately, as the sample s o loger well spread, the stadard errors wll crease. v) Mult Stage Samplg: volves selectg the sample more tha oe stage. For example, rather tha go to every vllage, smply select a sample of vllages, ad from those selected vllages, select a sample of households. Oce aga ths techque has cost savgs. As metoed, t s rare for just oe of these techques to be appled to a sample survey. Ofte 2, 3 or eve 4 of these techques are adopted for the oe survey. For example, for a HIES, you may frst splt the populato of terest to two strata (urba ad rural). From there you may wsh to lst all the vllages each of the stratum ad apply PPS samplg to select the vllages. For each selected vllage, t may the be desrable to ru a skp through the vllage order to select 10 households. Ths approach uses Stratfed Samplg, PPS Samplg, Systematc Samplg, ad Two-Stage Samplg. formula The formula used to calculate the Stadard Error for a sample survey s depedet o the sample selecto methodology appled. Whe a combato of samplg techques s appled, the the Data Aalyss Course Topc 7-151

8 formula ca become extremely complex. Ofte approxmato techques (eg, Jack-kfe varace estmato) are adopted to overcome ths problem. To gve a example of what the formulae look lke however, to calculate the Stadard Error of the estmated populato mea, populato total ad populato proporto from a smple radom sample we use the followg formula: s SE (estmated populato mea) = {(1 ( )) * } N 2 2 SE (estmated populato total) = N *{(1 ( )) * } N 2 s ( p(1 p)) SE (estmated populato proporto) = {(1 ( )) * } N Where s 2 s the estmate of the populato varace N p s the sample sze s the umber of uts the populato s the estmate of the proporto the populato kava example So the Kava example above, our proporto was , our sample s sze 7 ad our populato s 57, so the Stadard Error s: SE (estmate of the proporto the populato) 7 (0.5438( )) = {(1 ( )) * } = So we say that the stadard error of the sample proporto s terpretg stadard errors The frst thg we eed to ask ourselves whe we produce a stadard error s what does t mea? If you take 2 stadard errors ether sde of a estmate, the you are 95% cofdet that the true value wll le wth ths rage. That s, for the kava example above, we ca say that we are 95% cofdet that the true value for the proporto of kava drkers s betwee: (2*0.1763) = (0.1912, ) relatve stadard error (RSE) Aother way of showg the accuracy of a estmate s to compute the relatve stadard error (RSE), whch s smply the stadard error as a percetage of the estmate. For the example above, the relatve stadard error for the estmate would be calculated as follows: RSE / * % It dffers from perso to perso as to what costtutes a good estmate terms of RSE, but most statstcas would agree that ay estmate wth a RSE below 5% s a good oe. Whe rug a Topc Secretarat of the Pacfc Commuty

9 sample survey, t s desrable to produce key estmates from the survey wth RSEs of 5% or below. For other estmates of sgfcace, the RSEs should ot exceed 20%. It would therefore be far to coclude that the estmated proporto of Kava drkers above s ot a very good oe. Ths s largely due to the sample sze of 7 beg too small. If the sample sze was creased to 30, ad a estmate of was stll geerated, the a RSE of 11.5% would have bee acheved. SE (estmate of the proporto the populato) 30 (0.5438( )) = {(1 ( )) * } = RSE / * % zero error a Cesus Oe addtoal ote s that f we ru a cesus the the sample sze equals the populato sze ( = N). So, the estmate of the populato value should be exactly correct because we have every oe the populato the sample. We ca also see that the formula, that whe = N the: So there s o error our estmate. N ( 1 ) (1- ) 11 N N o-samplg errors Remember that samplg error s oly oe compoet of the total survey error. There are may ways that error ca be troduced to surveys other tha the use of samplg methods. Feld eumerato error, respodet error, questoare desg flaws ad processg errors ca all troduce errors to the overall survey (ote that these errors are depedet of the samplg process ad would have occurred eve f a cesus had bee coducted). (NB: Ths compoet of the total survey error, ca ofte be sgfcatly hgher tha the samplg error, so every care should be take to udertake correct survey procedures, to mmse ths mpact.) 0 TIP The mportat thg s to mmse these o-samplg errors ad esure that errors that troduce a systematc bas to the survey results are avoded. Data Aalyss Course Topc 7-153

10 Topc Secretarat of the Pacfc Commuty

11 ExercseS Usg the formato below, calculate the followg: (a) Stadard error for the mea aual household come (b) Stadard error for the total aual household expedture (c) Stadard error for the proporto of households wth aual household come > $10,000 (d) The equvalet RSE for each of these 3 estmates (b: assume that the sample was selected usg Smple Radom Samplg) Number of households the populato (N): 123,653 Number of households the sample (): 3,425 Estmated average household come = $8,932 Estmated total household expedture = $970,552,397 Sample varace (S 2 ) for aual household come: 645,657,234 Sample varace (S 2 ) for aual household expedture: 553,265,867 Proporto of households the sample wth aual household come > $10,000: 0.32 Data Aalyss Course Topc 7-155

12 Topc Secretarat of the Pacfc Commuty

13 Excel stadard errors Whe calculatg stadard errors you have to take to accout the sample desg used. The worksheet method here s applcable for a smple radom sample (that s, that the sample households were radomly selected from a lst of households). If your sample was selected by clusters, wth blocks of households selected ad all households wth these blocks beg selected, ths clusters the sample selecto ad creases the sample error of the results. It s more complcated to calculate the stadard error from a cluster sample ad, ufortuately, the effect of such clusterg s ot well kow. Eve developed coutres such as Australa the effects of clusters are ot kow for certa surveys (lke household come ad expedture). However, t s suspected that the effect o the stadard error s a crease betwee 10% ad 30%. If you are calculatg stadard errors from a cluster sample cotact the Statstcs Programme at the SPC for techcal assstace. Remember that samplg error s oly oe compoet of the total survey error. There are may ways that error ca be troduced to surveys other tha the use of samplg methods. Feld eumerato error, respodet error, questoare desg flaws ad processg errors ca all troduce errors to the overall survey (ote that these errors are depedet of the samplg process ad would have occurred eve f a cesus had bee coducted). The mportat thg s to mmse these o-samplg errors ad esure that errors that troduce a systematc bas to the survey results are avoded. I the aalyss of the results there was o evdece of such systematc bas beg troduced to the survey results, but ths does ot mea that ths dd ot occur, ad was hdde from the aalyss phase of the project. Settg up a worksheet to calculate stadard errors Bascally you set up a worksheet whch calculates the dfferet parts of the Stadard Error formula so from your data you fd the values for x, x, x, x, etc. It ca be complcated to set up the worksheet, so for 2 2 ay advce cotact the Statstcs Programme at the SPC. As well as the varable you are calculatg the stadard error estmates for, you wll also eed the sample weghts (to make the estmate of N). WARNING! The example data used here s lmted to oe varable wth 20 records ad equal weghts. You would work through the same steps f you had 200 varables ad 60,000 records wth the same weght. If, for example, you have dfferet weghts for dfferet regos, you would calculate the SE for each rego SEPARATELY. Data Aalyss Course Topc 7-157

14 1. The frst thg s to square the observatos. I ths example, the varable s called Veges, ad s expedture o frut ad vegetables over a two-week perod. The formula to square a value s: =cell referece ^ 2 (eter). If you were calculatg the stadard error for more tha oe varable, you could use aother worksheet the same workbook to square the values, but here t s besde the test data. 2. You copy the formula dow the rest of the page by clckg ad draggg o the fll hadle at the bottom rght corer of the cell wth the formula t. Clck ad drag o the fll hadle to copy the formula dow the colum: Clck ad drag the fll hadle Topc Secretarat of the Pacfc Commuty

15 3. Calculate x 2 : type a label for the calculato to sum the squared values a cell below your data. The formula to sum values s: =sum(cell ref) (eter). Here the cell referece s C2:C21. Your scree should look lke ths: 4. Calculate x : type a label for the calculato to sum the values a cell below your data. The formula to sum values s: =sum(cell ref) (eter). Here the cell referece s A2:A21. Your scree should look lke ths: Data Aalyss Course Topc 7-159

5. Fd ad N (the estmated populato sze): fd by coutg the umber of observatos the sample usg the formula = cout (A2:A21). Fd N by summg the weghts usg the formula = sum(b2:b21).

16 5. Fd ad N (the estmated populato sze): fd by coutg the umber of observatos the sample usg the formula = cout (A2:A21). Fd N by summg the weghts usg the formula = sum(b2:b21). Your worksheet should ow look lke ths: 6. Calculate the sample mea x : fd the arthmetc mea of the sample data usg the Excel formula: =C24/C25. You dvde the sum of the observatos by the cout of the observatos to gve you x - the mea of the sample. Your worksheet should ow look lke ths: Topc Secretarat of the Pacfc Commuty

Calculate the Fte Populato Correcto (FPC) factor: the varace calculato s multpled by ths factor to compesate for the sample data.

17 7. Calculate the estmate of the total for the populato: fd the estmated populato total by multplyg the sample mea by the sum of the weghts. Eter the Excel formula = (C26*C27). Your worksheet should ow look lke ths: 8. Calculate the Fte Populato Correcto (FPC) factor: the varace calculato s multpled by ths factor to compesate for the sample data. You do ot have to multply by the FPC. The formula for the FPC = N /N. Eter the Excel formula = (C26 C25)/C26. Your worksheet should ow look lke ths: Data Aalyss Course Topc 7-161

Your worksheet should ow look lke ths: 10.

18 9. Calculate the sample varace: you ow have all the values requred to calculate the sample varace usg the formula 2 ( x ) ( x ) FPC * 1 2. Eter the formula Excel =C29*(C23-(C24^2/C25))/(C25-1). Your worksheet should ow look lke ths: 10. Calculate the varace of the sample mea: ths s the result of the dvdg the sample varace by the sample sze. Eter the formula Excel = C30/C25. Your worksheet should ow look lke ths: Topc Secretarat of the Pacfc Commuty

Your worksheet should ow look lke ths: 12.

19 11. Calculate the stadard error (SE) of the sample mea: the stadard error s the square root of the sample varace. Eter the formula Excel =C31^0.5. Ths s the same as typg =sqrt(c31). Your worksheet should ow look lke ths: 12. Calculate the varace of the estmated populato total: the sample estmate multpled by the estmated populato squared (N 2 ). Eter the formula Excel =C31*(C26 ^2). Data Aalyss Course Topc 7-163

Calculate the relatve stadard error (RSE) of the sample mea: the Relatve Stadard Error Percet s commoly used as a measure of relablty whch ca be compared

20 13. Calculate the stadard error (SE) of the estmated populato total: the stadard error s the square root of the estmated populato varace. Eter the formula Excel = C33 ^ 0.5. Ths s the same as typg =sqrt(c33). Your worksheet should ow look lke ths: 14. Calculate the relatve stadard error (RSE) of the sample mea: the Relatve Stadard Error Percet s commoly used as a measure of relablty whch ca be compared across estmates ad across surveys. It s the Stadard Error of the mea dvded by the sample mea * 100. Eter the formula Excel = C32 * 100 /C27. Your worksheet should ow look lke ths: Topc Secretarat of the Pacfc Commuty

15. Calculate the relatve stadard error (RSE) of the estmated populato total: you calculate ths as a check. It equals the SE of the populato total dvded by the estmated total *100.

21 15. Calculate the relatve stadard error (RSE) of the estmated populato total: you calculate ths as a check. It equals the SE of the populato total dvded by the estmated total *100. The RSE X_bar should always equal the RSE X_Total. If the two are NOT the same there s a error somewhere your formula ad you have to go back ad fd t. Eter the formula Excel =C34*100/C28. Your worksheet should ow look lke ths: You have ow calculated the Stadard Errors ad the Relatve Stadard Errors for your sample data. WARNING! If RSE X_bar DOES NOT EQUAL RSE X_Total there s a error a formula ad you have to go back ad fx t. 16. To calculate results whe you have dfferet weghts: ofte the data wll have dfferet weghts. These weghts may correspod to dfferet regos or dfferet parts of the populato. These dfferet compoets are ofte called strata (or stratum). The overall total ca be foud by multplyg the sum of the x s (Sum X) by the weght for that stratum (the weght equals N/) ad the addg these compoets for all the dfferet strata. The overall mea s calculated by dvdg the overall total by the populato total (N). To work out the overall varace for the total for ths type of data t s ecessary to work out the dvdual varaces for each stratum. The dvdual varace of each total (Var X_tot) the have to be added together to get the overall varace. The square root of ths umber the gves the overall stadard error (SE X_tot). The varace of the overall X_bar ca be foud by dvdg Var X_tot by the populato total squared (N*N). Data Aalyss Course Topc 7-165

22 Topc Secretarat of the Pacfc Commuty

Types of Sampling Plans. Types of Sampling Plans. Sampling Procedures. Probability Samples -Simple Random sample -Stratified sample -Cluster sample

Types of Sampling Plans. Types of Sampling Plans. Sampling Procedures. Probability Samples -Simple Random sample -Stratified sample -Cluster sample Samplg Procedures Defe the Populato Idetfy the Samplg Frame Select a Samplg Procedure Determe the Sample Sze Select the Sample Elemets Collect the Data Types of Samplg Plas o-probablty Samples -Coveece