Design of a Multi-Stage Stratified Sample for Poverty and Welfare Monitoring with Multiple Objectives

Transcription

1 Policy Research Working Paper 7989 WPS7989 Design of a Multi-Stage Stratified Sample for Poverty and Welfare Monitoring with Multiple Objectives A Bangladesh Case Study Faizuddin Ahmed Dipankar Roy Monica Yanez-Pagans Nobuo Yoshida Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Poverty and Equity Global Practice Group March 2017

2 Policy Research Working Paper 7989 Abstract This paper describes the design of a multi-stage stratified sample for the Bangladesh Household Income and Expenditure Survey 2016/17. This survey instrument will be used by the Government of Bangladesh to estimate reliable poverty and welfare statistics at three different levels: (i) annual estimates at the district level, (ii) quarterly estimates at the national level, and (iii) annual estimates at the division level for urban and rural areas. The sample for this survey was designed to achieve these three objectives. The paper explains how the three objectives are prioritized and how inconsistencies in achieving more than one objective can be reconciled. Further, the paper modifies the standard formulas to estimate the optimal sample size and the allocation of the sample across strata by explicitly taking into consideration the effect of clustering in the sample. This paper is a product of the Poverty and Equity Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at The authors may be contacted at nyoshida@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team

3 Design of a Multi-Stage Stratified Sample for Poverty and Welfare Monitoring with Multiple Objectives: A Bangladesh Case Study Faizuddin Ahmed Dipankar Roy Monica Yanez-Pagans Nobuo Yoshida JEL Classification: C81, C83, I32 Keywords: stratified clustered sampling, household surveys, poverty measurement *Corresponding author: Nobuo Yoshida (nyoshida@worldbank.org), The World Bank. Faizuddin Ahmed (fahmed3@worldbank.org) and Monica Yanez-Pagans (myanezpagans@worldbank.org) are also at The World Bank and Dipankar Roy (dipankar.roy@bbs.gov.bd) is at the Bangladesh Bureau of Statistics. The authors are very grateful to Juan Munoz and Kristen Himelein for their advice and suggestions during the preparation of this paper. All errors and mistakes remaining in this paper are solely ours.

4 1. Introduction In recent decades, the need for detailed demographic and socioeconomic data to measure living standards in developing countries has increased dramatically. Data used for measuring living standards are traditionally collected through household income and expenditure surveys or through integrated household surveys. The data these surveys generate are critical for monitoring welfare and informing the design of policies to reduce poverty and promote equitable distribution of economic growth. In most developing countries, integrated household surveys are a central component of national statistical systems, which provide key inputs in social and development policy making and national accounts. Collecting expenditure or income data for welfare monitoring is expensive and time consuming. Generally, households are visited several times and need to report on their food and non-food expenditures for often more than 50 items over the past days or weeks either using a recall or a diary method. The complexity of the data collection can cause households to make mistakes when reporting their incomes or consumption or enumerators to make mistakes when recording their responses. As a result, collecting consumption or income data from an entire population in a country to monitor trends of key welfare statistics such as poverty headcount rates or mean income is very time-consuming and resource intensive to carry out. To overcome these challenges, National Statistical Offices (NSOs) conduct household income and expenditure surveys for only a sample of the population. The selection of a sample, which is called sampling, is an important subject in statistics and needs to be done carefully so that the select sample of individuals or households represent the entire population and data collected from the sample can therefore be used to estimate characteristics of the population with a reasonable level of precision. Sampling provides two clear advantages substantially reducing the cost and time needed to collect data and producing statistics that are representative of the entire population. For example, a sample of a couple of thousand households or individuals is often enough to estimate nationally representative statistics of a country with a population of one hundred 2

5 million. However, the risk of sampling is also large if the selection of households or individuals is not properly done, statistics constructed from the sample do not represent the characteristics of the population of interest. This is the reason why sampling is so critical when NSOs are designing a survey. This paper describes the sampling design for the 2016/17 Bangladesh Household Income and Expenditure Survey (HIES). The HIES, a large household survey collected by the Bangladesh Bureau of Statistics (BBS), is the main government instrument for monitoring welfare and living conditions in Bangladesh. The HIES is a well-established sample household survey in Bangladesh and has been conducted about every five years since , with the latest three rounds collected in 2000, 2005, and BBS decided to redesign the sample of the HIES 2016/17 to produce more frequent and more disaggregated poverty estimates than the previous rounds. Specifically, the samples of the latest three rounds of the HIES survey are designed to provide reliable annual poverty estimates at the division level for both urban and rural areas. 2 But in the HIES 2016/17, BBS decided to produce reliable annual district poverty statistics and quarterly national poverty statistics in addition to annual poverty estimates at the division level for urban and rural areas. As a result, BBS needed to change the design of sampling for the HIES 2016/17 significantly. For example, the sample size for the HIES 2016/17 needed to increase almost fourfold compared to the HIES This substantial increase in the sample size also forced a revision of the sampling frame to accommodate the larger number of household field interviews. The main challenge when designing the HIES 2016/17 was how to achieve multiple goals with one sampling design. As mentioned above, the objective of the HIES 2016/17 is three-fold: (i) producing reliable annual poverty and welfare statistics at the district level; (ii) producing reliable quarterly poverty and welfare statistics at the national level; and (iii) producing reliable 1 In addition to collecting detailed household consumption and income data, the HIES collects detailed information on household demographics, education, health, livelihoods and economic activities, quality of housing, asset ownership, migration, social safety-nets, micro-credit, access to services, and household shocks and coping strategies. 2 Divisions are the higher level of geographical divisions in Bangladesh. As of 2010, there were a total of seven divisions in the country, including Dhaka, Chittagong, Barisal, Khulna, Sylhet, Rangpur, and Rajshahi. 3

6 annual poverty and welfare statistics at the division level for urban and rural areas. A potential challenge is that the optimal sample for objective (i) is not necessarily identical to that needed to achieve objectives (ii) or (iii). Therefore, the design of the sample requires careful thinking on how to achieve the multiple objectives, which are not necessarily mutually consistent, within a single sample. This paper describes how the sample of the HIES 2016/17 was designed to achieve the aforementioned three objectives. This paper also reviews the formulas that are often used by NSOs and development practitioners to calculate the sample size and the optimal allocation of the sample across strata. The HIES 2016/17 follows a stratified clustered sampling. Clustered sampling is used when the sample is selected in multiple stages. That is, in a first stage, clusters of households or individuals are selected randomly and then households or individuals are selected from the clusters in a second stage. This is less efficient than simple random sampling (selecting household or individuals in one stage directly) in terms of minimizing sampling errors, but it is more cost-effective as selected households or individuals are physically located relatively close to each other in the same cluster. This paper shows that the standard formulas used by practitioners for defining the sample size and the allocation of the sample across strata assume stratified random sampling rather than stratified clustered sampling. As a result, the sample size and the allocation of the sample can be too optimistic in terms of minimizing the sampling errors. This paper is organized as follows. Section 2 reviews key literature on optimal sample allocation for stratified random samples and stratified clustered samples. Section 3 describes objectives of the survey, data used to design the HIES 2016/17 sample, sampling frame, stratification and domains of study, selection of target variables and accuracy statistics for sample performance evaluation, determination of the sample size, and allocation of Primary Sampling Units (PSUs) across strata. Section 4 summarizes evaluation of the proposed sampling design. Section 5 analyzes robustness of the proposed sample design. Section 6 describes survey sampling weights computation. Section 7 presents the main conclusions of the paper. 4

7 2. Background 2.1. HIES rounds sampling design prior to the HIES 2016/17 The HIES rounds have been central for monitoring poverty and welfare indicators in Bangladesh. Prior to the HIES 2016/17, the HIES rounds were designed to produce annual poverty estimates at the division level for urban and rural areas and Statistical Metropolitan Areas (SMAs) (see Munoz 1998 and 2004). The sample size of the HIES gradually increased over time and, as of the latest round completed in 2010, the sample size was 12,240 households. To avoid the effects of seasonality, the sample was proposed to be randomly split across quarters a full round of the HIES data collection takes 12 months but there was never an explicit attempt by BBS to produce quarterly poverty estimates before the HIES 2016/17. All of the previous HIES rounds follow a stratified clustered sampling design each of urban, rural, and SMA areas in a division forms a stratum. Primary Sampling Units (PSU) are constructed based on Census of Population and Housing Enumeration Areas and represent a cluster. PSUs are selected with probability proportional to size (PPS) and 20 households from each PSU are selected using systematic sampling. It is worth noting that PSUs for all of the previous rounds of the HIES were selected from the Integrated Multiple-Purpose Sample (IMPS). The IMPS is a master sample updated after each Census of Population and Housing, and which has been used in Bangladesh as the sampling frame for the selection of Primary Sampling Units (PSUs) in all previous rounds of the HIES and other surveys such as the Labor Force Surveys and Demographic and Health Surveys. The HIES 2016/17 deviates from the sampling design used in the previous rounds of the HIES in several ways. First, the objectives of the HIES 2016/17 have changed significantly from the one used to design the sample in previous rounds. Second, the HIES 2016/17 did not use the IMPS for the selection of PSUs. Third, the formulas used for the estimation of the optimal sample size and the allocation of the sample across strata were derived to explicitly take into account the clustered design of the sample. 5

8 2.2. Formulas for the optimal sample size and the optimal allocation of samples across areas In his influential book on survey sampling, Cochran (1977) presents two key formulas for designing any stratified simple random sampling. The first formula determines the minimum sample size required to achieve a certain level of statistical reliability regarding the relationship between one statistic and another targeted variable. The second formula, based on Neyman s (1934) original allocation rule, determines the optimal allocation of the sample across strata. 3 These two key formulas, presented in Cochran s (1977) book, assume that the sample follows a simple stratified random sampling design. 4 However, most surveys that we are interested in that is, household income and expenditure surveys and integrated household surveys designed to measure living standards and monitor poverty and welfare patterns are not a simple stratified random sample, but rather use multi-stage stratified clustered sampling designs. 5 Therefore, the original Cochran (1977) formulas for designing a sample for a living standards household survey is generally not appropriate and would result in an inadequate determination of the sample size and a sub-optimal allocation of the sample across strata. Nevertheless, the formulas are widely used for the selection of the sample size and allocations across strata. In this paper, we propose a modification to Cochran s sample size formula to estimate adequate sample sizes for multi-stage stratified clustered surveys. Cochran s (1977) original formula, building on work by Cornfield (1951) and Kish (1965), estimates sample size by calculating the minimum sample size required to achieve a certain level of precision (plus or minus x 3 The Neyman s (1934) original allocation rule minimizes the variance of the sample mean of a targeted variable given a fixed pre-established sample size assuming that the cost of sampling in each stratum is constant. 4 In sampling, stratification is the process of dividing members of a population into homogeneous subgroups before selecting the sample. This provides two main advantages over simple random sampling (SRS). First, it ensures that different subgroups of the population are properly represented in the sample. Second, it allows to gain statistical precision in the estimation of the variance of the mean of a targeted variable because the variability within the subgroups is lower compared to the variations across the entire population. 5 A stratified clustered sampling design consists in selecting the sample using multiple stages. In the first stage, the population is divided into groups (or clusters). In the second stage, one or more clusters are chosen at random and for everyone within the chosen cluster a second sampling unit (or ultimate sampling unit) is selected. 6

9 standard deviations) when comparing one variable to another. To modify this formula, we first calculate the sample size required to achieve the desired level of precision for a simple random sample (SRS). We then adjust this formula by the design effect (DEFF), which in this application measures the efficiency of a stratified clustered sample compared to SRS. We also derive two different formulas for estimating the sample size based on whether the targeted variable being studied is continuous or dichotomic. To determine optimal allocation of the sample across sampling strata, we also modify Neyman s (1939) original allocation rule, which incorporates the survey design effect and can therefore optimally allocate for multi-stage stratified clustering. The derivation of this formula follows Neyman s (1939) original closely and consists in solving an optimization problem that minimizes the stratified clustered sampling variance of the targeted variable of interest subject to a fixed pre-established sample size that is determined using the modified Cochran (1977) formula described above. The solution to this optimization problem gives us a more general optimal rule for the allocation of the sample across strata. This rule can be used both for multistage stratified clustered samples and for stratified simple random samples. 3. Sampling strategy 3.1. Objective of the survey As mentioned before, the main objective of the HIES 2016/17 is to produce reliable poverty statistics, measured by the poverty headcount ratio, at three different levels: (i) annual poverty estimates at the district level; (ii) annual poverty estimates at the division level for urban and rural areas; and (iii) quarterly poverty estimates at the national level. 6 A secondary objective of the HIES 2016/17 is to understand the rapid urbanization occurring in Bangladesh, and to 6 The poverty headcount ratio is defined as the proportion of people living below the official national poverty line. In Bangladesh, there are two official government poverty measures: the poverty headcount ratio and the extreme poverty headcount ratio. The poverty headcount ratio is based on a set of division-level poverty lines, defined separately for urban and rural areas, called upper poverty lines. In contrast, the extreme poverty headcount ratio is based on a set of division-level poverty lines, also defined separately for urban and rural areas, called lower poverty lines. Both upper and lower poverty lines include a food and non-food component, but they differ in how the non-food component is defined. For details on how the poverty lines and poverty estimates are produced, see World Bank (2013). 7

10 assess how urbanization is linked to poverty and welfare measures. The sample for the HIES 2016/17 is explicitly designed to produce reliable poverty estimates at the three levels, and is implicitly designed to produce an urban sample large enough to understand Bangladesh s urbanization patterns Data used for the sampling design The data used for the design of the sample for the HIES 2016/17 come from the latest round of the HIES collected in 2010 and include 12,240 households. The HIES 2010 sample was designed to provide reliable poverty and welfare statistics only at the division level for both urban and rural areas. Designing a household survey sample often entails using data from a previous household survey to estimate the key parameters needed to determine the sample size and how to allocate it across strata. In the case of Bangladesh, the latest round of the HIES was collected six years ago. Given the rapid growth observed in the country over the past decade, however, it is not clear whether these data provide a good representation of the actual living standards in the country or not. Still, previous household survey rounds usually provide the best source of information to design a sample. The main goal of Bangladesh s HIES 2016/17 survey was to estimate reliable poverty rates at more disaggregated levels than the HIES 2010 survey. 7 Standard errors of mean poverty rates estimated at the district-level poverty using the HIES 2016/17 might end up being a little bit higher than what is predicted under the proposed sampling design because reliable districtlevel poverty rates cannot always be constructed based on the HIES That is, some of the parameters estimated from the HIES 2010 to design the sample for the HIES 2016/17 might not provide accurate information of living standards in 2016/17. To address this concern, we analyze the robustness of the proposed sampling design for the HIES 2016/17 in several ways. The results of this robustness analysis are presented in Section 5. 7 The HIES 2016/17 survey instrument (i.e. questionnaire) is very similar to the one used in the HIES 2010 and field logistics are expected to follow similar protocols in terms of the survey implementation. 8

11 3.3. Sampling frame In the HIES 2016/17, the primary sampling units (PSUs) are the Census Enumeration Areas (EAs) used for Bangladesh s 2011 Census of Population and Housing. EAs are small geographic areas consisting of an average of 110 households. The sampling frame for the selection of PSUs consists in the list of all EAs covering people residing in non-institutional dwelling units (98.5 percent of the population in Bangladesh). 8 This is an important departure from the HIES 2010, where the sampling frame for the selection of PSUs consisted in the list of all EAs in the Bangladesh IMPS that excluded some geographic areas, such as urban slums. 9 For the HIES 2016/17, the IMPS could not be used because the most recent version of IMPS based on the 2011 Census of Population and Housing included only 2,012 EAs, an insufficient number to serve as a sampling frame for this survey. Table 1 summarizes the main characteristics of the sampling frame for the HIES 2016/17: Area Table 1 Sampling frame for the HIES 2016/17 Number of households Number of EAs Mean number of households in EA Rural 24,588, , Urban 5,020,815 43, City Corporation 2,253,737 21, Total 31,863, , Source: Bangladesh Bureau of Statistics, 2011 Census of Population and Housing 3.4. Stratification, sub-stratification, and domains of study In Bangladesh, the eight divisions are the first-level administrative geographical partitions of the country. As of 2016, the country had eight divisions: Barisal, Chittagong, Dhaka, Khulna, 8 The list of all EAs in the 2011 Census of Population and Housing includes 296,718 clusters, of which 293,340 EAs cover non-institutional dwelling units. 9 The IMPS is a master sample design that is updated after each Census of Population and Housing, and which has been used in Bangladesh as the sampling frame for the selection of PSUs in the HIES 2000, 2005, and Using a master sample frame for multiple surveys makes regular survey data collection more efficient. For details on how the Bangladesh IMPS Design was produced, see Maligalig and Martinez (2013). 9

12 Mymensingh, Rajshahi, Rangpur, and Sylhet. Each division is subsequently divided into 64 districts, or zilas. Each district is further subdivided into smaller geographic areas with clear rural and urban designations. In addition, urban areas in the main divisions of Chittagong, Dhaka, Khulna and Rajshahi are classified into City Corporations (CCs), or Metropolitan Areas and Other Urban areas. Table 2 shows the distribution of the population by division and across urban, rural, and CCs areas. Table 2 Distribution of population by division (%) Division Number of City Rural Urban Total districts Corporation (%) Barisal Chittagong Dhaka Mymensingh Khulna Rajshahi Rangpur Sylhet Total Source: Household Income and Expenditure Survey (2010) The sampling design for the HIES 2016/17 includes two different levels of stratification: (i) Division-Rural/Urban/CC: The eight administrative divisions in Bangladesh are initially used to construct 16 sampling strata: one urban and one rural for each division. In addition, each urban sampling stratum in the four main divisions of Chittagong, Dhaka, Khulna and Rajshahi is further divided into two sampling strata: City Corporation and Other Urban. This brings the total number of sampling strata to 20: eight in rural divisions, eight in urban divisions, and four main CCs. (ii) District- Rural/Urban/CC: As explained in detail below, PSUs in the HIES 2016/17 are allocated at the district level. Therefore, the sample is implicitly sub-stratified at the district level. Since there are a total of 64 districts in Bangladesh, the sample design includes a total of 132 sub-strata: 64 urban, 64 rural, and four main CCs. 10

13 The poverty and welfare domains of study for the HIES 2016/17 are slightly different from the sampling strata defined for the sampling design. Although the sample includes 20 sampling strata, official poverty statistics will only be produced at the following levels: (i) annual district level; (ii) quarterly national level; and (iii) annual division level for urban and rural areas (but not for CCs separately) Selecting target variables and accuracy statistics One of the first decisions when designing a sample is determining key target variables to study. That then determines the sample size and allows an assessment of the ability of the sample to achieve a certain level of precision to estimate statistics related to the key target variables. 10 In the case of the HIES 2016/17, the main objective is to generate reliable poverty estimates for Bangladesh at different levels of disaggregation. Poverty in Bangladesh is estimated based on per capita nominal consumption expenditures. As a result, three different target variables are used to design the HIES 2016/17 sample: (i) nominal per capita expenditure; (ii) nominal total household expenditure; and (iii) poverty headcount ratio. 11 Table 3 presents summary statistics for the three target variables by division. 10 In the case of integrated household surveys with multiple objectives, a decision needs to be made as to what the key target variables are for design of the sample. The number of target variables needs to be kept at a minimum, and must consist of closely related indicators with relatively similar levels of prevalence in the population of interest. Otherwise the proposed sample size would grow dramatically. 11 Nominal per capita expenditure is used as a target variable rather than real per capita expenditure in order to adhere to the traditional convention followed by BBS for the design of the samples for the previous three rounds of the HIES collected in 2000, 2005, and

14 Table 3 Summary statistics of target variables by division Nominal per capita Division expenditure (in 2010 taka) Barisal 2,154 (78) Chittagong 2,887 (144) Dhaka 2,870 (129) Mymensingh 1,827 (60) Khulna 2,182 (65) Rajshahi 2,231 (59) Rangpur 1,937 (63) Sylhet 2,184 (93) Nominal total household expenditure (in 2010 taka) 10,708 (425) 15,765 (797) 14,008 (703) 9,310 (334) 10,335 (323) 10,588 (345) 9,434 (343) 13,936 (727) Poverty headcount ratio (%) (3.33) (1.99) (1.64) (3.43) (2.28) (2.11) (3.22) (3.02) Note: Data for the estimation of these statistics comes from the HIES All estimates reported in this table have been produced using population level weights to match official statistics. For the proposed sampling design, however, we use household level weights. Standard errors adjusted for survey design are reported in parenthesis. A second important decision when designing a sample is selecting accuracy statistics. These are used to evaluate the performance of the proposed sample design to achieve a certain level of precision in the estimation of these statistics for the target variables of interest. The most common accuracy statistics are standard errors (SE) of the mean and relative standard errors (RSE) of the mean; most household surveys measuring living conditions try to achieve SEs of the mean of the target variable(s) that are no more than 5 percent, and RSEs of the mean that are no more than 10 percent. 12 For HIES 2016/17, we use three accuracy statistics to assess 12 The RSE of a sample mean is the SE of the mean divided by the mean and then expressed as a percentage. 12

15 the performance of the sample: (i) RSEs of less than 10 percent from the mean nominal per capita expenditure for each district and division (urban/rural), as well as for each quarter at the national level; (ii) RSEs of less than 10 percent from the mean nominal household expenditure for each district and division (urban/rural), and for each quarter at the national level; and (iii) SEs of less than 5 percent of the poverty headcount ratio for each district and division (urban/rural), and for each quarter at the national level. Final sample size determination, described in the next section, is based on the nominal per capita expenditure target variable, but we evaluate performance of the sample in terms of these three accuracy statistics to confirm that the final sample provides reliable poverty and welfare estimates at all these levels. It is important to highlight that RSEs are not used to evaluate the performance of the sample in terms of poverty rates. Instead, we use SEs for poverty rates. This responds to two reasons. First, the value of the RSE can go up very quickly when the poverty rates are small as it is the case in several districts in Bangladesh. For instance, in 2010 poverty rates of Kushtia and Noakhali were as low as 3.6 percent and 9.6 percent, while the poverty rates of the poorest districts of Kurigram, Barisal, and Shariatpur were as high as 63.7 percent, 54.8 percent, and 52.6 percent respectively. This large heterogeneity across districts in terms of poverty rates results in misleading results when interpreting RSEs. The poorest districts with elevated poverty rates will have relatively low RSEs, while better-off districts with low poverty rates will have relatively large RSEs. Therefore, the RSE is not a good accuracy statistic to use for target variables whose distributions show high levels of dispersion. Second, poverty measurement has traditionally focused on SEs to evaluate the precision of the poverty rates, which aligns more closely with the standard literature on poverty measurement and its practical applications Estimation of the sample size The HIES 2016/17 uses a stratified, two-stage clustered sampling design. In the first stage, PSUs are equally allocated across the 64 districts in Bangladesh: 36 PSUs are allocated to each of the 64 Bangladesh districts, bringing the total number of PSUs in the sample to 2,304 (36 PSUs*64 districts). An equal number of PSUs are allocated across districts because the HIES 13

16 2016/17 aims to produce poverty and welfare statistics with similar accuracy across all districts. In addition, we expect that achieving low RSEs and SEs in the estimation of districtlevel poverty and welfare will be more difficult than at the division level, as well as at the quarterly level. Therefore, allocating an equal number of PSUs across districts ensures that the final sample is evenly distributed across all districts. In the second stage, we randomly select 20 households from each PSU, resulting in 720 households allocated to each of the districts (36 PSUs*20 households). This brings the total targeted sample size to 46,080 households (2,304 PSUs*20 households). Assigning 36 PSUs to each district was determined as follows. First, this is the minimum sample size needed to achieve an RSE of less than 10 percent for the mean nominal per capita expenditure for each district and division (urban/rural), and for each quarter at the national level with 95 percent confidence (our selected accuracy statistics to determine sample size). To arrive at this number (36), we use a formula (derived in Annex 3) that is a simple extension of the original formula proposed by Cochran (1977) to determine sample size for continuous target variables under simple random sampling when controlling for a desired RSE. 13 This simple extension, which introduces survey design effects to the original formula, results in the following expression: / (1) where n is the minimum sample size required for allocation to each district in order to achieve a certain level in the accuracy statistic associated with the targeted variable ; is the coefficient of variation of the targeted variable estimated under the assumption of simple random sampling; DEFF is the design effect of the target variable; and / is the critical value of a standard normal distribution with α% confidence level Derivation of the formula to determine optimal sample size for proportions when one is interested in controlling for SE instead is provided in Annex This formula is derived under the assumption that the targeted variable is normally distributed. 14

17 In the case of the HIES 2016/17, is the 10 percent RSE desired for the mean total household expenditure estimated at the district level; DEFF is the average design effect of the target variable across all districts; is the coefficient of variation of total household expenditure estimated at the national level; and / is set at 1.96, which corresponds to a significance level of 5 percent. 15 We use a national estimate of the CV necessary to estimate the sample size needed to achieve the desired accuracy because we cannot always estimate district level CVs with enough precision based on the HIES 2010 data. Therefore, the above calculations assume that all districts have the same DEFFs and CVs for the target variable of interest. It is possible, that the final HIES 2016/17 sample might estimate district level welfare estimates with RSEs slightly larger than 10 percent, depending on the how close the CVs and DEFFs of the target district variables are to their actual values. It is also important to note that we focus on district-level subsamples to estimate sample size, because, as mentioned earlier, we expect that achieving accuracy statistics at the district level will be more difficult than at the higher division and quarterly levels of aggregation. By substituting all values in equation (1), we find that the sample size needed is 715 households for each district in order to obtain RSEs of mean total household expenditure below 10 percent. This brings the preliminary total sample size to 45,760 households (715*64 districts) for the HIES 2016/17. However, we allocate 720 households to each district to fit pre-established field protocols used by BBS in the HIES 2010, and to facilitate field work and survey implementation management. More specifically, 720 households is divisible by 36 (i.e. the number of PSUs ultimately allocated to each district), which allows easy allocation of 20 households per PSUs. Also, two enumeration teams can easily cover the 36 PSUs in each district over 12 months without having to move to a different district (each team of two enumerators will visit 18 PSUs each over one year). Lastly, the number PSUs, 36, allocated to each district is a multiple of 4, which allows dividing into quarters. 15 To estimate DEFF h, for each district, we use the Stata svy: mean, over(district) command combined with the estat effects, srssubpop and estat sd, srssubpop routine. An important note to mention here is that Stata uses a linear approximation to estimate survey design effects, which opens the possibility that for some district-stratum pairs we might observe DEFF values that are lower than 1. When this happens, the strategy adopted was to replace the estimated DEFF values with DEFF=1. 15

18 Although we can only focus on one accuracy statistic to estimate sample size, we also assess whether the sample also achieves the other proposed accuracy statistics in Section 4. Table 4 summarizes the key parameters of the HIES 2016/17 sample design. The final sample includes all 64 districts in Bangladesh and 20 strata (eight divisions*urban/rural=16, plus four main CCs). Each district is assigned 36 PSUs, an equal number for each district that we derived by modifying Cochran s formula to estimate the sample size for a target continuous variable for a stratified clustered sample design. The target variable used to estimate the sample size for the HIES 2016/17 is nominal total household expenditure, which follows the convention used in the three previous rounds of the HIES. The accuracy statistics used to assess the performance of the sample is an RSE within 10 percent for each district. The cluster is set at 20 households per PSU, which also follows the convention used in previous HIES rounds. The total number of PSUs is 2,304 and the targeted sample size is 46,080. In terms of field logistics, these numbers suggest that a total of 128 teams will be needed to collect field data, with each team consisting of two people (enumerators) that will visit 18 PSUs over 12 months. Table 4 HIES 2016/17 summary of sampling design Description Number Number of districts 64 Number of strata 20 Number of PSUs in each district 36 Number of households in each PSU 20 Total number of PSUs in sample 2,304 Total sample size 46,080 Total number of teams 128 Total number of enumerators Allocation of PSUs across sub-strata within districts In the second stage, we allocate the 36 PSUs assigned to each district across rural, urban, and CC using a modified version of the Neyman s allocation rule ( optimal allocation rule ) that 16

19 minimizes the standard error of a target variable while explicitly accounting for the stratified clustered sample design. 16 The standard Neyman allocation rule provides an optimal allocation of PSUs when the sample follows a stratified random sample design (i.e. squared root of the design effect of DEFTh=1 for each stratum h), and when finite sample correction is zero (i.e. fh=0). For stratified clustered samples, the standard Neyman allocation requires modification to achieve an optimal allocation. The modified Neyman allocation rule is given by the following expression: (2) where h is a sub-stratum or stratum within a district; is the optimal sample allocation in sub-stratum h; n is the total number of households in a district; is the population share in each sub-stratum h; is the standard deviation of the target variable of interest; DEFT is the square root of the design effect of the target variable of interest; and 1 is a finite sample correction factor. For the sample design of the HIES 2016, we assume finite sample correction is zero. 17 All parameters required to estimate the optimal allocation of PSUs across sub-strata are estimated based on the HIES 2010 data. Since the cluster size in the HIES 2016/17 is identical to the cluster size used in the HIES 2010 (20 households), the intra-cluster correlation coefficients used to estimate the DEFTs can be computed directly from the HIES 2010 data. 18 After deriving the modified Neyman allocation to produce reliable poverty statistics at the district level, the following five additional adjustments are made to achieve the other two goals 16 The Neyman allocation rule is derived from an optimization problem consisting in minimizing the variance of a target variable subject to a pre-determined fixed sample size. The full derivation of the modified Neyman allocation rule used for the allocation of PSUs in the HIES 2016/17 is presented in Annex The finite sample correction factor must be taken into account when the sample size corresponds to a large percentage of the size of the population of interest. This condition rarely holds in practice. 18 The performance of the sample is directly affected by the decision to fix ex-ante the cluster size in each PSU to 20 households. This follows the decision of the BBS to follow field data collection protocols closely aligned with those used in the three previous rounds of the HIES collected in 2000, 2005, and This paper does not assess the performance of the sample under the selection of different cluster sizes. 17

20 of the sample design producing accurate quarterly poverty estimates at the national level and producing accurate annual estimates at the division level for urban and rural areas. First, we round up numbers calculated using the modified version of the Neyman equation. Second, although the modified Neyman allocation is supposed to minimize standard errors of poverty rates at the district level given the original allocation of the sample across districts, this would fail to assign sub-samples of sufficient size to some CCs. Since producing reliable poverty statistics at the stratum level (urban, rural and CC areas of each division) is also a goal of the HIES 2016/17, we deviate from the original modified Neyman allocation and move some of the PSUs from urban and rural areas to CCs. This deviation will slightly increase the standard errors of poverty statistics for the district level including the CCs, but it will overall reduce poverty statistics at the stratum level. 19 Third, we allocate a minimum of four PSUs to each stratum of a district so that at least one PSU is allocated to each quarter. Fourth, the HIES 2010 sample did not assign any PSU to the urban Khulna district, but the minimum of four PSUs are assigned to this stratum in the HIES 2016/ Lastly, to produce nationally representative poverty rates for each quarter, the sample needs to be allocated to the four quarters of the data collection uniformly. Munoz (2004) suggested that to avoid seasonality, the entire sample should be randomly split into four quarters. However, randomly allocating the sample across quarters does not guarantee a full removal of seasonality. In particular, if the number of PSUs allocated to a stratum in a specific district is a multiple of four, then we can allocate the same number of PSUs to all quarters. Yet, if the number of PSUs allocated to a stratum in a specific district is not a multiple of four, there is 19 Note that we are not changing the sample size of districts. This is certainly suboptimal. Ideally, we can shift samples from districts with very low projected standard errors to districts with the problematic strata; however, how to design such an allocation requires more careful analysis and is beyond the scope of this paper. 20 The HIES 2010 sample frame was based on the IMPS, which in urban areas was constructed using the concept of Statistical Metropolitan Areas (SMA). The concept of SMA was replaced by the concept of Rural/Urban/CC (RUC) in the 2011 Census of Population and Housing. Of the 64 districts, only in three does the old SMA concept not match perfectly with the new RUC: in the districts of Gazipur and Narayanganj in the Dhaka division (districts 33 and 67, respectively), and the district of Khulna in the Khulna division. For Gazipur and Narayanganj, a perfect match can be achieved by replacing all SMA areas to Other Urban areas. For Khulna district, however, a match is not straightforward as the SMA area was divided into CC and Other urban areas. In addition, all of the PSUs from the Khulna district available from the HIES 2010 come from SMA areas, and there is therefore no baseline for Other Urban Khulna district. To conduct all the analysis for the design of the sample for the HIES 2016/17, we use the same sub-stratum values for both CC and Other Urban areas. 18

21 no way we can allocate the same number of PSUs to all quarters. As a result, the distribution of the sample across strata in these districts might differ across quarters. Now, let s look at Table 5, which illustrates the challenges faced when trying to allocate PSUs across urban and rural strata within a district. For example, 26 PSUs are assigned to the rural stratum and 10 to the urban stratum. 26 is not a multiple of four, so we need to assign the closest integers to each of the quarters (6.5) which are 6 and 7. For example, Table 5 assigns 7 PSUs to quarters 1 and 2 and 6 PSUs to quarters 3 and 4. We repeat the same procedure for the urban stratum. This allocation in Table 5 is problematic for two reasons: (i) the share of rural areas differs across quarters; and (ii) the total sample size of quarters 1 and 2 is different from that of quarters 3 and 4. The first issue is problematic because poverty rates can differ across quarters if poverty rates in urban areas are different from those in rural areas even if true poverty rates do not actually change. The second issue is problematic because the precision of the poverty statistics across quarters will differ. Table 5 Example of challenge for allocating samples over quarters Division District Stratum Quarter 1 Quarter 2 Quarter 3 Quarter 4 Total 40 1 Rural Urban Sample for district by quarter Share of rural sample 70% 70% 75% 75% 72% To address these issues, we first tackle the second problem assigning the same number of PSUs to all quarters at the district level. Given that 36 is a multiple of four, we make sure we assign 9 PSUs to each of four quarters. This means for quarter 1 and 2, we assign 7 PSUs for the rural stratum of this district; so we need to assign 2 PSUs to the urban stratum of this district (Table 6). A problem of this adjustment is that the share of the rural sample within this district differs across quarters, even more than in the previous allocation (Table 5). In Table 5, the share of the rural sample ranges between 70 percent and 75 percent, but in Table 6, the share ranges between 67 percent and 78 percent. 19

22 Table 6 Example of a modified allocation of samples over quarters Division District Stratum Quarter 1 Quarter 2 Quarter 3 Quarter 4 Total 40 1 Rural Urban Sample for district by quarter Share of rural sample 78% 78% 67% 67% 72% To maintain balance in the distribution of PSUs across strata at the national level over quarters, we randomly select quarters with more PSUs than other quarters. More specifically, the allocation is done as follows. The selection of quarters with more PSUs than others is made only at the first stratum of each district. In the example in Table 6, the first stratum is always the rural stratum of each district. PSUs in other substrata are allocated so that the sum of PSUs for each quarter is 9. For each of the first stratum within a district, we randomly select a quarter from which we start a sequential allocation of PSUs over quarters. This is how the sequential assignment works. For example, for the rural stratum of district 4 in division 10 in Table 7, we randomly select quarter 1 as the starting point for the sequential allocation. Then, we assign one PSU for each quarter in order. Since we start from quarter 1 and the total number of PSUs allocated to this stratum of the district is 25, the sequential allocation of PSUs ends with quarter 1 and as a result, only quarter 1 has 7 PSUs and all others have 6 PSUs. For the rural stratum of district 6 in division 10, we select quarter 3 as a start of this sequential assignment of PSUs. Given that the total number of PSUs allocated to this area is 26, the sequential allocation ends with quarter 4 and thus quarters 3 and 4 receive 7 PSUs while quarters 1 and 2 receive 6 PSUs. Since the selection of starting quarters is random, if we repeat this allocation, we can achieve a similar distribution of PSUs across urban, rural and CC areas at the national and division levels. The example illustrated in Table 7 shows that with just three districts, the share of rural sample is already converging to 54 percent. 20

23 Table 7 Example of an allocation of samples over quarters and across districts Division District Stratum Quarter 1 Quarter 2 Quarter 3 Quarter 4 Total 10 4 Rural Urban Rural Urban Rural Urban Sample for district by quarter Share of rural sample 52% 52% 56% 56% 54% Note: Numbers highlighted represent the first quarter of the sequential allocation of PSUs across quarters. While incorporating these five manual adjustments to the original Neyman sample allocation implies deviating slightly from the optimal allocation, our assessment of the sample performance presented later in the paper shows that the final sample achieves the specified levels of accuracy for the targeted variables. Table 8 shows the final allocation of PSUs across sub-strata after these four manual adjustments. Table 8 HIES 2016/17 allocation of PSUs by district and quarters Division District Stratum Quarter 1 Quarter 2 Quarter 3 Quarter 4 Total

24 Division District Stratum Quarter 1 Quarter 2 Quarter 3 Quarter 4 Total

25 Division District Stratum Quarter 1 Quarter 2 Quarter 3 Quarter 4 Total