PREFACE. An overview of the NSAF sample design, data collection techniques, and estimation methods

Transcription

1

2 PREFACE 2002 NSAF Sample Design is the second report in a series describing the methodology of the 2002 National Survey of America s Families (NSAF). The NSAF is part of the Assessing the New Federalism project at the Urban Institute, conducted in partnership with Child Trends. Data collection for the NSAF was conducted by Westat. The NSAF is a major household survey focusing on the economic, health, and social characteristics of children, adults under the age of 65, and their families. During the third round of the survey in 2002, interviews were conducted with over 40,000 families, yielding information on over 100,000 people. The NSAF sample is representative of the nation as a whole and of 13 states, and therefore has an unprecedented ability to measure differences between states. About the Methodology Series This series of reports has been developed to provide readers with a detailed description of the methods employed to conduct the 2002 NSAF. The 2002 series of reports include: No. 1: No. 2: No. 3: No. 4: No. 5: No. 6: No. 7: No. 8: No. 9: No. 10: No. 11: No. 12: An overview of the NSAF sample design, data collection techniques, and estimation methods A detailed description of the NSAF sample design for both telephone and inperson interviews Methods employed to produce estimation weights and the procedures used to make state and national estimates for Snapshots of America s Families Methods used to compute and results of computing sampling errors Processes used to complete the in-person component of the NSAF Collection of NSAF papers Studies conducted to understand the reasons for nonresponse and the impact of missing data Response rates obtained (taking the estimation weights into account) and methods used to compute these rates Methods employed to complete the telephone component of the NSAF Data editing procedures and imputation techniques for missing variables User s guide for public use microdata 2002 NSAF questionnaire

3 About This Report Report No. 2 describes the sample design for the 2002 NSAF. As in previous rounds of the survey, the 2002 NSAF sample consists of a random digit dial (RDD) telephone sample supplemented by an area probability sample of nontelephone households. While the nontelephone sample for previous NSAF rounds were both nationally and state representative, the 2002 NSAF used only a nationally representative nontelephone sample. The report covers both the telephone and nontelephone sample design, adjustments made to the sample design during the field period, within household sampling procedures and achieved sample sizes. For More Information For more information about the National Survey of America s Families, contact: Assessing the New Federalism Urban Institute 2100 M Street, NW Washington, DC nsaf@ui.urban.org Web site: Adam Safir and Tim Triplett ii

4 CONTENTS Chapter Page 1 OVERVIEW PRINCIPAL FEATURES OF SAMPLE DESIGN BY ROUND The Survey Survey Components Number of Completed Interviews Projected Effective Sample Size RANDOM DIGIT DIAL HOUSEHOLD SAMPLING Sampling Telephone Numbers Subsampling Households Subsampling Adult-Only Households Subsampling High-Income Households Household Sampling Revisions during Data Collection Achieved Response and Eligibility Rates AREA SAMPLE First-Stage Sampling Second-Stage Sampling Exclusion of Block Groups with High Telephone Coverage Rates Segment Stratification and Selection Chunk Selection Achieved Response and Eligibility Rates WITHIN-HOUSEHOLD SAMPLING AND ACHIEVED SAMPLE SIZES Sampling Children Sample Selection of Other Adults in Households with Children Sample Selection of Adults from Adult-Only Households Achieved Sample Sizes and Response Rates iii

5 CONTENTS (continued) Chapter Page 6 CONCLUSION REFERENCES... R-1 Tables Table 2-1 Number of Completed Interviews by Round, Sample Type, and Interview Type Projected Design Effects for the RDD Sample, by Interview Type and Study Area Projected Effective Sample Sizes for the RDD Sample, by Interview Type and Study Area Assumed Proportion of Households by Household Type and Poverty Status Assumed Misclassification Rates, by Income Categories Assumed Residential and Response Rates Subsampling or Household Retention Rates Income Levels for Determining Less than 200 Percent of the Poverty Level Revised Subsampling Rates for Households with Children Screening as High-Income, by Release Group Revised Subsampling Rates for Adult-Only Households, by Release Group Reserve Sample Released, by Study Area Screening for Residential Status and Presence of Children iv

6 CONTENTS (continued) Tables (continued) Table Page 3-10 Subsampling Screener Refusals and Response Rates Outcomes of Household Screening of Telephone Households Outcomes of Income Screening of Telephone Households with Children Outcomes of Income Screening of Adult-Only Telephone Households Number of Round 3 Primary Sampling Units Maximum Telephone Service Rates Allowed in Covered Block Groups Segment Counts, by Planned and Unplanned Chunking (Including Sample Supplement) Outcomes of Area Listing Outcomes of Area Prescreening and Screening Proportion of Random Digit Dial Adult-Only Households with Three Other Adults under Age 65 in which Just One Adult Is Selected Within-Household Sampling and Extended Interviews of Children in Telephone Low-Income Households Within-Household Sampling and Extended Interviews of Children in All Subsampled Telephone Households Within-Household Sampling and Extended Interviews of Other Adults in Subsampled Telephone Households with Children v

7 CONTENTS (continued) Tables (continued) Table Page 5-5 Subsampling and Extended Interviews of Adults in Subsampled Adult-Only Telephone Households Sources of Adult Telephone Interviews Within-Household Sampling and Extended Interviews of Children in Nontelephone Low-Income Households Within-Household Sampling and Extended Interviews of Children in All Nontelephone Households Within-Household Sampling and Extended Interviews of Other Adults in Nontelephone Households with Children Subsampling and Extended Interviews of Adults in Adult-Only Nontelephone Households Sources of Adult Nontelephone Interviews Within-Household Sampling and Extended Interviews of Children in Telephone and Nontelephone Low-Income Households Within-Household Sampling and Extended Interviews of Children in All Telephone and Nontelephone Households Within-Household Sampling and Extended Interviews of Other Adults in Telephone and Nontelephone Households with Children Subsampling and Extended Interviews of Adults in Adult-Only Telephone and Nontelephone Households Sources of Adult Telephone and Nontelephone Interviews vi

8 CONTENTS (continued) Figures Figure Page 2-1 Study Areas Sampling Frame Inclusions and Exclusions Household Subsampling Operations vii

9

10 1. OVERVIEW This report describes the sample design for the 2002 National Survey of America s Families (NSAF). This survey is the third round of the NSAF, and the objective is to estimate both the characteristics of households and persons in 2002 and changes in those characteristics since the 1997 and 1999 NSAF. While the designs for all three rounds of the NSAF are similar, several important differences exist. The first survey in 1997 was a dual-frame survey of both households with telephones and those without telephones developed to serve as a baseline for evaluating changes over time. The Round 2 survey was designed to improve estimates of change between 1997 and 1999 by retaining a substantial portion of the Round 1 sample. Analysis of the Round 2 data showed that the design changes did not improve the precision for estimates of change between rounds as well as expected. Furthermore, the retention of a portion of the sample resulted in additional operational and design complications. Based on the findings from Round 2, the sample design for Round 3 was developed to be similar to the Round 1 design, in the sense that the sample was mainly independent of the sample from previous rounds. However, the sample design for Round 3 did include important modifications from the previous rounds designs that were intended to reduce data collection costs. The most important design change was the reduction of the sample size for nontelephone households in the study areas. This change also has important implications for the estimation strategy, which is discussed in 2002 NSAF Sample Estimation Survey Weights, Report No. 3. This report describes the sample design and how it relates to the designs from previous rounds. It also provides the details needed to appreciate the considerations that went into the decisions that resulted in the features of this large and complex survey. Chapter 2 summarizes the survey goals and the sampled units, and introduces its two major components, the telephone and in-person surveys. One of the main objectives of Chapter 2 is to describe the similarities and differences between the Round 3 sample design and the designs for the previous rounds. The remaining chapters focus primarily on the Round 3 design. Chapter 3 describes the random digit dial (RDD) telephone sample design and the sampling of households in the telephone component. The subsampling procedures for households without children and for high-income households are included in this chapter. Chapter 4 gives a detailed account of the sampling for the in-person survey component. It discusses the changes in the sampling needed to move from a sample for each study area to an overall national sample for this component of the survey. Chapter 5 presents the methods used to sample children and adults from within the sampled households. It contains tables on the number of sampled and interviewed persons from the survey. Chapter 6 provides some concluding remarks. 1-1

11 2. PRINCIPAL FEATURES OF SAMPLE DESIGN BY ROUND The samples for all three rounds of NSAF Round 1 in 1997, Round 2 in 1999, and Round 3 in 2002 have similar designs. A sample design Report No. 2 similar to this one is available for each earlier survey. These reports describe the details of the sample design for the specific survey. In this chapter we discuss the design features for Round 3 in relation to the features from the previous rounds. We focus on how the Round 3 design differs from that used in Round 1 and Round 2. The specifics of the Round 3 design are given in subsequent chapters. 2.1 The Survey The NSAF collected information on the economic, health, and social dimensions of the wellbeing of children, adults under the age of 65, and their families in 13 states and the balance of the nation. The Urban Institute selected these study areas (see figure 2-1) in 1996 prior to the first survey because they represent a broad range of fiscal capacity, child well-being, and approaches to government programs. Data were also collected in the balance of the nation to permit estimates for the United States as a whole. Figure 2-1. Study Areas Alabama Massachusetts New Jersey California Michigan New York Wisconsin Colorado Minnesota Texas Balance of nation Florida Mississippi Washington In Round 1 and Round 2, Milwaukee County in Wisconsin was a separate study area that had its own sample. A separate sample from the balance of Wisconsin was selected in these rounds to produce estimates for the entire state of Wisconsin by combining Milwaukee and the balance of Wisconsin. In Round 3, the separate Milwaukee County study area was eliminated and the entire state of Wisconsin was treated as a single study area, as shown in figure 2-1. The primary goal of the survey in all three rounds was to obtain social and economic information about children in low-income households (those with incomes below 200 percent of the poverty threshold), since the impact of New Federalism was likely to be greatest on these children. Secondary goals included obtaining similar data on children in higher-income households, plus adults under age 65 (with and without children). With few exceptions, the decision was made to limit the survey to children, adults, and families living in regular housing. Figure 2-2 explains the concept of regular housing through examples of specific inclusions and exclusions. Although one impact of New Federalism could be the displacement of persons from regular housing, including the population that lives outside of 2-1

12 Figure 2-2. Sampling Frame Inclusions and Exclusions Inclusions Houses, apartments, and mobile homes occupied by individuals, families, multiple families, or extended families where at least one occupant is under the age of 65 Houses, apartments, and mobile homes occupied by multiple unrelated persons, provided that the number of unrelated persons is less than nine and at least one occupant is under the age of 65 People in workers dormitories and camps Military personnel living on post with their families, as well as military personnel living off post with or without their families Included Persons in Excluded Structures People living temporarily away from home were enumerated at their usual residences. This includes college students in dormitories, patients in hospitals, vacationers, business travelers, snowbirds, and so on. Structures that were expected to primarily include only such people were excluded Exclusions The institutionalized population. Examples of institutions include prisons, jails, juvenile detention facilities, psychiatric hospitals and residential treatment programs, and nursing homes for the disabled and aged Noninstitutional group quarters, including communes, monasteries, convents, group homes for the mentally or physically disabled, shelters, halfway houses, dormitories, and dwelling units with nine or more unrelated persons The homeless People in transient hotel/motel rooms, tents, recreational vehicles, trailers, and other similar temporary arrangements Military barracks and ships 2-2

13 regular housing was considered infeasible within the survey context. The elderly population was also excluded. College students were enumerated at their parents homes. Most of these inclusions and exclusions are typical of those made in other household surveys. For example, the Current Population Survey (CPS) has essentially the same rules (U.S. Bureau of the Census 2000). The major difference is that the CPS includes military personnel living on post with families but excludes those living in noninstitutional group quarters. 2.2 Survey Components The sample for each round of NSAF has two separate components: an RDD survey of households with telephones, and an area survey of households without telephones. The RDD component provides a cost-effective method to collect the desired data on a large number of households for each study area and nationally. The area component enables the survey to cover households without telephones. The area component is important in this survey because lowincome households are more highly concentrated in nontelephone households than the entire universe of households. For example, Giesbrecht et al. (1996) estimated that about 20 percent of families in poverty live in households without a telephone and that about 10 percent of families with one child 3 years old or under have no telephone based on CPS data. Even though these results are out-of-date given the changes in technology in the past 5 to 10 years, the only recent data on the relationship of poverty and having a telephone in the household published are from the 2000 Census and are discussed below. Even these data do not distinguish between landline and wireless telephones. After the area sample was selected, households were screened to find households without telephones. Only these nontelephone households were interviewed from the area sample. When Round 1 was designed, the dual-frame approach to this type of problem was relatively new and data to optimize various aspects of the design were not available. Despite the lack of data, it was clear that the dual-frame approach produced more precise estimates than a pure areasampling approach of the same cost. In addition, it produced less biased estimates than a pure RDD approach of the same cost. Waksberg et al. (1997) describe some of the options that were considered early in the process of designing the Round 1 survey. The Round 2 sample design used data from Round 1 to improve survey efficiency. The research supporting revisions to the Round 2 design are described in chapter 3 of 1999 NSAF Sample Design, Report No. 2. The primary emphasis in Round 2 shifted from producing estimates of the current level (the Round 1 design objective) to estimating changes from 1997 to As a result, a substantial proportion of the sample from Round 1 was retained and included in the Round 2 sample. For the RDD sample, the proportion of the sample of telephone numbers retained depended on the outcome of the Round 1 interview. Round 1 telephone numbers that were residential and cooperative were sampled at higher rates than those numbers that were either nonresidential or uncooperative. In addition to retaining a portion of the Round 1 sample, telephone numbers not in existence in Round 1 were sampled to provide complete coverage of telephone households in The Round 2 area sample was largely the same as used in 2-3

14 Round 1, but an additional sample from the balance of the nation was added to improve the precision of national estimates. Analysis of the Round 2 data found that the expected improvements in the precision of change estimates due to retaining a portion of the Round 1 sample were not as large as had been expected. The reduction in variances for change estimates depended on retaining a substantial fraction of the Round 1 households and the characteristics of the retained households being highly positively correlated across time. The design assumptions regarding both of these key factors were somewhat optimistic. In the component of the sample in which an interview was completed in Round 1 and the telephone number was retained for Round 2, the proportion of identical households was less than assumed. In addition, even when the same household responded in both rounds, the correlation for important estimates was lower than expected. Given these Round 2 findings, it was decided that it would be unwise to retain a sample of telephone numbers from the previous rounds for Round 3. Retaining telephone numbers has its own costs in terms of introducing differential sampling fractions that decrease the precision of the estimates and lower response rates. Report No. 8 in the 1999 series showed the screener response rate in the retained sample was about 2 or 3 points lower than the response rate in the newly sampled numbers from the same sampling frame. This result is consistent with panel surveys (see Kalton, Kaspryzk, and McMillen, 1989). We examine the 2002 response rates in Report No. 8 and provide a more direct assessment of the effect of retaining telephone numbers in the sample there. The retained sample also added some complexity operationally. Thus, it was decided that the Round 3 telephone sample should be independent of the samples selected in Round 1 and Round 2. Although the sample for Round 3 was selected independently, the allocation of the sample to the study areas and the sampling procedures were very similar to those used in Round 1. On the other hand, the design for the Round 3 area sample was very different from that used in the previous rounds. In Round 1 and Round 2, separate area samples were selected in each study area and for the balance of the nation. The area and RDD samples were combined to produce estimates of both telephone and nontelephone households for each study area and nationally. Because of the cost of screening a large sample of households to find nontelephone households, the sample size for the area sample was relatively small in each of the study areas. The small sample size of nontelephone households caused some instability in the estimates of the precision for the study areas (see 1997 Report No. 4 for more details). At the national level, the sample size was sufficiently large that the variance estimates were reliable. These concerns about the area sample for producing reliable estimates of nontelephone households at the study area level led to research to find a lower-cost alternative. Ferraro and Brick (2001) studied various methods for adjusting a sample of telephone households to account for the undercoverage of households without telephones. This research found that an approach called modified poststratification had better statistical properties for NSAF study areas than previously considered approaches. In addition, new data from the 2000 Census of Population showed the percentage of households without telephones was much smaller than reported in the 2-4

15 CPS for the same time period. The census estimates only 2.4 percent of households did not have a telephone, compared with the 4.9 percent reported in the CPS for the same time. Based on cost, stability of the variance estimates, new research on statistical adjustment methods, and lower estimates of the percentage on households without telephones, the sample design for the area sample for Round 3 was changed significantly from that used in previous rounds. The Round 3 design included a sufficient sample to produce reliable national estimates of all households, using the area sample to represent households without telephones. At the same time, the area sample size for the study areas was reduced, and the plan was to rely on the modified poststratification approach for estimating characteristics of all households for the study areas. Thus, the sample size for the area sample and the cost of collecting these data were significantly reduced. Tables later in this chapter show the changes in sample size at the study area level and overall. The other basic features of the sample design were very consistent across the three rounds of data collection. In all three rounds, costs were reduced through the use of screener-based subsampling of households contacted in the RDD component. In this approach the RDD screening interview includes a very short income question. Those households that report no children in the household or reported incomes above 200 percent of the poverty threshold were subsampled. The extended interview has more detailed and reliable income questions for those included in the subsample. In Round 1, the inconsistency between the responses to the short and detailed versions of the income questions was greater than anticipated, and the Round 2 subsampling rates were revised in Round 2 to account for this difference. In Round 3, subsampling rates similar to those suggested by the Round 2 research (see 1999 NSAF Sample Design, Report No. 2) were used. In all the rounds and across both the RDD and the area samples, the number of household members that could be sampled and interviewed were limited. The main reason to impose these limits was to reduce the respondent burden for the household as a whole. Even if there were several children under age 6 in a household, only one was randomly selected. Similarly, only one child age 6 to 17 was sampled in a household. The most knowledgeable adult (MKA) in the household for the child was interviewed about the sample child. During the MKA interview, additional data were collected about the MKA and about the MKA s spouse/partner, if that person was living in the same household. The MKA provided all the data about the spouse/partner. Generally, every question about the MKA was repeated with reference to the spouse/partner. However, some questions on health insurance and health care usage were asked about only one of the two. The appropriate person targeted for these questions was randomly assigned as either the MKA or the spouse/partner. Some questions were asked only about the MKA, related to feelings, religious activities, and opinions. These items were not repeated for the spouse/partner because proxy response did not seem sufficiently valid or reliable, and because self-response on these few questions was operationally impractical. All these rules for subsampling persons within sample households were the same for all three rounds. 2-5

16 Two other within-household subsampling steps were used in all three rounds of data collection. Other adults in households with children (adults who were not the MKA of any children in the household) and adults in adult-only households were subsampled. The rules for the subsampling were complex and are described in detail in the 1997 NSAF Sample Design, Report No. 2. Selfresponse was required for sample adults. During the interview with a sample adult, additional data were collected about the sample adult s spouse/partner, if they were living in the same household. As in the MKA interview, the data were always collected by asking the sample person to respond for the spouse/partner. No attempt was made to collect these data directly from the spouse of a sample adult. As in the MKA interview, some questions were asked only about the sample adult, related to feelings, religious activities, and opinions. In all three rounds of NSAF, the sample design set in place at the beginning of the survey was revised somewhat as data on the outcomes of the data collection became available. For example, the expected sample sizes depended on assumptions about residency rates, response rates, poverty rates, and other parameters. Since the observed rates differed from the assumed rates, sometimes revisions in sampling rates or the number of sampled units had to be implemented as the data collection proceeded. For the first two rounds, the adjustments that took place are documented in the previous sample design reports. In Round 3, the outstanding changes that took place during the field period involved releasing more telephone numbers than originally planned due to lower-than-expected residency and response rates, and using a refusal subsampling procedure for the RDD sample to speed up the data collection and reduce costs slightly. These changes to the original sample design are discussed in later chapters, along with the other more minor changes. 2.3 Number of Completed Interviews The number of completed interviews for each round of NSAF is given in table 2-1. The table shows the number of completed interview by type of sample (RDD or area sample) and by type of interview (various types of adult interviews and child interviews). Note that one MKA interview can provide data on up to two sample children and that there were a few MKA interviews with parents under the age of 18. In addition to the total number of interviews, the table also shows the range of the number of completed interviews in each study area and in the balance of the nation by round. The table shows that the number of completed RDD interviews is about the same for all three rounds. The big difference is in the number of completed area interviews, where the number completed in Round 3 is lower than the previous rounds. This difference is particularly obvious in the study areas. This difference is a consequence of the sample design change described earlier. 2-6

17 Table 2-1. Number of Completed Interviews by Round, Sample Type, and Interview Type 2-7 Total Range for Study Areas Balance of the Nation Type of Sampled Interview Round 1 Round 2 Round 3 Round 1 Round 2 Round 3 Round 1 Round 2 Round 3 RDD sample All adults 46,621 45,025 43,133 2,347 3,771 2,085 3,746 1,933 3,798 4,913 6,168 7,251 Adult MKAs 27,248 29,054 28,208 1,431 2,025 1,309 2,382 1,254 2,196 2,669 3,610 4,603 Other adults in households 2,407 3,054 2, with children Adults in households 16,966 13,917 12, , ,346 2,019 2,062 2,127 without children Children under age 6 12,067 11,990 12, ,179 1,462 1,987 Low-income children under age 6 7,246 4,923 5, Children age 6 to 17 21,210 22,841 21,864 1,103 1,557 1,056 1,881 1,001 1,708 2,064 2,872 3,557 Low-income children age 6 11,813 8,560 8, ,182 1,097 1,501 to 17 Area sample All adults 1,682 1, Adult MKAs Other adults in households with children Adults in households without children Children under age Low-income children under age 6 Children age 6 to Low-income children age 6 to

18 2.4 Projected Effective Sample Size In surveys like NSAF that sample study areas and specific subgroups of the population at different rates, the number of completed interviews does not give a full and accurate picture of the precision of the estimates. Sampling with differential probabilities generally reduces the precision of estimates aggregated over groups with different rates, even though it improves the precision for those subgroups sampled at higher rates. For example, households in the study areas are sampled at higher rates than are households in the balance of the nation, and the precision of national estimates is lower than it would be if the rates for the groups were identical. Of course, the higher sampling rates in the study areas provide reliable estimates for these areas that would not be possible otherwise. In designing the sample, the losses in efficiency of the samples due to the differential sampling rates were taken into account in determining the needed sample sizes. The other source of efficiency losses in typical household surveys is clustering of sampled households in geographic areas. List-assisted RDD surveys do not cluster households, so the only clustering of sampled households in the NSAF is for nontelephone households. Since the nontelephone households are a small proportion of the total sample, we developed our projections of the precision based only on the RDD sample. The design effect or deff (the ratio of the variance of an estimate under the actual design to what would be obtained with a simple random sample of the same size) is one method of accounting for the efficiency of the sample. Another way of thinking about the design effect is that it is the inverse of the efficiency of the sample, so that a sample with a deff of 2 is equivalent to a simple random sample of half the sample size. Design effects were estimated from the samples for the previous two rounds and were reported in the corresponding Report No. 4 for the round. These average deffs were instrumental in projecting design effects for Round 3. The primary goal of the sample design was to obtain approximately the same number of lowincome child interviews as achieved in Round 2, with a secondary objective of obtaining about the same number of low-income adult interviews. Since the efficiency of the sample is directly tied to the rates used for subsampling the different groups, keeping the subsampling rates as consistent as possible with the optimal rates developed in Round 2 while still completing about the same number of interviews was also important. Other factors also affected the sample design. The sample size allocated to the Milwaukee study area could now be used in other areas because this was dropped as a stand-alone study area for Round 3. The sample size for this area was reallocated to California, the balance of the nation, and Michigan to improve the precision for these areas and the national estimates. Once the sample sizes were determined, projected design effects and effective sample sizes for the key interview groups could be computed. The interview groups are children, other adults (who are not MKAs), and all adults. Since the survey is critically interested in people in families with incomes less than 200 percent of the poverty level, table 2-2 has the projected deffs for both low-income and all persons. Using these projected design effects and an additional adjustment due to other losses of efficiency in the sample design, the effective sample sizes for each study area and for the nation were computed. These projected effective RDD sample sizes are shown in table 2-3 for all groups. 2-8

19 The projected design effects and effective sample size shown in tables 2-2 and 2-3 were computed prior to the start of data collection. Thus, changes in the design that occurred during data collection and the actual sample sizes achieved are not reflected in these tables. The average design effects based on the data collected in Round 3 are given in Report No. 4. The next two chapters describe the procedures used to select the RDD and area sample of households. 2-9

20 Table 2-2. Projected Design Effects for the RDD Sample, by Interview Type and Study Area Children Other Adults All Adults Study Area Low-Income All Low-Income All Low-Income All Alabama California Colorado Florida Massachusetts Michigan Minnesota Mississippi New Jersey New York Texas Washington Wisconsin Bal. of nation National Table 2-3. Projected Effective Sample Sizes for the RDD Sample, by Interview Type and Study Area Children Other Adults All Adults Study Area Low-Income All Low-Income All Low-Income All Alabama 663 1, ,207 California 845 2, ,720 Colorado 634 1, ,236 Florida 662 1, ,012 Massachusetts 594 1, ,546 Michigan 665 1, ,804 Minnesota 562 1, ,216 Mississippi 690 1, ,080 New Jersey 619 2, ,530 New York 747 1, ,198 Texas 802 1, ,269 Washington 629 1, ,422 Wisconsin 563 1, ,647 Bal. of nation 1,699 4, ,546 1,588 3,729 National 5,225 12,854 1,025 3,643 3,960 11,

21 3. RANDOM DIGIT DIAL HOUSEHOLD SAMPLING This chapter describes the sample design and implementation for the RDD component of the survey. The first section describes the sampling telephone numbers using the list-assisted sample design for each study area and the balance of the nation. The second section describes the overall plan for subsampling households. The third and fourth sections go into detail about the main subsampling procedures, subsampling adult-only households and high-income households, respectively. The fifth section describes changes to the sampling parameters made during the data collection period and based on monitoring the progress of the earlier stages of the sample. The final section of the chapter presents tables on the sample sizes achieved using the methods discussed. 3.1 Sampling Telephone Numbers The RDD sample for all three rounds of NSAF used a list-assisted approach to select the sample of telephone households. These households were screened to identify low-income households with children and other households of interest as described later in this chapter. Casady and Lepkowski (1993) describe list-assisted sampling and a recent update to the application of this method is Tucker, Lepkowski, and Piekarski (2002). In list-assisted sampling, the set of all possible residential telephone numbers is divided into 100-banks. Each 100-bank contains the 100 telephone numbers with the same first eight digits (i.e., the identical area code, telephone prefix, and first two of the last four digits of the telephone number). The frame consists of all 100-banks with at least one residential number listed in a published telephone directory. Any household telephone number that is not in these 100-banks is not covered by the sample. A simple random or a systematic sample of telephone numbers is selected from this frame. List-assisted RDD sampling is now the standard sampling procedure for telephone surveys. The key advantages of this method relative to the Mitofsky-Waksberg method (Waksberg 1978) is that the sample is unclustered, and the full sample of telephone numbers can be released to interviewers without the sequential impediment in the Mitofsky-Waksberg method. A disadvantage of list-assisted RDD is a small amount of undercoverage due to excluding households in 100-banks with no listed households. Studies of the undercoverage due to this exclusion have shown that only a small percentage of households are excluded, and households excluded from the frame are not very different from those that are included (Brick, et al. 1995; Tucker et al. 2002). Together, these attributes indicate the undercoverage bias from excluding 100-banks with no listed households is small. Once the sample was selected from the 100-banks, two steps were used to improve the working residential rate and thereby reduce costs. The first procedure eliminated numbers listed only in the yellow pages. The second procedure used a vendor to dial the household automatically to eliminate many nonworking numbers. If a tritone signal was detected, then the telephone was classified as a nonworking number and was never dialed by Westat interviewer. Both steps were used in all three rounds of NSAF data collection. The procedures were enhanced by the vendor 3-1

22 over time, and a larger percentage of numbers was classified as nonresidential or business in Round 3 compared to earlier rounds. When the vendor s new procedures were introduced, Westat examined the accuracy of the results by comparing the outcomes from the vendor s process with those from a recently conducted survey (the 2001 National Household Education Survey). Based on this study, Westat obtains the results of the purging from the vendor and specifies which cases will not be dialed so that less than 1 percent of the purged numbers are residential. The sample was selected all at once, when the newest quarterly update of the sampling frame was available in December The sample was then assigned to waves for data collection as described in section 3.5. Consideration was given to sampling for later releases from the next quarterly frame update that was expected by March However, the entire sample was selected at once because most of the sample was planned to be in process before a sample from the new frame could be prepared. Furthermore, the differences in the frames by quarter are typically very small. Because of the magnitude of the number of telephone numbers in the sample, the purging of the sample of telephone numbers was done in five batches, each containing about 100,000 telephone numbers. The dates the batches were purged were January 7 21, January 14 28, January 22 February 5, January 28 February 11, and February The order of the batches corresponded to the waves of releasing the numbers. Westat interviewers dialed any telephone number not eliminated by the tritone and yellow page purges to determine its working residential status. When a residential telephone number was reached, the interviewer asked about the age composition and income of the household. These questions were used to subsample households, as discussed in the next section. 3.2 Subsampling Households As in Rounds 1 and 2, households were subsampled for the NSAF interview using subsampling rates that depended on whether there were children in the household and whether the household income was below 200 percent of the poverty threshold. Specifically, households with children under 18 and incomes below 200 percent of the poverty threshold were subsampled at 100 percent (in other words, they were retained with certainty). Households with all members 65 and older were not subsampled (none were retained). All other households were retained for the interview with subsampling rates greater than zero and less than one. The rationale and procedures for the subsampling are the same as in previous rounds. Essentially, the subsampling rates were developed to obtain the desired sample sizes and effective sample sizes for the targeted subgroups needed for analysis. The rationale is discussed in more detail in the 1997 and 1999 sample design reports. Two distinct subsampling steps were implemented. The first subsampled households with no children. The second subsampled households with and without children if the reported income was above 200 percent of the poverty threshold or if the response to income screener question was missing. Thus, four strata with different subsampling fractions were created, as illustrated in figure 3-1. The two subsampling steps are discussed in more detail in the next sections. 3-2

23 Figure 3-1. Household Subsampling Operations Children present? Yes Keep No Low income? Yes Keep No Subsample Yes Anyone under age 65? No Subsample Low income? Yes Keep No Drop Subsample 3-3

24 The subsampling rates were computed based on a variety of assumptions. Most assumptions were computed using data from Round 2. The key assumptions used to compute the subsampling rates are as follows: Type of household Table 3-1 gives the proportion of child, adult, and elderly households estimated from Round 2. Low-income rates Table 3-1 also gives the estimated proportion of households that are poor (less than 200 percent of the poverty threshold), nonpoor, and unknown, for both child and adult households. These proportions are the expected rates for households that respond to the screener income item. The Round 2 rates were used for Round 3 despite the recent economic downturn to be conservative. Misclassification by income The rates of households being misclassified by income are shown in table 3-2. The false negative rate is the proportion of households that report high income when they are actually low income. The false positive rate is the proportion of households that report low income when they are actually high-income. The proportions in the table are the observed (weighted) rates from Round 2 and are discussed more fully later in the chapter. Residential rates The assumed residency rates are given in the first column of table 3-3. The rates are three points less than the residency rates estimated from Round 2. Lower rates were used because the residency rate has been decreasing rather steeply during this time period. The ring no answer (NA) and answering machine (NM) cases are treated as nonresidential for this purpose since they do not result in completed interviews. Response rates The assumed response rates are shown in the remaining columns of table 3-3. The screener response rates are 2 percentage points less than the estimated response rate from Round 2, again without any allocation for NA and NM cases. We also assumed the adult and child extended response rates would be 2 percentage points less than the observed Round 2 rates. Once again, the lower rates were assumed because response rates to most RDD surveys have been decreasing in recent years and the Round 2 rates were some of the highest response rates we have seen in RDD surveys in the past three years. The rates for subsampling adult-only households and households by income were derived using these assumptions. These subsampling or retention rates are shown in table 3-4. The required number of completed number of screeners and sampled telephone numbers was then derived from these assumptions. 3-4

25 Table 3-1. Assumed Proportion of Households, by Household Type and Poverty Status 3-5 Household Type Child Households Screening Adult-Only Households Screening Study Area Child Adult Elderly Poor Nonpoor Unknown Poor Nonpoor Unknown Alabama California Colorado Florida Massachusetts Michigan Minnesota Mississippi New Jersey New York Texas Washington Wisconsin Bal. of nation Total

26 Table 3-2. Assumed Misclassification Rates, by Income Categories Child Households Adult-Only Households Study Area False Positive False Negative Unknown to Low-Income False Positive False Negative Unknown to Low-Income Alabama California Colorado Florida Massachusetts Michigan Minnesota Mississippi New Jersey New York Texas Washington Wisconsin Bal. of nation Total Table 3-3. Assumed Residential and Response Rates Screener Response Rate (%) Adult Extended Response Rate (%) Residency Rate Study Area (%) Alabama California Colorado Florida Massachusetts Michigan Minnesota Mississippi New Jersey New York Texas Washington Wisconsin Bal. of nation Total Child Extended Response Rate (%) 3-6

27 Table 3-4. Subsampling or Household Retention Rates Child Households Adult-Only Households Study Area Above Poverty Unknown Income Adult Above Poverty Unknown Income Alabama California Colorado Florida Massachusetts Michigan Minnesota Mississippi New Jersey New York Texas Washington Wisconsin Bal. of nation Subsampling Adult-Only Households Since one goal of the survey was to estimate characteristics of all adults under age 65 and of the subset of these adults who lived in households without children, the same procedure for subsampling adult-only households used in the previous rounds was also used in Round 3. Together with the subsampling of persons within households with children, these subsampling procedures provide a sufficient sample size for making reliable estimates for these adults. As in previous rounds, the MKAs and their spouse/partners are called Option A adults. Other adults are called Option B adults. Option B adults who live in households with children are called Option B stragglers. Statistics about nonelderly adults are formed using data from the Option A interviews about the MKAs and their spouses and using data from the Option B interviews about all other adults under age The group of all other adults under age 65 consists of nonelderly adults in adult-only households and some nonelderly adults in households with children. In households with children, an adult was eligible for sampling if the adult did not have any children under age 18 living in the household and if the adult had not already been identified as the MKA for a focal child or as the spouse/partner of an MKA. 1 Option A interviews were administered to the MKA about the focal child(ren). It also obtained income, earnings, health insurance, and other information about the MKA and his/her spouse/partner. The Option B interview obtained the same information about the sample adult and his/her spouse/partner as obtained by the Option A interview only questions about children were missing from the Option B version of the questionnaire. 3-7

28 The target sample sizes for adults were set for the combined group of Option B adults and Option B stragglers. The targets for each study area were approximately equal to the observed sample sizes in Round 2. Since the Round 3 sample had a much smaller sample of nontelephone households in Round 3 compared with previous rounds, the target sample sizes were close to the number of telephone interviews from Round 2. The retention rates for adult-only households in the RDD sample are shown in table 3-4. The retention rates ranged from 53 percent to 88 percent across the study areas. These rates are generally higher than they were in earlier rounds in order to reduce the variability in the estimates due to subsampling adults at differential rates. The research supporting this approach is described in 1999 NSAF Sample Design, Report No. 2. The subsampling was implemented by loading the retention rate table into the computer-assisted telephone interviewing (CATI) system. Each telephone number was randomly assigned as either a child-only household or a child and adult household using a random sampling procedure before the case was loaded. Households subsampled as child and adult were classified as adult-only households if there were no children under age 18 present but at least one person was under the age Subsampling High-Income Households The same procedure used in the previous rounds was used in Round 3 to subsample households by income. In households with children and in subsampled adult-only households, all households were asked a simple question on household income. Those households reporting income above 200 percent of the poverty threshold were subsampled. The income levels for determining 200 percent of poverty are shown in table 3-5. The subsampling rates are given in table 3-4 for both types of households. Table 3-5. Income Levels for Determining Less than 200 Percent of the Poverty Level Household Size by Person Without Children With Children 1 $18,300 $18,300 a 2 $23,500 $24,200 3 $27,500 $28,300 4 $36,200 $35,800 5 $43,700 $42,400 6 $50,200 $47,900 7 $57,800 $54,500 8 $64,600 $60, $77,700 $71,400 a. This type of household can occur only if an emancipated minor is living alone. 3-8