The Impact of Tracing Variation on Response Rates within Panel Studies

Transcription

1 The Impact of Tracing Variation on Response Rates within Panel Studies Christine Carr Jennifer Wallin Kathleen Considine Azot Derecho Sarah Harris Barbara Bibb RTI International is a trade name of Research Triangle Institute AAPOR 2008 Conference May 16,

2 Introduction Locating and contacting survey participants is one of the biggest challenges for any longitudinal study. In order to achieve high response rates, panel studies should think about the following topics: Locating methods for maintaining contact with respondents over a number of years Respondent cooperation Budget Constraints 2

3 Objectives Describe measures taken to locate renters over a 5- year period Discuss the level of effort to locate and interview sample members Discuss the costs associated with each tracing method used 3

4 Past Studies Show. Study Sample Demographic Sample Size Time between Baseline and Follow-up % Located after Batch Tracing % Located after Interactive Tracing Tracing Methods Used to Locate Nonrespondents (in order of Intensity) Andresen et al. (2008) Workers' Compensation claimants from St Louis and Missouri 3170 N/A (study used tracing to contact respondent s from a database) 49% 68% Basic (National Change of Address, TeleMatch), Basic Hands-On (Internet White Pages, Phone Disc, Calls to contacts collected at baseline), Advanced Hands-On (Credit bureaus, public records databases) Weinberger et al. (2002) Participants of clinical study of blood pressure 708 between yrs 51% 84% Hospital and study audits and records, mailing of follow-up letters, Internet searches, Death Registries, Commercial Person Locater Morrison et al. (1997) Adolescent orthodontic patients 16,915 2 years 75% 92.5% Study and clinician records, directory assistance, follow-up letters via Certified mail, CD ROM and reverse phone 4 directories, additional incentives.

5 Study Background Study Design Longitudinal Panel Study of Low to Moderate Income Renters 5-Year Study Control group for an ongoing study of Low to Moderate Income Homeowners Purpose Assess how being a renter affects families socially and economically and determine how renters differ from homeowners who live in the same areas. Capture changes in homeownership status, wealth and assets, living situations, and household composition 5

6 Sample Selection Screened sampled telephone numbers and identify sample members based on a certain set of criteria. Recruited low to moderate income renters that lived in the same neighborhood as the homeowner panel Located in 30 Metropolitan Statistical Areas (MSAs) Selected renters that had annual incomes less than the threshold which was based on the Area Median Income for the homeowners MSAs 6

7 Sample Design Eligibility Criteria Between the ages of years of age. Household income was less than or equal to 80 percent of the area median income Renter was not a Full-Time student Paid cash rent to the owner of the unit Name on the rental lease agreement 7

8 Data Collection YEAR MODE YEAR OF DATA COLLECTION SAMPLE YEAR 1 TELEPHONE 2004 RDD- QUOTA YEAR 2 FIELD 2005 LIST YEAR 3 TELEPHONE 2006 LIST YEAR 4 TELEPHONE 2007 LIST YEAR 5 FIELD 2008 LIST 8

9 Methods to Prevent Respondent Attrition The literature on respondent tracking provides several steps that should be taken for panel maintenance. Baseline Tracking Activities Obtain several sets of contact information during baseline (i.e., friends and relatives) Collect Social Security Number and Date of Birth Post Baseline Tracking Activities Locator Mailing Develop Tracking System Conducted for Renters? v-collected name and phone number of someone not living with the renter v-dob is collected every year and SSN was collected during the In-Person Interview Conducted for Renters? v-locator mailing conducted between each wave of data collection. v -Control System developed to monitor the number of undeliverable letters Provide Incentives v -Incentives are provided for each year of participation. 9

10 Locating Sample Members Type of Tracing Description Advantages Disadvantages Batch Tracing Batch vendors used to confirm or update address and phone information Least expensive Quick Can return incorrect information Interactive Tracing Specialized staff trained for tracing procedures, resources, and investigative techniques Dedicated staff trained to used various resources to find hard to locate cases Success depends on level of effort allotted. Field Tracing Field interviewers used to locate sample members in person. Able to locate sample members who no longer have land lines or use call screening devices Costly (i.e., field staff labor hours, mileage, training costs) 10

11 Baseline Demographics Age, mean, years Female, No. (%) Race, No. (%) White Black American Indian Asian or Pacific Islander Other Hispanic (of any race), No. (%) (71.5) 753 (46.1) 509 (31.2) 104 (6.4) 25 (1.5) 243 (14.9) 300 (18.2) 11

12 Baseline Demographics Education, No. (%) High school or less Some college Bachelor s degree (BA, AB, BS) More than a bachelor s degree Other 813 (49.4) 527 (32.0) 206 (12.5) 82 (5.0) 18 (1.1) 12

13 Baseline Demographics Marital Status, No. (%) Married or living with a partner Widowed Divorced Separated Never been married Other 1093 (66.3) 32 (1.9) 165 (10.0) 76 (4.6) 259 (15.7) 23 (1.4) 13

14 Baseline Demographics Currently working for pay, No. (%) Household income in 2002, mean, $ No. of household members, mean No. of children in household, mean 969 (58.9) 20,

15 Response Rates by Tracing Method Used and Year of Data Collection No. Sent to tracing No. Located % Located No. Completed Interviews % Completed Interviews Among Located % Completed Cumulative % Completed YEAR 2 Batch Interactive YEAR 3 Batch Interactive

16 YEAR 4 Response Rates by Tracing Method Used and Year of Data Collection No. sent to tracing No. located % located No. completed interviews % completed interviews among located % completed Cumulative % completed Batch Interactive Field

17 Response Rates by Cost per Case by Tracing Method Used: Year 4 Tracing method Total Cost (a) No. Sent (b) Cost per case (a)/(b) No. completed (c) Cost per case (a)/(c) Batch $1, ,217 $ $1.97 Interactive $8, $ $85.33 Field Tracing $11, $92 50 $

18 Summary Conducting various levels of tracing paid off for our renter panel. Adding the field tracing component increased our response rate in Year 4 to 74%. Having SSN for a respondent and blending multiple sources of information helped our study locate more respondents through batch tracing and at a lower cost than interactive tracing. The percentage of completed interviews among located cases was higher among those field traced than those respondents located through interactive tracing in Year 4. However, adding additional tracing such as field tracing, comes at a cost. 18

19 Implications In this study, field tracing costs DID benefit the study by contributing to the sample size and increasing the response rate. However, tracing efforts will vary by population. What works for one sample may not pay off with another population. Factors to Consider: Sample (demographics) Response rate requirements Budget restrictions 19

20 Future Research Does the non-response sample differ from the original sample? How do demographic characteristics differ based on methods used? 20