Effects of Poststratification and Raking Adjustments on Precision of MEPS Estimates Sadeq R. Chowdhury

Transcription

1 Effects of Poststratification and Raking Adjustments on Precision of MEPS Estimates Sadeq R. Chowdhury The opinions expressed in this presentation are those of the authors and do not reflect the official position

2 Overview Introduction to Medical Expenditure Panel Survey (MEPS) Weighting in MEPS Variance Estimation in MEPS Assessing Effects of PS/Raking Methodology Results Conclusion

3 MEPS - Introduction

4 MEPS Household Component Conducted by AHRQ since 1996 Collects data on health status, insurance coverage, healthcare use, expenditures, sources of payments, etc. Panel design - a new panel selected every year - previous year s NHIS respondents - each panel is followed for 2 years - two panels in each full year (FY) file

5 MEPS Household Component (Cont.) Five in-person interview rounds to cover 2-year period About 14,000 HHs and 30,000 persons in each FY file. Roughly half in each panel.

6 MEPS Overlapping Panel Design Panel 13 R1 R2 R3 R4 R5 Panel 14 R1 R2 R3 R4 R5

7 Weighting in MEPS

8 New Panel (Year 1) HH Base Weight: MEPS Year 1 Weighting NHIS Final HH Weight*(1/MEPS Selection Prob) Initial Poststratification (PS) using NHIS estimates Round 1 nonresponse adjustment Person Base Weight: HH=>Family=>Person PS Adjustment using CPS estimates Round 1 to end of Year 1 NR adjustment Year 1 Raking using CPS estimates

9 MEPS Year 2 Weighting Old Panel (Year 2) Year 2 NR adjustment Year 2 Raking adjustment using CPS Combined Panels Final Raking adjustment using CPS

10 Figure 3: Person-Level PIT and FY Weight Development Process MEPS Weighting Scheme FY Weighting Panel X Panel X-1 HH BASE WT FIN PER WT (Y1) PS: HH WT NR: HH WT PS: R1 FAM WT PS : R1 PER WT NR: PER WT (Y1) RK: FIN PER WT (Y1) RK: Combined PRLM FY WT NR: PER WT (Y2)* RK: PER WT (Y2)* POV_RK: FIN FY WT

11 MEPS Raking Variables Variables Used in PS/Raking Race/Ethncty: Hispanic, NH Black, Asian, Other Sex: Male, Female Region: Northeast, Midwest, South, West MSA: MSA/Non-MSA AgeCat: <1, 1-19, 20-29, 30-44, 45-64, 65+ Poverty Status: Below Poverty, %, %, %, 400+% Control totals obtained from CPS

12 Variance Estimation

13 Variance Estimation for Complex Surveys

14 Variance Estimation in MEPS TSE is commonly used VARSTR and VARPSU are provided in PUF Replicate Weights are not provided in PUF A BRR replication structure (in form of a set of half sample indicators) is provided. Users can compute single- step shortcut BRR or Fay s BRR replicate weights Not all weighting adjustments are applied to each replicate separately

15 PS/Raking and Variance PS/Raking mainly performed for reducing NR/coverage bias But PS/Raking using known control totals also reduces variance Reduction in variance due to PS/Raking generally not captured by TSE method One-step shortcut BRR Variance generally goes up due to additional adjustments Proper BRR approach should account for the impact of PS/raking adjustments on variances

16 Methodology

17 Measuring Effect of PS/Raking Difference in estimates of variances between TSE and proper BRR is loosely considered as effects of PS/Raking on variance Differences are considered at different stages of weighting

18 Variance Estimation for Comparison Variance Estimates produced using TSE and Fay s BRR method For TSE, usual VATSTR and VARPSU were used For BRR, 128 replicates formed Fay s BRR Replicate weights were computed starting with base weight and applying all subsequent adjustments to each replicate Replicate weights computed at each step were saved to facilitate comparison at different stages of weighting

19 Variance Compared Variances Estimates (RSE%) under both methods are computed and compared using weights at the following stages DU Base Weight (DUWT1) Round 1 DU Final weight (DUWTF) Round 1 Final Person weight (PNWT) Panel Specific Final FY Weight (FYWT) Combined Panel final FY Weight (FYWTF)

20 Data Used 2008 MEPS FY File - Panels 12 & 13 Cases in-scope on 12/31/08 Full year respondents

21 Effectiveness of PS/Raking Adjustment For both bias and variance reduction, PS/Raking variables must be correlated with target variables. Raking/PS model must have reasonable explanatory power for a target variable To assess, models are fitted with selected target variables as dependent variables and raking dimensions as independent variables

22 Explanatory Power of Raking Model Table 1. Explanatory power (R 2 ) of the final raking model for different target variables Categorical Target Variables Continuous Target Variables Dependent Variable R 2 Dependent Variable R 2 Insurance Status 12.8% Total Expense 8.3% 5+ Office-based Provder Visits 15.2% Office-based Expense 6.4% 1+ Outpatient Visits 8.6% Prescription Expense 10.8% 1+ Inpatient Stays 4.0% Out-of- pocket Expense 9.2% 1+ ER Visits 2.3% Outpatient Expense 1.9% 5+ RX - Prescription 25.3% Inpatient Expense 2.4% Daily Activity Limitation 9.3% Emergcy Room Expense 2.3% Any Limitation 20.3% Wage Income 50.8% Poor Health 8.7% Dental care expense 3.4% Poor Mental Health 3.8% Unable to Get Healthcare 4.5%

23 Results

24 Comparison of RSEs (TSE vs. BRR) For Proportions

25 Comparison of RSEs (TSE vs. BRR) For Proportions

26 Comparison of RSEs of Means (Effective Raking Models)

27 RSEs of Estimates of Means (Ineffective Raking Model)

28 Conclusion Variances under both methods are similar at the initial stage With various adjustments, generally variances under TSE increase but variances under BRR decrease Variances under BRR are generally 5-10% lower but in some cases 20-25% lower. Extent of difference in variance between BRR and TSE depends on the explanatory power of the raking model

29 Conclusion BRR captures the reduction in variance for adjustment with known control totals but TSE does not TSE (frequently used for MEPS) overestimates variances of most MEPS estimates Imputation variance and variances in control totals are not captured in either method

30 Conclusion For estimates not subject to imputation, TSE is overestimating variances For estimates with imputation, overestimation in TSE can be considered as compensating for imputation variance Further research needed to see the impact of imputation variance on expenditure estimates

31 Thank You