CCES 2014 Methods and Survey Procedures

Transcription

1 CCES 2014 Methods and Survey Procedures Sundance Conference June 12, 2015

2 2 Simultaneous CCES Studies CCES "Regular" CCES "Panel"

3 CCES Regular 48 Teams N = 56,200 matched 48,853 interviews in post 86.9% reinterview rate

4 CCES Panel 9,500 participants who took both waves of the 2010 CCES, 2012 CCES, and 2014 CCES i.e., Waves 5 and 6 were administered in 2014

5 Sampling Frame American Community Survey, 2010 Gender Age Race Hispanic origin Education level Family income Marital status # Children under 18 Employment status Citizenship State Metropolitan area Weighted stratified sample (with replacement) of 500,000 cases in state age race gender education strata

6 Sampling Frame Current Population Survey, November 2010: Voting and Registration Supplements Voter registration Turnout Stratified match to ACS sample using weighted Euclidean distance metric conditioning on voter registration in state age race gender education strata

7 Sampling Frame Pew Religious Landscape Survey, 2007 Interest in politics Religion Born again / evangelical Church attendance Regional stratified match to ACS/CPS matched sample using weighted Euclidean distance metric Many media surveys Party identification Ideology Target samples for teams selected by SRS within strata of gender age education race

8 Sampling Frame Pew Religious Landscape Survey, 2007 Interest in politics Religion Born again / evangelical Church attendance Regional stratified match to ACS/CPS matched sample using weighted Euclidean distance metric Many media surveys Party identification Ideology Target samples for teams selected by SRS within strata of gender age education race

9 Sampling Frame Pew Religious Landscape Survey, 2007 Interest in politics Religion Born again / evangelical Church attendance Regional stratified match to ACS/CPS matched sample using weighted Euclidean distance metric Many media surveys Party identification Ideology Target samples for teams selected by SRS within strata of gender age education race

10 Within Panel Selection Panel sources YouGov Panel ResearchNow MyPoints SSI GMI usamp Targets: age gender race education state Voter registration status Everyone who completed pre-election interviews was invited to the post-election wave

11 Matching and Weighting Process for CCES Teams and Common Content Teams were matched and weighted individually Target sample was drawn from the sampling frame (typically N=1000) and used to make matched sample Matched sample was weighted to the full sampling frame Matched set of teams was stacked to create the Common Content survey Common content was weighted to the full sampling frame

12 Matching variables Age Race Education Gender Interest in politics Employment status Marital Status Ideology Baseline Party ID Born again Voter Registration Status Took post-election survey

13 Entropy weighting (EBal) Previously CCES used Propensity Score Weighting - repeatedly estimating logit models and predicting propensity score weights EBal generates a set of weights that balances pre-specified moment conditions Starting with a set of base weights (often uniform), EBal adjusts the weights to match moment distributions in target and frame while keeping the weights as close as possible to the starting weights Goals: fit the target distributions, maximize efficiency Benefits: "Easier" because we don t need to guess the proper functional form Another benefit: CCES Common has smaller weights than previously

14 Entropy weighting procedure Matched cases and the frame were combined The combined cases were balanced on multiple moment conditions using entropy balancing Moment conditions: Age, gender, education, race, voter registration, ideology, baseline party ID, born again, political interest, plus interactions Post-stratification by Age, gender, education, race, voter registration, as needed Post-stratification across states and races as needed in Common

15 Good news Overall sample quality was the best it has ever been All teams were able to get 80%+ reinterview rates due to extent of overcollection (Often 85%+) However - higher breakoff rates within the survey than in 2012

16 Distribution of Common Weights

17 Location of Weights Gender Male 1.0 Female 1.0 Age Race White 1.0 Black 1.0 Hispanic 1.0 Other 1.0 Registered to Vote Yes.8 No/DK 2.1 Education Less than HS 3.5 High school 1.1 Some college 1.0 College grad 0.7 Post grad 0.8 Age x Gender Male Female 1.0 Race x Education Non-white Post grad 0.8 Non-white HS or less 2.1 Continuing... Non-white, unregistered, 5.3 lt HS, male

18 Demographic characteristics of CCES Common Citizen Frame Weighted CCES Common Female 52% 52% % 22% Black 12% 12% Latino 10% 10% Less than HS 12% 9% HS Grad 29% 32% Registered to vote 65% 75%

19 Senate Vote in the CCES Common Content

20 Gubernatorial Vote in the CCES Common Content

21 Demographic characteristics of CCES Panel Citizen Frame Weighted CCES Panel Female 52% 52% % 19% Black 12% 11% Latino 10% 9% HS or less 41% 37% Registered to vote 65% 81%

22 Common and Panel had different reinterview rates in the November post-election wave Common Panel Total Total interviews 87,389 15, ,027 Reinterview rate 70.7% 84.1% 72.7% Matched interviews 56,200 9,500 65,700 Reinterview rate of matched 86.9% (100%)

23 Reinterview rates for CCES Common (matched and unweighted) Group Education Took post Less than HS 76 % High school 86 % Some college 85 % College grad 88 % Post grad 92 % Age % % % % Race White 90 % Black 74 % Hispanic 79 % Other 84 % Gender Male 86 % Female 88 % Registered to Vote Yes 88 % No 78 %

24 Respondent Satisfaction Completion rates for teams ranged from 84% to 100%

25 What do panelists hate?

26 What do panelists hate? With Common included, the overall break-off rate was 22.7% - about 5% worse than 2012 Question Type Percent breakoff Providing address 4.4% Religion 3.7% Approval of officeholders 0.9% Roll call list was shorter this year and had fewer breakoffs than in the past.

27 Team killers Interview length Videos Conjoints Long reading (possibly)

28 LOI and Question type both influenced completion rates

29 LOI and Question type both influenced completion rates

30 CCES 2015 Post-election only 10 minutes Common content, 10 minutes team content $12,500 for a module of 1000 cases Extra cases for $10 cpi Questionnaires due August 1, 2015