REPUBLIC OF SOUTH AFRICA GOVERNMENT-WIDE MONITORING & IMPACT EVALUATION SEMINAR Session III The Regression Discontinuity Design (RD) Sebastian Martinez June 2006 Slides by Sebastian Galiani, Paul Gertler and Sebastian Martinez ORGANIZED BY THE WORLD BANK AFRICA IMPACT EVALUATION INITIATIVE IN COLLABORATION WITH HUMAN DEVELOPMENT NETWORK AND WORLD BANK INSTITUTE
Regression Discontinuity Assignment to treatment is based on a clearly defined index or parameter with a known cutoff for eligibility RD is possible when units can be ordered along a quantifiable dimension which is systematically related to the assignment of treatment The effect is measured at the discontinuity estimated impact around the cutoff may not generalize to entire population
Indexes are common in targeting of social programs Anti-poverty programs targeted to households below a given poverty index Pension programs targeted to population above a certain age Scholarships targeted to students with high scores on standardized test CDD Programs awarded to NGOs that achieve highest scores
Example: effect of cash transfer on consumption Target transfer to poorest households Construct poverty index from 1 to 100 with pre-intervention characteristics Households with a score <=50 are poor Households with a score >50 are nonpoor Cash transfer to poor households Measure outcomes (i.e. consumption) before and after transfer
Regression Discontinuity Design - Baseline Outcome 60 65 70 75 80 20 30 40 50 60 70 80 Score
Regression Discontinuity Design - Baseline Outcome 60 65 70 75 80 Poor Non-Poor 20 30 40 50 60 70 80 Score
Regression Discontinuity Design - Post Intervention Outcome 65 70 75 80 20 30 40 50 60 70 80 Score
Regression Discontinuity Design - Post Intervention Outcome 65 70 75 80 Treatment Effect 20 30 40 50 60 70 80 Score
Identification yi = β0 + β1 Treatmenti + δ( score) + εi Where Treatment = 1 if score <=50 Treatment = 0 if score >50 Assumption: δ ( score) is continuous around the cutoff
Sharp and Fuzzy Discontinuity Sharp discontinuity discontinuity precisely determines treatment equivalent to random assignment in a neighborhood Example: Social assistance payments assigned strictly on age Fuzzy discontinuity discontinuity is highly correlated with treatment use the index as an IV Example: Rule for class size that is not strictly enforced
Examples Effect of transfers on labor supply (Lemieux and Milligan, 2005) Effect of old age pensions on consumption (Martinez, 2005) BONOSOL in Bolivia Effect of class size on scholastic achievement (Angrist and Lavy, 1999)
Lemieux & Milligan - Incentive Effects of Social Assistance Social assistance to the unemployed: Low social assistance payments to individuals under 30 Higher payments for individuals 30 and over What is the effect of increased social assistance on employment?
Insert Slide Title
Insert Slide Title
BONOSOL Old age pension to all Bolivians 65 years and older Pension transfer to large group of poor households Pre- and post- data (1999-2002), pensions paid as of 2001 Known eligibility criteria: 65+ Estimate effect of BONOSOL on consumption
Figure 1.2b: Rural Consumption Per Capita - Fan regression Consumption Per Capita 100 120 140 160 180 200 220 240 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 Age of Oldest HH Member Treatment Year Non-Treatment Year
Angrist & Lavy: Using Maimonides Rule Effect of class size on learning outcomes i.e. test scores in third and fourth grades Use Maimonides rule class size of 40 or less one class class size over 40 split class in two
Angrist & Lavy: Using Maimonides Rule Maimonides rule not always implemented exactly classes with 42 students Fuzzy Discontinuity Instrumental Variable Use maimonodes rule to estimate class size Use these estimates to relate the size of the class to test scores
Potential Disadvantages of RD Local treatment effects not always generalizable Power: effect is estimated at the discontinuity, so we generally have fewer observations than in a randomized experiment with the same sample size Specification can be sensitive to functional form: make sure the relationship between the assignment variable and the outcome variable is correctly modeled, including: Nonlinear Relationships Interactions
Advantages of RD for Evaluation RD yields an unbiased estimate of treatment effect at the discontinuity Can many times take advantage of a known rule for assigning the benefit that are common in the designs of social policy No need to exclude a group of eligible households/individuals from treatment
References Angrist, J. and V. Lavy Using Maimonodes Rule to Estimate the Effect of Class Size on Scholastic Achievement Quarterly Journal of Economics, 114, 533-575 Lemieux, T. and K. Milligan Inentive Effects of Social Assistance: A Regression Discontinuity Approach. NBER working paper 10541. Hahn, J., P. Todd, W. Van der Klaauw. Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design. Econometrica, Vol 69, 201-209.