Assessing the impact of 4Ps on school participation of Filipino children using Propensity Score Matching (PSM)

Size: px

Start display at page:

Download "Assessing the impact of 4Ps on school participation of Filipino children using Propensity Score Matching (PSM)"

Jocelin Bridges
6 years ago
Views:

Mina Making Impact Evaluation Matter: Better

1 Assessing the impact of 4Ps on school participation of Filipino children using Propensity Score Matching (PSM) Celia M. Reyes and Christian D. Mina Making Impact Evaluation Matter: Better Evidence for Effective Policies and Programs Asian Development Bank, Manila 5 September 2014

2 Average daily wage rate of wage/salary workers by educational attainment, 2011 Php 1,200 1,137 1, Source of basic data: July 2011 LFS

3 Data 2011 Annual Poverty Indicators Survey (APIS) and July 2011 Labor Force Survey (LFS) - for the estimation of probability of participating in the 4Ps and for matching 2003 and 2006 Family Income and Expenditure Surveys (FIES), merged with January 2004 and January 2007 LFS, respectively - for the estimation of probability of being selected in the 4Ps

4 Data Data set Advantage(s) Disadvantages 2003 FIES/LFS 2006 FIES/LFS used by DSWD in the estimation of the PMT model most appropriate to provide the pre-program characteristics can only estimate the probability of being selected in the program predicted poor 4Ps the patterns found in 2003 might no longer hold in 2008 can only estimate the probability of being selected in the program predicted poor 4Ps 2011 APIS/FS can estimate directly the probability of program participation can easily satisfy the balancing tests characteristics of a large proportion 4Ps families that have been in the program in 2008 or 2009 might have been changed significantly

5 Estimated Logit Models Variable PS Model FIES Parameter PS Model FIES Dependent variable: Poverty status Independent variables: Household head profile Sex (0.0539) (0.0243) Age (0.0082)* (0.0043)** Square of age (0.0001)* (0.0000)** Household composition Natural logarithm of family size (0.0551)*** (0.0267)*** Proportion of members aged (0.0015)* (0.0008) Proportion of members aged (0.0017)* (0.0010)*** Proportion of members aged 65 & above (0.0020) (0.0011)*** Education of members Proportion of members who are currently attending school (0.0000)*** (0.0003)*** Proportion of members who are elementary undergraduate (0.0011)*** (0.0006)*** Proportion of members who are elementary graduate (0.0012)*** (0.0007)*** Proportion of members who are high school undergraduate (0.0013)*** (0.0007)*** Proportion of members who are elementary undergraduate (0.0015)*** (0.0008)*** Proportion of members who are elementary undergraduate (0.0020)*** (0.0010)*** Proportion of members who are elementary undergraduate (0.0037)*** (0.0016)*** Proportion of members who are elementary undergraduate (0.0231)* (0.0245)*** Employment of members Proportion of employed members (0.0019)*** (0.0010)*** Proportion of laborers & unskilled workers (0.0013)*** (0.0006)*** Proportion of members who are agricultural workers (0.0014)*** (0.0009)***

6 Estimated Logit Models Variable Parameter PS Model FIES PS Model FIES Proportion of members who are wage/salary workers (0.0015)*** (0.0007)*** Proportion of members who are employers (0.0030)*** (0.0016)*** Proportion of members who are self-employed (0.0018) (0.0010)*** Proportion of members who are permanent workers (0.0010)*** (0.0005)*** Proportion of members who are overseas contract workers (0.0094)*** (0.0032)*** Housing characteristics and tenure Single house (0.0923)** (0.0408)*** House constructed by strong/predominantly strong materials (0.1147) (0.0486)*** House/lot owned/rented with owner's consent (0.0814)** (0.0376)*** Access to basic amenities Safe water (0.0382)* (0.0184)*** Sanitary toilet facility (0.0392)*** (0.0202)*** Electricity (0.0436)*** (0.0226)*** Ownership of asset Television set (0.0460)*** (0.0213)*** VTR/CD/DVD player (0.0571)*** (0.0195)*** Refrigerator (0.0629)*** (0.0226)*** Washing machine (0.0785)*** (0.0272)*** Airconditioner (0.1821)*** (0.0758)*** Car/motor vehicle (0.1030)*** (0.0306)*** Telephone/cellular phone (0.0762)*** (0.0180)*** Computer (0.4142)*** (0.1118)*** Microwave oven (0.4765)*** (0.1421)***

7 Estimated Logit Models Variable Parameter PS Model FIES PS Model FIES Location Region (base category: Region XIII (NCR)) Region I (Ilocos Region) (0.1137)*** (0.0429)*** Region II (Cagayan Valley) (0.1241) (0.0482) Region III (Central Luzon) (0.1121)*** (0.0410)*** Region IV-A (CALABARZON) (0.1078)*** (0.0394)*** Region IV-B (MIMAROPA) (0.1167)*** (0.0469)*** Region V (Bicol Region) (0.1123)*** (0.0435)*** Region VI (Western Visayas) (0.1108)*** (0.0420)*** Region VII (Central Visayas) (0.1124) (0.0417)*** Region VIII (Eastern Visayas) (0.1162)*** (0.0457)*** Region IX (Zamboanga Peninsula) (0.1204)*** (0.0479)*** Region X (Northern Mindanao) (0.1164)*** (0.0480)*** Region XI (Davao Region) (0.1171)*** (0.0454)*** Region XII (SOCCKSARGEN) (0.1170)*** (0.0480)*** Region XV (ARMM) (0.1193)*** (0.0471)*** Region XIV (CAR) (0.1281)*** (0.0509)*** Region XVI (Caraga) (0.1180)*** (0.0462)*** Urban/rural (0.0400) (0.0177)*** Constant (0.2546)*** (0.1217)*** *** significant at 1%; ** significant at 5%; * significant at 10% Source of basic data: Merged files of 2003 FIES & January 2004 LFS; Merged files of 2006 FIES & January 2007 LFS, NSO

8 Propensity score (PS) models PS Model 2003 FIES (Y = poverty status) PS Model 2006 FIES (Y = poverty status) PS Model 2011 APIS (Y = 4Ps) Matching methods Nearest neighbor (one-to-one), without replacement Nearest neighbor (one-to-one), with replacement Nearest neighbor (one-to-two), with replacement Radius (caliper = 0.01) Kernel, normal, bandwidth = 0.01 Kernel, epanechnikov Local linear regression, normal, bandwidth = 0.01 Local linear regression, epanechnikov

9 Diagnostic checking Common support - ensured through imposition of common support through the Stata syntax psmatch2, common Propensity Score Untreated Treated

10 Diagnostic checking Balancing property - checked through the following criteria: 1. % reduction in bias (after matching) in: o o propensity scores: 2003 FIES: % 2006 FIES: % 2011 APIS: % all covariates; 2003 FIES: 68-77% 2006 FIES: 68-75% 2011 APIS: 91-96%

11 Diagnostic checking Balancing property - checked through the following criteria: 2. Two-sample t-tests of means in the covariates (between treatment and comparison) > 50% are of all the covariates are not significant 3. % reduction in Pseudo R-square after matching reduced by more than 50%

12 Diagnostic checking Conditional independence - checked through sensitivity tests of hidden bias through Mantel-Haenszel bounds

13 Example: PS Model APIS; Age 6; 1-to-1 matching without replacement Mantel-Haenszel (1959) bounds for variable prop_mem6_attsch Gamma Q_mh+ Q_mh- p_mh+ p_mh e-06 Gamma : odds of differential assignment due to unobserved factors Q_mh+ : Mantel-Haenszel statistic (assumption: overestimation of treatment effect) Q_mh- : Mantel-Haenszel statistic (assumption: underestimation of treatment effect) p_mh+ : significance level (assumption: overestimation of treatment effect) p_mh- : significance level (assumption: underestimation of treatment effect)

14 Mean comparison tests By matching method - Estimated treatment effects are not significantly different across matching method for all age groups By PS model - Estimated treatment effects significantly vary across PS models

15 Best matching estimators, by age group and data set used Age Group 2003 FIES 2006 FIES 2011 APIS 6-14 Nearest neighbor (1), wr Nearest neighbor (1), wr Kernel, normal, bw= LLR, normal, bw=0.01 LLR, normal, bw=0.01 Radius (cal=0.01) 7 Nearest neighbor (2), wr LLR, normal, bw=0.01 Radius (cal=0.01) 8 Kernel, epanechnikov LLR, normal, bw=0.01 Kernel, epanechnikov 9 Radius (cal=0.01) Kernel, epanechnikov Nearest neighbor (1), wor 10 LLR, epanechnikov Radius (cal=0.01) Radius (cal=0.01) 11 Nearest neighbor (1), wor Radius (cal=0.01) Radius (cal=0.01) 12 Nearest neighbor (2), wr Kernel, normal, bw=0.01 Kernel, normal, bw= Kernel, normal, bw=0.01 Radius (cal=0.01) Radius (cal=0.01) 14 Nearest neighbor (1), wor Nearest neighbor (1), wr Radius (cal=0.01) Nearest neighbor (2), wr Radius (cal=0.01) Kernel, normal, bw= Radius (cal=0.01) Nearest neighbor (1), wor Kernel, normal, bw= Nearest neighbor (1), wor Nearest neighbor (1), wor Radius (cal=0.01) 17 Nearest neighbor (1), wor Nearest neighbor (2), wr LLR, normal, bw= Kernel, normal, bw=0.01 Kernel, epanechnikov Radius (cal=0.01)

16 Average treatment effect on the treated (ATT) estimates, by age group and by PS model ATT estimate 10 8 PS Model FIES PS Model FIES PS Model APIS Age

17 Average treatment effect on the treated (ATT) estimates~, by age group and by PS model Age group PS Model FIES PS Model FIES PS Model APIS * 4.6 * 3.0 * * 7.4 * 5.3 * * 3.1 * 1.5 ^ * 1.9 * 1.1 ^ * 2.8 * 2.3 * * 2.7 * 1.4 * * 2.4 * 1.9 * * 3.4 * 2.0 * * 3.6 * 3.1 * * 7.5 * 7.6 * * 5.5 * ~ Results of the best matching estimators; * significant at 5%; ^ significant at 10%

School participation rate of children aged 6-18, 2007 and 2011 % 100 97.3 92.

6 90.4 86.8 82.2 85.2 2007 2011 70 60 50 40 70.1 67.0 57.3 54.4 47.0 44.

18 School participation rate of children aged 6-18, 2007 and 2011 % Age Sources of basic data: 2007 & 2011 APIS

19 Proportion of children in 4Ps families who are attending school, by age group % Age Source of basic data: APIS 2011, NSO

20 Reasons for not attending school among children (aged 6-18) in 4Ps families Lack of personal interest High cost of education Employment/looking for work Illness/Disability Marriage Housekeeping Cannot cope with school work School are very far Others No regular transportation Problem with school record Problem with birth certificate Too young to go to school No school within the barangay Finished schooling Source of basic data: APIS 2011, NSO Ps Non-4Ps

21 Proportion of children in matched 4Ps families who are attending school and/or working, by single year of age, 2011 % Age out-of-school, working studying, not working Source of basic data: Matched files of 2011 APIS and July 2011 LFS

22 Recommendations (in 2012) Extend the period of assistance to existing children-beneficiaries to go beyond 5 years Cover families with children aged 15-18

23 Policy influence In the President s State of the Nation Address in July 2013, the President announced that the 4Ps will be extended to high school to cover children up to 18 years old or up to when they finish high school whichever comes first On June 2014, the DSWD started with the Modified Conditional Cash Transfer (MCCT)- Expanded Age Coverage (EAC) of the 4Ps The MCCT-EAC covers beneficiaries which are not covered by the regular CCT. As of 31 July 2014, there are 888,350 households with children and 924,066 children aged included in the program.

BNRS IN FIGURES. Monthly Report as of April 2016

BNRS IN FIGURES. Monthly Report as of April 2016 BNRS IN FIGURES Monthly Report as of SCOPE Business Name (BN) Registration by Scope per Source VS April 2015 2016 2015 % Change NEW TELLER RENEWAL NEW WEB RENEWAL NEW Grand NEW TELLER RENEWAL NEW WEB RENEWAL