EVALUATING INDONESIA S UNCONDITIONAL CASH TRANSFER PROGRAM(S) *

Transcription

1 EVALUATING INDONESIA S UNCONDITIONAL CASH TRANSFER PROGRAM(S) * SUDARNO SUMARTO The SMERU Research Institute * Based on a research report Of safety nets and safety ropes? An Evaluation of Indonesia s compensatory unconditional cash transfer Program, by Samuel Bazzi, Sudarno Sumarto and Asep Suryahadi

2 OUTLINE Introduction The Unconditional Cash Transfer Programs (UCT/BLT): Targeting and Transfer Mechanism Data Empirical Methods Main Results Summary of Findings Policy Implications 2

3 Introduction (1) The price of oil in Indonesia was fixed by the government and highly subsidized. Escalating global oil price in 2005 forced the Government of Indonesia (GoI) to slash fuel subsidies, raising regulated fuel prices by a weighted average of 114% in September, resulted in over USD 10 billion in budgetary savings. Around USD 2 billion was used for a large scale unconditional cash transfer (UCT) program called BLT, providing a quarterly transfer of around USD 30 per recipient household. The BLT program was designed to prevent poor households from having to reduce expenditures on essential commodities, health, and education in the midst of strong inflationary pressure. 3

4 Introduction (2) Delivery of the cash payment is conducted through the post office by direct payment. In terms of the number of beneficiaries, it is the largest cash transfer program in the world, targeting an estimated 19.2 million households or over 75 million individuals. This paper assesses the impact of the BLT on household welfare. Source: Kompas Source: Tempo 4

5 The Unconditional Cash Transfer Programs (UCT/BLT): Targeting & Transfer Mechanism 5

6 Share of National Expenditure on Fuel 6

7 Targeting Criteria and Mechanism The selection to determine the beneficiary households for the BLT program was based on: 1) potential eligibility lists drawn up by local leaders; and 2) the data collected in the 2005 Socioeconomic Survey (PSE05), which was carried out by BPS (Statistics Indonesia). The method used proxy means testing, based on 14 country wide indicators and district specific weights to capture the specific nature of local poverty 7

8 Targeting Criteria The 14 indicators (must satisfy 9 indicators): 1. Meal frequency of family members: once or twice per day; 2. Able to purchase new clothes only once a year; 3. Have no access to medical services at public health center (puskesmas) for sick family members; 4. Main field of work of household head: agricultural worker, fisher, casual worker, or similar (income less than Rp600,000 per month); 5. The highest level of education of household head: elementary school or lower; 6. Have no savings, gold, color television, livestock, or motorcycle (at the minimum value of Rp500,000); 7. Floor area per capita less than 8 square meters; 8. Broadest floor is made of dirt/bamboo/low quality wood; 9. Broadest wall is made of bamboo/low quality wood/noncemented bricks; 10. Have no toilet facility; 11. Have no electricity facility; 12. Source of drinking water: nonprotected well/river/rain water; 13. Type of cooking fuel: wood/charcoal/kerosene; 14. Consume meat/chicken/milk only once per week. 8

9 Cash Transfer Mechanism Three institutions are involved with setting up the delivery system for the cash transfer program: BPS, Post Office (PT Pos) and Bank Rakyat Indonesia (BRI). BPS Step1: BPS produced & distributed the payment list to the Post Office Step2: PT Pos printed the beneficiary cards and receipt coupons and gave them back to BPS for a second visit distribution. Post Office (PT Pos) Step3: BPS distributed these cards to beneficiary households and do final verification then informed them when and where payments will be made Step 5: Beneficiaries were asked to come to the indicated Post Office on a certain date. Step4: the local Post Office has an account at the local BRI, where the fund will be transferred. Households Bank Rakyat Indonesia (BRI) 9

10

11 DATA 11

12 Data: Susenas Panel The data used in this paper come from Susenas (national socioeconomic) surveys in February 2005, 2006 and The February 2005 Susenas provides a good baseline for impact evaluation. A balanced panel for 2005 and 2006 containing 9,050 households: 2,444 of which received BLT. With the 2007 data, a balanced three year panel of 7,016 households: 1,715 received BLT. Among the 2,444 recipient, 639 had only received a single disbursement at the time of enumeration in 2006, while the remaining 1,805 households had received two disbursements. 12

13 Timeline of events: Subsidy, Transfers, and Surveys 13

14 Data: Targeting outcomes Only 50 percent of baseline poor households received UCT 14

15 EMPIRICAL METHODS (see appendix for the details) 15

16 Empirical Methodology Δ ln C ht = θ + α UCT h + Δ ε ht 16

17 Matching Treated to Controls (1) We used matched difference in difference method to estimate the impact of the UCT on household and individual outcomes ( panel, 2005 and 2007 panel) using propensity score matching with the following steps: Estimate the probability of UCT receipt based on household characteristics observable in February 2005 from underlying PMT variables plus: full set of household size indicators other household demographics: age and gender composition other past social program participation land holdings (quadratic) province fixed effects Use predicted values from logit to generate propensity score p(x i ) for all treatment and comparison group members 17

18 Matching Treated to Controls (2) Match Pairs: Restrict sample to common support (as in Figure/appendix) Determine a tolerance limit: how different can control individuals or villages be and still be a match? Matching methods include: nearest neighbor, kernel, local linear, radius, and weighted regression Estimate mean differences in expenditure growth, educational/health outcomes, poverty transition, and employment status between recipients and nonrecipients 18

19 Propensity Score Model 19

20 Baseline Expenditures 20

21 MAIN RESULTS 21

22 Binary Treatment Effects 22

23 Binary Treatment Effects 23

24 Multi-Valued Treatment Effects: Expenditures 24

25 Multi-Valued Treatment Effects: Expenditures 25

26 Intensity of Treatment Effects: Expenditures 26

27 MULTI-VALUED TREATMENT EFFECTS: EXPENDITURES τ10 τ21 τ20 No. of Households total ,010 (0.030) (0.033) (0.017) food ,010 (0.030) (0.034) (0.017) rice ,777 (0.038) (0.044) (0.023) tubers ,733 (0.075) (0.086) (0.055) fish, meat, dairy ,338 (0.063) (0.070) (0.034) fruit, nuts, vegetables ,850 (0.046) (0.051) (0.021) prepared food ,653 (0.070) (0.078) (0.037) alcohol, tobacco ,330 (0.069) (0.081) (0.049) nonfood ,008 (0.058) (0.063) (0.030) education, health ,507 (0.112) (0.128) (0.063) housing, utilities ,008 (0.054) (0.057) (0.024) transport, communication ,480 (0.105) (0.117) (0.056) appliances ,904 (0.067) (0.077) (0.044) debt, taxes ,997 (0.124) (0.137) (0.056) Notes: Each row corresponds to a separate regression with the log difference in the given expenditure category on the left hand side. All rows estimated by weighted least squares where the weights for treatment households equal one and the weights for control households are given by the normalized w () = Ph/(1 Ph).Allestimatesincludea5thorderpolynomialinthepropensityscoresthatisallowedtovaryby treatment and control. Standard errors clustered by village. All estimates include province fixed effects. 27

28 BLT Increased Outpatient Healthcare Utilization Short term: Recipients Non Recipients Recipients vs. Non Recipients (adjusted for selection) Δ Outpatient Visits/capita * Public Private * Δ Inpatient Visits/capita Public Private Medium term: Δ Outpatient Visits/capita Public * Private Δ Inpatient Visits/capita Public Private

29 BLT Reduced Child Labor Short term: Recipients Non Recipients Recipients vs. Non Recipients (adjusted for selection) Δ Share of Students/HH Working * Δ Hours Worked/Student * Δ Days Worked/Student * Medium term: Δ Share of Students/HH Working * Δ Hours Worked/Student Δ Days Worked/Student

30 SUMMARY OF FINDINGS 30

31 Summary of Findings on BLT Impact Household expenditures: Binary treatment effects: Short term: Recipients has 3 4% lower expenditure growth than comparable non recipients Medium term: No statistical difference in expenditure growth Multi valued treatment effects: Short term: 2 tranches recipients have 7% higher expenditure growth than 1 tranche recipients Medium term: No statistical difference in expenditure growth Treatment intensity effects: Every Rp 100,000 higher per capita transfer leads to 4 7% higher per capita growth Poverty transition: Higher per capita transfer leads to reduce probability of households to become chronic poor and to fall into poverty, higher probability to become never poor, but no significant impact for poor households to move out of poverty 31

32 Summary of Findings on BLT Impact Hours worked: Multi valued treatment effects: Short term: No statistical difference in worked hours Medium term : 2 tranches recipients worked 2 hours more per week than 1 tranche recipients Treatment intensity effects: No impact from higher per capita transfer on worked hours Health: The BLT enabled households to increase their utilization of outpatient health services Education: BLT is mildly associated with higher school dropout rates Currently enrolled children residing in recipient households experience sharper declines in labor supply than children in nonrecipient households 32

33 POLICY IMPLICATION 33

34 The Future BLT? If reform is implemented and BLT is reinstituted, the findings of this study points to some features that need to be reconsidered: Targeting improvement Size of transfer Frequency of distribution Unified Database for targeting? Per Capita? Monthly? Specific transfer for students from poor families? 34

35 THANK YOU 35

36 36 APPENDIX

37 Matching Treated to Controls We used matched difference in difference method to estimate the impact of the UCT on household and individual outcomes ( panel, 2005 and 2007 panel). If we know the selection criteria (X), then we know who is eligible. If X consists of discrete variables we can match treated and non treated with similar eligibility (according to X): 1. Assign (or match) observations into cells with similar X, 2. drop the cells where we have only treated or controls, 3. and simply do a non parametric regression in each cell 4. to retrieve ATE for respective cells However, matching suffers from the curse of dimensionality! If selection criteria X includes 1 or two 2 variables: no problem But as the dimensions of selection increase matching becomes impossible very quickly 37

38 Propensity Score Matching An alternative approach: do not match on observed characteristics, but on the probability of being treated. Propensity score matching (PSM) is a method that summarizes X to a single indicator: the probability of selection as a function of X: p X Pr T 1X F β X This is the propensity score: the conditional probability of receiving treatment given X This greatly reduces dimensionality: now only match on 1 variable! 38

39 Propensity Score Matching Rosenbaum and Rubin (1983): 1. Balancing score property If p x is the propensity score T X p X 2. Unconfounded assignment given the propensity score Y,Y T X Y,Y T p X In other words, if p x is the propensity score, and outcomes are independent of treatment conditional on X, then outcomes are also independent of treatment conditional on p X This means conditioning on p X, we have removed all sources of bias 39

40 Propensity Score Matching: Overview Indiv 1: Indiv 2: Indiv N: Pr T 1X 1 p 1 Pr T 1X 2 p 2 Pr T 1X N p N Given that T 1 Given that T 1 Given that T 1 Match for Indiv 1: Match for Indiv 2: Match for Indiv N: Pr T 1X M1 p 1 Pr T 1X M2 p 2 Pr T 1X MN p N Given that T 0 Given that T 0 Given that T 0 E Y Y p,t 1 E Y Y p,t 1 E Y Y p,t 1 average ATET 40

41 Estimating the Propensity Score PSM thus involves estimating p X and then matching treatment and control units based on this p X We can get p X by means of logit or probit estimation and then predicting the probability of selection p X Pr T 1X F β X Choice of X: Quality of data is important Understand selection process: qualitative work X should only contain baseline or otherwise exogenous data Use same survey instruments, match within same regions/markets/context 41

42 Range of Common Support (1) Common support: The distribution of p X may differ for treated and controls Especially for controls it can be hard to find high values of p X Matching is only possible if there is a similar range of p X for both treated and control units! Restrict the match to the range of common support! Look where distributions of the propensity score overlap Plot p X for the treated and non treated Drop non treated who fall outside of the region of common support 42

43 Range of Common Support (2) 43

44 Range of Common Support (3) Propensity Score Untreated Treated: On support Treated: Off support 44

45 Steps in Score Matching (1) Estimate the probability of UCT receipt based on household characteristics observable in February 2005 from underlying PMT variables plus: full set of household size indicators other household demographics: age and gender composition other past social program participation land holdings (quadratic) province fixed effects Use predicted values from logit to generate propensity score p(x i ) for all treatment and comparison group members 45

46 Steps in Score Matching (2) Match Pairs: Restrict sample to common support (as in Figure) Need to determine a tolerance limit: how different can control individuals or villages be and still be a match? Matching methods include: nearest neighbor, kernel, local linear, radius, and weighted regression Estimate mean differences in expenditure growth, educational/health outcomes, poverty transition, and employment status between recipients and nonrecipients 46

47 Marginal effects in the propensity score model (1) Notes: *Significant at 10% level. **Significant at 5% level. ***Significant at 1% level. 47

48 Marginal effects in the propensity score model (2) Notes: *Significant at 10% level. **Significant at 5% level. ***Significant at 1% level. 48

49 Marginal effects in the propensity score model (3) Notes: *Significant at 10% level. **Significant at 5% level. ***Significant at 1% level. 49

50 Marginal effects in the propensity score model (4) Notes: *Significant at 10% level. **Significant at 5% level. ***Significant at 1% level. 50