Evaluating Indonesia s Unconditional Cash Transfer Program,

Transcription

1 Final Report International Initiative for Impact Evaluation Evaluating Indonesia s Unconditional Cash Transfer Program, Samuel Bazzi Sudarno Sumarto Asep Suryahadi March 2012 Abstract Targeted cash transfer programs can be an effective means to compensate households adversely affected by the removal of commodity subsidies in developing countries. In 2005, after cutting fuel subsidies, the Government of Indonesia (GOI) implemented the world s largest unconditional cash transfer (UCT) program to date. Between October 2005 and September 2006, nearly 19 million households received quarterly disbursements of around 30 USD. This paper reports results from the first rigorous evaluation of this program with respect to several outcomes of interest over two time horizons: (i) a short-term period after which beneficiary households had received one or two quarterly disbursements and (ii) a mediumterm period by which time the program had ceased. The stated goal of the program was to sustain consumption levels among recipient households faced with commodity-specific and generalized price shocks, but health, education and labor supply outcomes are also examined in detail. A rich array of nonexperimental identification strategies offer a mixed view of the program s effectiveness. Our first set of findings suggest that the transfers did not translate into expenditure growth among recipients at the same rate as comparable non-recipients. However, we put forward evidence suggesting substantial differences between recipient households in terms of the timing of the first two disbursements of the transfer. Moreover, there are economically meaningful differences in the expenditure effects depending on household size whereby smaller households experiencing larger increases in (nonlabor) income per capita as a result of the transfer experience relatively higher expenditure growth. We also uncover a number of important sources of heterogeneity in program impacts according to location of residence, baseline income, exposure to rice price shocks, among others. We find more nuanced impacts on education, health, and labor supply. First, the added liquidity from the UCT enabled households to increase their utilization of outpatient health services at both public and higher quality private institutions. Second, although the UCT is mildly associated with higher school dropout rates, currently enrolled children residing in recipient households experience sharper declines in labor supply than children in non-recipient households. For non-enrolled, working-age adults, however, UCT receipt is associated with negligible changes in labor supply. We gratefully acknowledge financial support from the International Initiative for Impact Evaluation (3IE) and thank the Central Bureau of Statistics (BPS) of Indonesia for providing data. Umbu Raya provided excellent research assistance. We thank Michael Clemens and Craig McIntosh for useful discussion. We also thank Robert Sparrow for assistance in matching households across survey waves. Any errors that remain are exclusively ours. Department of Economics, University of California, San Diego. sbazzi@ucsd.edu. The SMERU Research Institute, Jakarta, Indonesia. ssumarto@smeru.or.id. The SMERU Research Institute, Jakarta, Indonesia. suryahadi@smeru.or.id. 1

2 Contents 1 Introduction 4 2 Program Background, Context, and Data Subsidy removal and price shocks Program implementation Data: pros and cons Empirical Methods Binary treatment effects Exploiting variation in the timing of UCT disbursements and follow-up enumeration Exploiting variation in household size to identify the intensive margin of treatment 16 4 Main Results Targeting Outcomes Propensity Score Model as an Approximation to Targeting Propensity scores and covariate balance Aggregate expenditure outcomes Heterogeneous treatment effects on aggregate expenditure growth Health outcomes Education outcomes Labor supply of non-enrolled household members Discussion Whither compensation? Threats to internal validity Marginal utility, permanent income, and timing issues The impact on poverty Conclusion: Policy implications and a way forward 34 Bibliography 36 Figures 39 Tables 45 List of Figures 1 Benefit Incidence of Fuel Subsidies, Subsidies, Transfers and Surveys: A Timeline of Events Import Ban and Rice Prices, Distribution of Transfers per Capita through February

3 5 Leakage and Undercoverage (?) Baseline Distribution of Household Expenditures per Capita Comparing Propensity Score Estimates and Approximated Quasi-PMT Scores 42 8 Overlap in Binary ( ˆP ) and Generalized Propensity Scores ( ˆP 2 ) Imbalance in the Re-weighted Baseline Distribution of Household Expenditures per Capita Comparing the Empirical Distribution of Growth in Expenditures per Capita A Nonparametric View of the Treatment (Non-)Effect Heterogeneity Along Baseline Expenditures List of Tables 1 Expenditure statistics, 2005 and Expenditure statistics, Baseline household characteristics, February Binary propensity score model, P(D h > 0 X h ) Generalized propensity score model, P(D h = d X h ) Balancing properties of ˆP h and ˆP h d Multivalued Treatment Effects: Aggregate Expenditures Multivalued Treatment Effects with Treatment Intensity: Aggregate Expenditures Binary Treatment Effects with Treatment Intensity: Aggregate Expenditures Treatment Effect Heterogeneity: Urban vs. Rural Treatment Effect Heterogeneity: Female Headed Household Treatment Effect Heterogeneity: Baseline Rice Expenditure Shares Treatment Effect Heterogeneity: Baseline Fuel Product Expenditure Shares Treatment Effect Heterogeneity: Increase in Consumption Basket Prices at Poverty Line Treatment Effect Heterogeneity: Baseline Expenditure per Capita Quintile Treatment Effect Heterogeneity: Pre-Program Enumerator Visit Changes in Healthcare Utilization, 2005 to Changes in Child Labor, 2005 to Binary Treatment Effects with Treatment Intensity: Health Outcomes, / Binary Treatment Effects with Treatment Intensity: Education Outcomes, / Binary Treatment Effects with Treatment Intensity: Adult Labor Supply, / Welfare Transitions and UCT Receipt

4 1 Introduction In the midst of escalating global oil and gas prices in 2005, the Government of Indonesia (GoI) slashed fuel subsidies, raising regulated prices by a weighted average of 29% in February and then again by 114% in September. The measures yielded over $10 billion in annualized budgetary savings, a portion of which the GOI put towards the country s first large-scale unconditional cash transfer (UCT) program. 1 The UCT program was designed to prevent poor households from having to reduce expenditures on essential commodities, health, and education in the midst of strong inflationary pressure. From October 2005 to September 2006, the GOI distributed quarterly installments of 300, 000 Rupiah (Rp) (about 30 USD) to over 19 million households, effectively embarking upon the world s largest UCT program. In this paper, we utilize several well-established difference-in-difference (DID) evaluation methods to assess the impact of the UCT on per capita household expenditures as well as health, education, and labor market outcomes. Whether this program compensated households for higher fuel prices and the attendant generalized inflation is a question of central policy importance. The government has since introduced a second round of subsidy reductions and attendant cash transfers in 2008 and And at the time of writing, efforts are underway to implement another round of subsidy cutbacks and cash transfers. Yet, to date, there exists little rigorous research on the effects of the initial large-scale subsidy reform and public transfer program in Indonesia. After exploring in detail the targeting of benefits, we focus first on aggregate household expenditures per capita, a central dimension of welfare. This is a natural focal point since the explicit public goal of the UCT program was to sustain expenditure levels among poor households for whom the subsidy cutbacks were expected to adversely affect purchasing power. At the same time, the government also launched other programs in 2005 targeting education and health outcomes, rendering the study of schooling and health impacts of the UCT more challenging. We attempt to identify the direct effect of the cash transfers relative to these other tied forms of aid received by millions of Indonesian households over the same period. In particular, we examine the impact of the UCT not only on health and education expenditures but also on various types of healthcare utilization, school continuation rates, and the labor supply of current students and adult household members. A rich array of nonexperimental identification strategies offer a nuanced view of the program s effectiveness. On the one hand, we find that the UCT led to increased utilization of outpatient healthcare services as well as a reduction in the labor supply of currently enrolled schoolchildren. With regards to expenditure outcomes, however, the program did not produce 1 It is estimated that the budgetary savings for October through December 2005 were approximately Rp 25 trillion (about 2.5 billion USD). Of this, Rp 4.7 trillion was initially allocated to the cash transfer program (see Sen and Steer, 2005). Subsequent allotments increased as beneficiary lists expanded. The program eventually cost approximately 0.8 percent of GDP in 2005 and

5 growth among recipients at the same pace as comparable non-recipients. If anything, we observe a negative binary treatment effect. However, by utilizing the staggered program rollout, we show that households receiving two disbursements by January-February 2006 (closer to follow-up enumeration) achieved higher consumption growth than did households that received only a single disbursement. Moreover, because the size of the transfer was fixed regardless of household size, the scale of benefits varied considerably across recipient households. Exploiting this unique feature of the program, we find that conditional on the recipient vs. nonrecipient differential, expenditure growth is increasing in the amount of transfer per capita. These multivalued and intensive margin treatment effects point to the importance of timing and scale of treatment in evaluating the impact of transfer programs. In general, the findings suggest that the targeting of UCT benefits may have been a crucial determinant of the outcomes observed in this evaluation. Given the absence of the underlying proxy means test (PMT) eligibility scores, forging a plausible counterfactual to UCT recipient households proved challenging. Our best attempt at reconstructing those PMT scores using available data reveals not only that the eligibility threshold was not binding but also that our estimated propensity scores capture as much if not more variation in treatment status. While we observe substantial overlap in propensity scores relative to other nonexperimental evaluations in the literature, our (generalized) propensity score model explains merely one-fifth of the variation in treatment status across households. We utilize a wide array of household characteristics with relatively flexible functional forms, but our baseline data only offer a subset of characteristics observable to enumerators and local officials. It is therefore possible that the unobserved local targeting process successfully identified those households for whom future earnings potential were lowest, thereby inducing a mechanical downward bias in the estimated treatment effects. Indeed, a recent study by Alatas et al (2010) on targeting in Indonesia demonstrates that local enumerators have the capacity to identify the poorest households where standard survey-based methods may be ineffective. If this were the case in the UCT we examine, then it is possible that the beneficiaries are in fact much poorer (in a permanent income sense) than the matched households in the control group; the strong overlap in propensity scores notwithstanding. As we carefully demonstrate, the propensity scores are ultimately insufficient to re-balance the distribution of baseline expenditures among recipients and non-recipients, as required for constructing a proper counterfactual. For this reason, we emphasize the identification of program effects using treatment intensity based on the staggering of the second disbursement and variation in household size rather than solely the conventional binary treatment effect based on recipient status. Our emphasis is supported by the facts that staggering across regions was largely exogenous and that household size is (unconditionally) balanced at baseline and moreover provides policy-relevant variation. We consider other candidate explanations for the lack of a robust positive binary treatment 5

6 effects on aggregate expenditures. First, we acknowledge the possibility of non-classical measurement error in expenditure reporting in the follow-up survey data. The first few months of the UCT program in 2005 generated a great deal of public controversy surrounding the allocation of benefits and widespread perception of mistargeting. During that period, eligibility lists were not only being expanded but in some cases redrawn altogether. Recipient households enumerated in February 2006 may therefore have perceived their ongoing participation as being contingent on reported welfare levels. Such beliefs could have led to systematic underreporting of expenditure levels among precisely those households which had received UCT disbursements by early 2006, and especially those households awaiting a second disbursement. This quasi-ashenfelter s dip also potentially impart a substantial downward bias in estimated treatment effects for expenditure and other easily manipulable non-expenditure outcomes. There are additional plausible explanations for the seeming puzzle that UCT beneficiaries expenditures grew slower than comparable non-recipients. Some have argued that the urban poor received too little compensation relative to rural households less affected by the subsidy cutbacks, thereby diluting potentially large effects of income transfers (e.g., Yusuf and Resosudarmo, 2008). Yet, a negative binary treatment effect may still be consistent with economic theories concerning household responses to predictable and transitory income shocks such as the UCT. A simple marginal utility of income argument suggests that the response to a positive nonlabor income shock at time t should, all else equal, be declining in wealth levels at time t 1. Considering that a sizable share of UCT recipients were non-poor, this hypothesis could explain the low average treatment effects we find. Indeed, we find that the gap in expenditure growth between recipients and non-recipients is growing as we move up the distribution of baseline household expenditures per capita. Lastly, we demonstrate how a concomitant upswing in rice prices detailed in Section 2.1 may have dampened the observed treatment effect of the program. Taken together, the multivalued and intensive margin treatment effects as well as the positive impact on health service utilization and reduction in child labor point to an important role of cash transfers in cushioning the poor against inflationary shocks. Though more difficult to monetize, the latter benefits surely deserve the attention of policymakers. In a period of rapidly escalating inflation and economic downturn, temporary adverse health and schooling shocks can have permanent effects. Cash transfers are of course fungible across household expenditure allocation priorities. That the UCT program provided households with differential access to health services and child labor-saving investments should not be overlooked when weighing the overall impact of the program. The findings in this paper contribute more generally to a large literature on the role of cash transfers in developing countries (see Hanlon, Barrientos, and Hulme, 2010). Unlike numerous programs in Latin America and elsewhere, however, the UCT in Indonesia was not explicitly designed as a poverty alleviation program. Rather, the GOI instituted the program as a means to compensate households for the increase in fuel prices and the subsequent inflation of other 6

7 commodity prices. Similar subsidy reforms have either recently been implemented or are being considered across a number of developing countries. It is hoped that the results in this paper speak to the ongoing debate over optimal design of transfers and subsidies both within Indonesia and elsewhere in the developing world. The remainder of the paper proceeds as follows. Section 2 provides background on the program and the dataset employed in the analysis; Section 3 motivates the evaluation model and details the identification strategy; Section 4 presents the primary empirical results; Section 5 considers a range of competing explanations for the findings; and Section 6 concludes. 2 Program Background, Context, and Data In early 2005, the macroeconomic rationale for downgrading fuel subsidies seemed clear to most macroeconomic observers. 2 For proponents of the subsidy cutbacks and cash transfers, the package was at once politically expedient, reformist on efficiency and equity grounds, and development-friendly. This section addresses the latter features of the program. First, we discuss the ex ante effects of the subsidy downgrades in terms of the effects on inflation and the distributional implications. We also briefly discuss a coincident shock to the domestic price of rice unrelated to the subsidy downgrades but with potentially more important consequences for welfare among the poor. Second, we detail the implementation of the UCT program with respect to program design, targeting, and distribution of benefits. Third, we describe the actual data used in our evaluation. 2.1 Subsidy removal and price shocks The GOI intended that the UCT program would cushion poorer, more vulnerable households from the economy-wide inflationary upswing induced by higher transportation and production costs. Although the fuel subsidies were mostly regressive, poor households relied on subsidized cooking oil and were therefore expected to be hurt directly by any policy changes targeting kerosene. Furthermore, the 17.9% year-on-year inflation from February 2005 to February 2006 was expected to hit the poor hardest. The subsidy reform proceeded in two stages. In March 2005, the government raised gasoline and automotive diesel prices by 33 and 27 percent respectively. The price of kerosene was left untouched. After several months and some publicity, the GOI dramatically slashed subsidies on October 1st, effectively raising prices of the three fuel products by a weighted average of 114 percent. Previously immune to policy change, kerosene prices nearly tripled increasing by At the same time, it was not obvious that the consequences of the status quo would be severe enough to warrant the extent of policy shift implemented later that year. According to Sen and Steer (2005), the fuel subsidies were not a fundamental threat to macroeconomic stability. World Bank and other analyses suggested that A $10/barrel increase in the world price of oil would thus increase the central government budget by $1 billion (0.3% of GDP), whereas the whole-of-government budget deficit would increase by only about $500 million (0.15% of GDP). 7

8 percent while gasoline and diesel prices grew another 88 and 105 percent respectively. The direct effect of these price shocks on household welfare depends first and foremost on the incidence of fuel consumption. Based on nationally representative household survey (Susenas) data from February 2004 prior to the first round of subsidy downgrades, over 95 percent of Indonesian households consume at least one of the three main fuel products, and over 90 percent consume kerosene. Figure 1 examines the distribution of national fuel expenditures across deciles of household expenditure per capita in The figure conveys the share of national expenditures on the three fuel products separately for urban and rural households. Automotive diesel and gasoline subsidies are most clearly regressive while the overall incidence of kerosene consumption tends to be relatively flat across the distribution of income. The slight dip in kerosene consumption among wealthier urban households suggests a modest progressive element to kerosene subsidies in urban areas. Yet, urban households also unsurprisingly benefit most from the gasoline subsidies. It is important to keep in mind that fuel products comprise a small share of overall household expenditures. Consider first the difference in budget shares between the top and bottom decile of household expenditures calculated within urban and rural areas separately. On average, the poorest (decile of) households allocate 3.7 percent of total monthly expenditures to kerosene while the richest households spend only 1.9 percent. Moreover, 93 percent of the poorest households and 80 percent of the richest households purchased kerosene in the month preceding enumeration in February Meanwhile, only 6 percent of the poorest households directly purchase gasoline compared to 46 percent of the richest households. The corresponding average budget shares for gasoline were 0.1 percent for the poorest households and 2.3 percent for the richest households. 3 Thus, although a large swathe of the population stood to be adversely affected by the kerosene and gasoline subsidy removals, these small budget shares suggest that the pass-through to purchasing power, if indeed as negative as initially feared, would have to occur through more indirect channels. The indirect effect of the subsidy removals would depend on more complex general equilibrium channels through which fuel prices affect not only public transportation services and transportation of goods to market but also production of goods with substantial fuel-based inputs. 4 Figure 2 shows the timing of the subsidy removals and the subsequent pass-through to other consumer goods and services in addition to the fortuitous timing of the Susenas enumeration detailed further below. The year-on-year inflation rate provides a convenient benchmark as Susenas surveys are conducted on annual basis. Fuel products fall under the Transport/Communications component of the Indonesian CPI. Several features stand out. First, the economy-wide effects from the limited downgrade of gasoline and diesel subsidies in early 2005 were relatively small. Second, the slight drop in education costs prior to the onset of the school 3 Only 1 percent of all Indonesian households consume any diesel fuel. 4 For reasons noted in Section 2.3, we are unable to disentangle in Susenas data direct expenditures on the three fuel products from expenditures on transportation services and household electricity use. 8

9 year in August 2005 could be attributed to a national program providing schools with funds aimed at lowering out of pocket expenses required by students. Third, the inflationary upswing appears to persist through late Lastly, the path of food prices appears to follow a similar trajectory as fuel prices albeit for mostly orthogonal reasons. Around late 2005, the price of domestically-produced rice the main staple among the majority of Indonesian households began a steep upward ascent due in small part to rising transport costs but mostly due to the government decision to ban rice imports in January While a boon to net producers, the spike in rice prices had arguably more severe consequences for poor households than did the downsizing of fuel subsidies. 5 Given that the median household spends 20 percent of its budget on rice, this additional price shock constitutes an important confounding event in the midst of the UCT evaluation period. The graphs in Figure 3 provide a glimpse into the divergence of domestic from world prices (left) as well as the spatial variation in the magnitude of the shock (see Bazzi, 2012). In Section 5.1, we consider how heterogeneity in rice consumption affects the observed treatment effects. 2.2 Program implementation Given the challenges of identifying ex ante losers from the subsidy downgrade, targeting of UCT benefits would prove difficult. Moreover, the relatively short period between program conception and rollout left enumerators and implementing agencies little time relative to other countries experiences with such large-scale transfer programs. Political exigency and prior experience with social welfare programs nevertheless dictated the targeting procedures and institutional implementation. We provide a glimpse into these program features here, but more detailed background can be found in SMERU (2006a). The targeting of beneficiaries proceeded in three stages. First, local government officials devised a large list of potential recipients in August Second, using a minimalist survey instrument (known as PSE05.RT), the regional public statistical bureaus enumerated those households and others in some cases. In practice, only 35 percent of households report ever being visited by enumerators (8 percent did not know whether or not their household was visited). The majority of those enumerated were visited not by BPS officials but by local government officials. 6 Lastly, the Central Statistics Bureau (BPS) used the survey data to implement a proxymeans test to generate the final list of eligible households by the end of September. Unfortunately, neither the raw data used to devise the official proxy-means test nor the PMT scores are yet available to researchers. The National Development Planning Agency (BAPPENAS) along with BPS established the proxy means test based on 14 variables from the September eligibility survey. The questions concerned: 1) floor type, 2) wall and roof type, 3) toilet facility, 4) electrical source, 5) cooking fuel source, 6) drinking water source, 7) frequency of meat consumption, 5 See Simatupang and Timmer (2008) for details on the policy environment and McCulloch (2008) for an evaluation of the (expected) effects on poverty. 6 These and all figures reported in this subsection are tabulated from the Evaluation Module of the Susenas 2006 panel. 9

10 8) frequency of meal consumption, 9) frequency of purchase of new clothes, 10) access to public health facilities, 11) primary source of income, 12) educational attainment of household head, 13) amount of savings and type of assets, and 14) floor width. 7 Given the initial program budget, all poor and near-poor households (within 20 percent above the poverty line) were technically eligible. In practice, as documented in Section 4, many non-(near-)poor households received program benefits while numerous poor and near-poor households were excluded. The distribution of eligibility certificates proceeded mostly through village officials (56 percent) while nearly 80 percent of recipients retrieved the quarterly disbursements through the post office to which the average travel and queuing time was less than 2-3 hours. Widespread socialization through local institutions and media resulted in relatively high levels of knowledge regarding program details. Yet, over 25 percent of households did not know the program was intended primarily to benefit poor households. Although nearly 80 percent of households knew the total amount of quarterly benefits to which recipients were entitled, local officials in some regions succeeded in extracting a portion of the officially mandated 300,000 Rp disbursements. Approximately 6.5 percent of recipients were subject to these informal taxes at the time of obtaining their first UCT disbursement in late 2005, and 8.5 percent of recipients paid a tax on their second disbursement in the first two months of These taxes went primarily to officials in the village (desa) and sub-village or hamlet (dusun). According to recipients subjected to these taxes, the proceeds were meant to cover local ID/certificate administration, security at disbursement centers, but most were intended for redistribution to non-recipients deemed deserving by local officials. The portion allocated to supposed local redistribution increased from 40 percent at the first disbursement to 62 percent at the second disbursement. Among those taxed, the median amount also increased from 20,000 Rp to 50,000 Rp. These increases were likely due in part to the rising discontent with the initial eligibility lists. In fact, by early 2006, the GOI extended UCT benefits to an additional 4 million households. But just how large were these UCT benefits accruing to over one-quarter of all Indonesian households and providing local officials with a novel source of informal tax revenue? The full transfer amounted to approximately half of baseline median monthly household expenditures among recipients and almost twice as much as median expenditures in per capita terms (see Table 1). The UCT benefits were larger than any direct welfare transfers in the past, but given rapid inflation, the real value of the transfers was arguably more limited. Recipient households reported a range of uses for the transfer: 23 percent reported using the 7 The specific questions were chosen based on statistical exercises aimed at identifying the most powerful predictors of poverty using Susenas survey data from Program architects assigned weights to each of the survey responses through a series of Tukey tests and stepwise logistic regressions of poverty status on the relevant predictors. These weights varied across districts so as to account for the wide variation in the correlates of poverty across Indonesia s vast archipelago. 10

11 transfers for kerosene expenditures (the tagging effect); 9 percent for education; 31 percent for health; 6 percent for food; 12 percent for capital; 20 percent for other. While household income is certainly fungible, these subjective assessments provide a glimpse into household perceptions of the program. Additional evidence can be found in SMERU (2006a), which reports a relatively large share of households report using BLT funds to draw down debts in addition to these other expenditures borne out in the nationally representative Susenas data. Disentangling the UCT benefits from other intervening household income shocks will be a primary task for the econometric analysis below. 2.3 Data: pros and cons The data used in this paper come from Susenas surveys in February 2005, 2006 and We are able to identify most households across all three survey rounds allowing for both a shortand medium-term evaluation of the program. Although implemented prior to the design of the UCT program, the February 2005 Susenas provides a good baseline for impact evaluation since (i) the fuel subsidy reductions did not begin until March 2005, and (ii) the first installment of the UCT program was not disbursed until October 2005 (see Figure 2). The baseline survey contains 10,574 households, while the follow-up in February 2006 contains only 9,892 households. After matching on geographic and household identifiers, we obtain a balanced panel for 2005 and 2006 containing 9,050 households, 2,444 of which received at least one UCT disbursement by the time of enumeration. Among non-recipient households, 37 had received an eligibility card by February 2005 but had not yet received any UCT disbursements; we group these eligible non-recipients with the ineligible non-recipients for evaluating the short term effect of the program between 2005 and The February 2007 survey meanwhile contains more than 55,000 households, a subset of which were interviewed in the two preceding years. Adding the 2007 survey, we obtain a balanced three-year panel comprised of 7,016 households, 1,715 of which received UCT benefits prior to the program s cessation in late A unique feature of the 2006 survey allows us to go beyond a simple evaluation of binary treatment effects. Approximately 27 percent of households in the treatment group not yet received the second UCT disbursement by enumeration in February Among the 2,444 everrecipient households identified in 2006, 639 had only received a single disbursement at the time of enumeration while the remaining 1,805 recipient households had received two disbursements. 9 We exploit this staggered implementation to examine multivalued treatment effects along this intensive margin of benefits. Furthermore, because the (single tranche) transfer size is fixed regardless of household size, we can also exploit variation in transfers per capita across households as an additional source of heterogeneity in the intensity of treatment. 8 See the notes to Tables 1 and 2 for details on the panel construction. 9 The date on which households in the panel data received the transfers is unfortunately not recorded. Using crosssectional data for around 250,000 Indonesian households in June 2006, we find that 64% of recipients obtained their first tranche in October 2005, 78% by the end of 2005, and an additional 20% in the first half of

12 Despite the convenient panel setup, the survey data have a few important limitations. 10 A key limitation is that only a subset of the abovementioned eligibility questions from the PSE.05 survey exist in the Susenas data available to researchers. In particular, we directly observe only eight of the fourteen eligibility indicators in the February 2005 baseline survey. Among the most important questions unavailable in the Susenas survey are those concerning frequency of meal consumption, assets, and savings proxies. While it is not possible to obtain the actual household proxy mean scores that would allow us to implement a regression discontinuity design, we can use the available questions in Susenas coupled with the district-specific coefficients for each qualifying criteria to construct a quasi-pmt score. In Section 4, we compare these quasi- PMT scores to results obtained from predicting the probability of selection into treatment in a regression framework on the basis of a larger set of observable household characteristics. Another issue concerns the translation of nominal expenditures into real terms. Since our primary estimation strategy involves taking a double difference, deflating nominal expenditures in 2006 and 2007 by using uniform national or provincial CPI measures across all households will leave the results unaffected. That is, deflating all households by the same measure merely shifts the overall or province-specific intercepts in estimates of the treatment effects. Taking an alternative approach that deflates nominal food and non-food expenditures separately before constructing total expenditures for each household leaves the main qualitative results in the paper unchanged Empirical Methods In this section, we detail our empirical strategy. Lacking the proxy means scores used in assigning UCT eligibility, we consider two alternative identification strategies to recover the effect of the UCT on household welfare. Both are based on a difference-in-difference estimating framework. The first approach is based on new insights from reweighting and propensity score matching 10 The data structure also present a somewhat nonstandard attrition problem. We do not know which attritors between 2005 and 2006 actually received the UCT. We observe recipient status among the 2034 attritors between 2006 and 2007, and somewhat reassuringly the ratio of recipients to non-recipients remains essentially unchanged across years. In the absence of further information from survey administrators, we are unable to know whether attrition is due to our inability to merge households using available second-best methods (see notes to Table 1) or as a result of purposive household relocation. Although inter-survey attrition is potentially a non-negligible problem, we ignore its consequences in the econometric results presented below. This is mostly a practical decision given the complexity of the attrition problem in a multi-year panel. Nevertheless, the primary results are robust to reweighting the sample so as to account for the probability of attrition. Additionally, at baseline in February 2005, the subsequent attritors appear much more similar to non-recipients than recipients. This can be seen in the descriptive expenditure statistics in Table 1 and also Table 6 which plots the nonparametric density of the log of household expenditure per capita in 2005 for several groups of households in the sample. 11 Deflating in this manner generally reduces the binary treatment effect of the UCT on expenditure growth still further since the poor consume relatively more food, the CPI for which increased relatively faster than non-food due to the aforementioned rice price shock. An alternative approach taken by others working with Indonesian data (see, e.g., Skoufias, 2003) is to construct household- and good-specific price deflators. In the absence of a more rigorously founded household-level demand system, such an approach will not in general lead to substantive departures from our findings below. 12

13 methods. We build a rich propensity score model incorporating as many elements of the targeting design and other information plausibly available to local enumerators as data constraints permit. We then show that quasi-pmt scores constructed using region-specific PMT coefficients and available data in Susenas perform worse than these estimated propensity scores in terms of re-balancing treatment and control households at baseline. Nevertheless, as demonstrated in Section 4, neither approach to predicting UCT receipt suffices to achieve sufficient balance at baseline in terms of household expenditures per capita to warrant interpretation of the binary treatment effects as causal. Therefore, we also pursue a second approach exploiting two features of the program design that generate plausibly exogenous variation in the timing and scale of transfers. The first source of variation comes from the staggered rollout of the second tranche of disbursements. Given the timing of our follow-up survey in early 2006, around one-quarter of UCT beneficiaries had not yet received their second disbursement. After reweighting the sample of one-tranche and two-tranche recipients to account for the possibility of endogenous staggering (e.g., one-tranche recipients live in poorer, more remote villages), we can identify the differential treatment effect of two vs. one transfer more reliably than the binary treatment effect. The second source of variation comes from the fact that the one-off transfer amount was fixed regardless of household size. Given unconditional balance across recipients and non-recipients in baseline household size, we exploit variation in transfers per capita to identify a plausibly exogenous and policyrelevant intensive margin of treatment. 3.1 Binary treatment effects Under the simplest formulation of the binary treatment effects model, the primary estimating equation is given by Y ht = θ + αuct h + ε ht. (1) where Y ht is the difference in the outcome of interest for household h between t =February 2006 (or February 2007) and t 1 =February 2005; UCT h is an indicator for program receipt; and ε ht is an idiosyncratic shock. Absent random assignment of UCT, we cannot consistently estimate α by straightforward OLS. To identify α in a non-experimental context, one would estimate the average treatment effect on the treated (ATT) using DID and incorporating estimated propensity scores. The primary ATT estimand of interest is E [Y h (1) Y h (0) UCT h = 1, X h ] where Y h (1) is the given outcome of interest for household h as a UCT beneficiary, Y h (0) is the outcome for household h as a non-beneficiary and X h includes observable household characteristics. Since we only observe Y h = UCT h Y h (1)+(1 UCT h )Y h (0) for each household h, we must construct an appropriate counterfactual to which UCT recipient household h can be compared. The empirical analysis below implements several DID matching and reweighting estimators, the asymptotic properties for which are elaborated in Heckman, Ichimura and Todd (1998), Hirano, Imbens and Ridder (2003), Abadie (2005), and Abadie and Imbens (2006; 2008). Crucially, the DID approach allows UCT recipients and comparable non-recipients to differ systematically 13

14 in terms of unobserved time-invariant determinants of both the outcome and treatment status. Empirical implementation proceeds as follows. Let D be the number of UCT disbursements received, let ˆP P(D h > 0 X h ) be the estimated binary propensity score for household h, let N 1 and N 0 be the number of UCT recipients and non-recipients respectively, and denote recipients by subscript h and non-recipients by h. The DID estimator of the ATT 12 is then given by ˆα = 1 N UCT h Y ht 1 N 1 N 0 h=1 N h =1 (1 UCT h ) Y h t w(h, h ). (2) [ We employ the normalized weights w(h, h ) = ˆPh /(1 ˆP [ h )] / N0 1 N0 ˆP h =1 h /(1 ˆP ] h ) to estimate four versions of reweighting estimators put forward in the earlier evaluation literature and discussed at length in Busso, DiNardo and McCrary (2011) (hereafter BDM): (i) basic inverse probability weighting estimates equation (1) by weighted least squares (WLS); (ii) double robust augments equation (1) with ˆP or X (i.e., the covariates in the propensity score model) and estimates by WLS; (iii) homogeneous control function augments equation (1) with 5th order polynomials in ˆP and estimates by WLS; (iv) heterogeneous control function augments equation 1 with 5th order polynomials in ˆP and estimates by WLS separately for recipients h and non-recipients h, obtaining ˆα as the difference between the two predicted outcomes Y ht Y h t. 13 In addition to these straightforward reweighting estimators, we also utilize several popular matching methods, which BDM demonstrate to be mere reformulations of w(h, h ) in (2). In particular, kernel, local linear, and nearest neighbor matching can be derived as particular forms of w(h, h ) depending on the distance between ˆP h and ˆP h. 14 Though matching procedures have been more common in recent evaluation literature, we prefer reweighting estimators for a few reasons. First, the estimators are far less computationally demanding. Second, and more importantly given the context here, reweighting estimators have been shown to outperform standard matching estimators in terms of finite sample properties and especially in situations with considerable overlap in estimated propensity scores. 12 An additional estimand of interest is the quantile treatment effect on the treated or QTT (see Firpo, 2007). The QTT allows one to identify treatment effects that vary along the distribution of Y and is given by the following expression φ τ UCT =1 = q 1,τ UCT =1 q 0,τ UCT =1 where q k,τ UCT =1 inf q P[Y (k) q UCT = 1] τ. Although we estimate QTT for several outcomes of interest, the results available upon request are generally uninformative beyond what can be learned from the ATT estimates. Moreover, the asymptotic justifications for a DID approach using QTT have not been fully developed. 13 These last estimates are obtained by first separately demeaning the propensity scores for the recipient and nonrecipients (Imbens and Wooldridge, 2009). 14 We neglect other common matching procedures as their large sample properties have not yet been established in the literature. Heckman, Ichimura and Todd (1998) show that kernel and local linear matching consistently estimate the ATT under the following assumptions discussed at length elsewhere: (A.1) (A.2) E( Y ht P h, D h > 0) = E( Y ht P h, D h = 0) for any household i P(D h > 0 X h ) < 1 for all households Abadie (2005) establishes the consistency of ˆα in (2) under the same assumptions. 14

15 Ultimately, reliable identification in both the matching and the reweighting context still hinges on accurate prediction of treatment assignment. Despite our best efforts, however, we are unable to obtain propensity scores ˆP h capable of balancing treatment and control households along baseline expenditures (see Section 4). While the lack of balance calls into question the internal validity of our estimated ATT parameters ˆα for outcomes of interest, we do not abandon the reweighting approach altogether. Instead, we augment the reweighting results with additional sources of variation in treatment. 3.2 Exploiting variation in the timing of UCT disbursements and follow-up enumeration We identify multivalued treatment effects by exploiting the staggered second UCT disbursement. 15 The timing of the follow-up survey in early 2006 enables a test for whether receiving the second disbursement had an additional effect on top of the first disbursement D 1. Given the size of the transfers, the differential margin of treatment between single and multiple disbursement households could be quite important. Practically, the two groups of households undergo substantially different treatments. Statistically, distinguishing between the two groups can provide a cleaner estimate of program impacts than the usual comparison between recipients and non-recipients. For estimation, we rely on several reweighting procedures highlighted in Flores and Mitnik (2009). 16 The primary estimand of interest is given by τ ds = E[Y (d) D = d] E[Y (s) D = d] or the difference in outcomes between a household under treatment d relative to treatment s. The treatments d include: d = 0 for non-recipients; d = 1 if household h received the first UCT disbursement but not the second; d = 2 if household h received both UCT disbursements, D 1 and D 2. We recover estimates of τ 20, τ 10 and τ 21, from the following regression appropriately weighting by the inverse of the estimated generalized propensity score (GPS), ˆP d P(D h = d X h ), of the level of treatment actually received by household h, Y ht = 1{D h = 0} + τ 10 1{D h = 1} + τ 20 1{D h = 2} + ε ht, (3) where τ 21 can be obtained as the difference τ 20 τ 10. Despite the estimates of τ 20 and τ 10 being subject to the same criticisms as those of α noted earlier, we argue that τ 21 is relatively more well-identified. 15 See Imbens (2000) and Cattaneo (2010) for theoretical background on the estimation of multivalued treatment effects. 16 Like Flores and Mitnik (2009), we neglect to estimate the more complicated and computationally intensive multivalued extensions of the binary matching procedures mentioned above. For reasons discussed by the authors, these estimators are unlikely to yield any advantages over the more tractable reweighting methods. 15

16 3.3 Exploiting variation in household size to identify the intensive margin of treatment In addition to variation in the timing of the second disbursement, we also utilize the fixed transfer size to identify the effect of transfers per capita. Figure 4 plots the distribution of transfers per capita for all recipients demonstrating the clear variation across recipient households conditional on the number of disbursements d. Among recipients, median transfers per capita were Rp 120,000 (mean Rp 156,000). The two disbursement recipient households obtained median transfers per capita of Rp 150,000 (mean Rp 179,000), and single disbursement recipients Rp 75,000 (mean Rp 91,000). To identify this intensive margin of treatment, we augment the reweighted versions of equations (1) or (1) with transfers per capita and an exhaustive set of indicators for household size at baseline: 13 Y ht = θ + αuct h + ψ transfers/capita h + β k 1(HH size h,t 1 = k) + ε ht (4) Y ht = 1{D h = 0} + τ 10 1{D h = 1} + τ 20 1{D h = 2} + ψ transfers/capita h + 13 k=0 k=0 β k 1(HH size h,t 1 = k) + ε ht. (5) After removing the (unbalanced) binary treatment effect through reweighting and the independent effects of household size through β k terms, all that remains is information on the scale of UCT benefits. Under the assumption that E[HH size ε ht ] = 0 (after reweighting), ψ then identifies the marginal effect of an additional unit of non-labor income per capita on the change in welfare outcome Y. Of course, this source of identification is not without caveats of its own. We address these in turn Main Results After evaluating the targeting and distribution of UCT benefits, we demonstrate that our estimated propensity score, while superior to reconstruction of quasi-proxy mean scores, fail to achieve balance in baseline household expenditures. We proceed to estimate the problematic binary treatment effects based on these propensity scores, highlighting the difficulties therein, and calling attention to better identified estimates of multivalued and intensive margin treatment effects. We consider a range of outcomes in first-differences between 2005 and 2006 or 2007: log expenditures per capita, healthcare utilization, school enrollment, child labor, and adult labor supply. 17 In ongoing work, we explore other, potentially better ways of identifying the intensive margin of treatment in this context. 16