September 8th, 2016 GPED Forum Vanderbilt University Cash transfers, impact evaluation & social policy: the case of El Salvador The talk aims to present the experience of El Salvador in the implementation of the Comunidades Solidarias Cash Transfer Program, focusing on the impact evaluation (methodology & main results) and its link to social policies and social protection. Country Background El Salvador is densely populated, has a youth unemployment of around 20%, an informal sector about 70%, 35% of families (more than half a million families) in multidimensional poverty, historical low economic growth rate (average 2%) and with the highest levels of violence in LAC. In addition, the country is one of the most vulnerable and at risk, with constant threats due to external shocks of climate change (floods, droughts, etc.). If we look back in the history of the country, one of the key achievements that laid the foundations of democracy was the signing of the Peace Accords in 1992, nearly 25 years ago. Political change has been possible, four consecutive right wing governments followed by two left wing governments. The road has not been easy; we still face a major political polarization that has slowed the pace towards development. In this context, in 2005 under a right wing government, Red Solidaria was launched. A conditional cash transfer program, which was based on a rigorous design and on the experience and best practices of the first generation of CCTs programs in Mexico, Brazil and Chile. The results of the impact evaluation contributed to the continuation and improvement of the program -during the first leftist government in 2009-, becoming the platform for the conceptual design of Social Protection System in El Salvador. Overview of Social Protection System The Social Protection System is a rights-based approach and seeks to guarantee the population a wide, safe and adequate social protection, particularly to those in conditions of greater vulnerability and poverty, exclusion and social inequality. The 1
Social Protection System s implementation started during 2009 with the main social protection programs (CCT rural & urban, temporary income support program, school supplies, etc.), but it wasn t until 2014 that the Law of Development and Social Protection was passed. This law also seeks to institutionalize the mechanisms and procedures for conducting strategic social and effective coordination. The social protection system includes interventions in the following areas: Social Assistance, Social Security, Labor market programs (Active labor policies) and Social basic services. In El Salvador the interventions are concentrated in the poorest, excluded and most vulnerable population that face risks related to individual life cycle (sickness, death), economic (commodities price shocks), environmental (droughts, floods, landslides), and social/governance related (e.g. social violence). And the main informal families strategies for coping with these risks has been migration and remittances, as well as the known ones such as child labor, women as family provider, support of extended family, etc. Comunidades Solidarias, Cash Transfer Program Comunidades Solidarias (former Red Solidaria) is a program that was designed from the beginning as a comprehensive territorial intervention in the poorest municipalities, which includes a CCT component, social basic infrastructure, productive insertion and employability and local participation. Identification of the poorest municipalities was based on the Extreme Poverty Map 2005, and to target the poorest households in those municipalities a proxy mean test was applied 1. It was established as a rolling program, initiated its intervention in the 32 municipalities in severe Extreme Poverty Group (EPG) during 2005 and 2006, and in 2007-2009 the program was extended to the other 68 municipalities that conformed the high EPG, reaching more than 100 thousand families. 1 The PMT uses a set of proxies (usually between 10 and 30) that best explain welfare. Each proxy is given a weight based on its estimated impact on household expenditure. 2
One of its main constraints is that is a static program, does not include new families throughout its implementation; this has a major impact on the coverage of the program, which has decreased its coverage in 20 thousand families by 2015. Another limitation is the weak implementation of the productive component, the step needed to improve productive and working skills and entrepreneurship to generate own income. The program has a rural intervention (the original Red Solidaria) based on the Rural Extreme Poverty Map that identified the poorest municipalities (Partition Cluster). It has a double objective to smooth consumption with the cash transfer to the families and to promote human capital accumulation in order to break the cycle of poverty. The CCT seeks to improve health and nutrition among children under 5 years old and pregnant women with a transfer of $15 dollar a month per family 2, when complying with the health checkups according to the health protocol. The CCT also aims to improve school enrollment among children (ages 6-15 years) who have not completed primary school, with a transfer of $15 dollar a month per family, when complying with enrolment and school attendance of more than 80 percent of the time each month. 3 Families who are eligible for both health and education transfers receive received $20 per month. One of the main challenges of the program was to establish an effective coordination mechanism and develop the necessary instruments to ensure a transparent and successful implementation. The Management Information System (MIS) becomes the backbone of the program. This management tool for the program s administration, operation, monitoring and evaluation includes the identification (targeting, enrolment, exit), registration (database, validation, updating), certification (data collection, verification), payment (eligibility, payment & reconciliation) and control processes (complaints). 2 For the purpose of the program a family was considered as the nuclear family, so there could be more than one family in a household. 3 It was considered primary school and not middle school (with a much lower enrolment rate) because of the lack of supply in the rural areas, and the group age of 6-18 years because of overage in the rural areas. 3
The program is implemented by different state agencies, NGOs family support and monitoring of responsibilities, and local government participation in the different stages. In order to achieve a multiagency implementation, the following committees were established at different levels: political -social cabinet ministers-, technical executive & technical direction-, and local local government and communities representatives. The Technical and Planning Secretary of the Presidency conducts the General Coordination of the program. Impact Evaluation of Comunidades Solidarias Rurales The impact evaluation procurement was an international process, and had to consider experienced international institutions in partnership with a local one. The purpose of this specification was to shift capacity and knowledge of impact evaluation methodologies to the country. The impact evaluation included seven reports: Initial report 2007: Strategy and methodology for the Quantitative and Qualitative evaluations Second report 2008: Baseline results Third report 2008: Targeting Efficacy incidence of inclusion & exclusion errors Fourth report: Impact evaluation after the 1st year of implementation. Fifth report mid 2009: Participants satisfaction with the program Sixth report beginning of 2010: Impact evaluation after the 2nd year of implementation. Last report end of 2010: Program s Sustainability The quantitative evaluation considers Regression Discontinuity Design as the methodology that best fits the program s characteristics. The strategy use for targeting the program determined the selection of the evaluation methodology. 4
The qualitative evaluation had several stages, initiated end of 2008 until mid-2010. The main topics considered were the participant s satisfaction with the program and services provided, and the effect of the quality of services in the results. Different methods were applied, such as interviews, focal groups, studies ethnographic cases of communities and households, and observation of the service provided in health units and schools. The evaluators, Fundación Salvadoreña para el Desarrollo Económico y Social (FUSADES) in collaboration with the researchers of the International Food Policy Research Institute (IFPRI) collected the primary data set. The baseline survey was conducted in January and February 2008; the second survey between September and November 2008; the third between October and November 2009; and the fourth between August and September 2010. Evaluators collected the main database and they use also the Population Census 2007. To identify the program s participants households they use the programs census. The sample size consisted of 50 municipalities, 100 canton, and 2775 effective households (30 households per canton). Also includes some households with children 6 to 12 years, and children under 3 years or pregnant women, in order to evaluate the impact on several indicators of infant and maternal health, education, and nutritional status. Program s Sustainability The last report identified some aspects that can give sustainability to the program s evaluation results. These can be summarized as follows: women s empowerment, behavioral change in favor of preventive health; greater awareness of the importance of a balanced nutrition; and a positive attitude towards education between parents and children. Main conclusions and challenges 5
The impact evaluation as a strategy has provided in-depth information on the efficacy and importance of social protection programs, such as Comunidades Solidarias. It has also contributed for program administrators and policymakers to support the program and to give continuity in a new government. International cooperation established a fund for the program s strengthening and improvement. The importance of a rigorous impact evaluation of Comunidades Solidarias in El Salvador has contributed to incorporate evaluations of main social programs as an essential policy instrument. Comunidades Solidarias was the main pillar in which the Social Protection System was developed. Efforts and attainments in social and human development aspects in the country can easily backlash by external shock, particularly with climate related events; so the importance of the implementation of the Social Protection System is fundamental. One of the main challenges is the need for the social protection schemes to transit from a conceptual and theoretical proposal to a set of policy instruments that can respond to a variety of events life cycle and others related to external shocks. Sample size and main impact evaluation results presented in the Program s Sustainability Report 6
7
8
References Ley de Desarrollo y Protección Social de El Salvador. http://www.asamblea.gob.sv/eparlamento/indice-legislativo/buscador-dedocumentos-legislativos/ley-de-desarrollo-y-protecion-social Extreme Poverty Map (2005). http://www.fisdl.gob.sv/temas-543/mapa-de-pobreza Various Impact Evaluation Reports of Comunidades Solidarias presented to the Government of El Salvador by IFPRI/FUSADES, 2007-2010. Related articles of the program s evaluation: IFPRI, Alan de Brauw Using the Regression Discontinuity Design with Implicit Partitions (2011); Can Conditional Cash Tranfers Improve Maternal Health and Birth Outcomes? (2011); Migration and child development during the food Price crisis in El Salvador (2011). Tatiana Feltosa de Britto del 2007 The Challenges of El Salvador s CCT Programme, Red Solidaria Other articles related to CCT programs and Social Protection: Blase Kornacki. Conditional Cash Transfers: Progress Towards The Millennium Development Goals http://digitalcommons.wcl.american.edu/cgi/viewcontent.cgi?article=1257&context= sdlp Laura B. Rawlings and Gloria M. Rubio. Evaluating the Impact of Conditional Cash Transfer Programs. World Bank Research Observer Volume 20, Issue 1 Published: March 2005. Pages: 29 56 A draft versionhttp://www1.worldbank.org/prem/poverty/ie/dime_papers/195.pdf The World Bank (2008). The design and implement of effective safety nets for protection and promotion. Margaret Grosh, Carlo del Ninno, Emil Tesliuc, and Azedine Ouerghi. The World Bank. Washington D.C., 2008. http://siteresources.worldbank.org/splp/resources/461653-1207162275268/for_protection_and_promotion908.pdf John Blomquist. Impact Evaluation of Social Programs: A Policy Perspective Revised Draft, September 2003 9
http://siteresources.worldbank.org/safetynetsandtransfers/resources/evalu ation_blomquist.pdf Social Protection within Development Policy 4 Source: The World Bank. 2008. (Figure 2.1) 4 Source The World Bank 2008. Resilience refers to preventing drops in well-being from a range of shocks and key sources of resilience are social insurance programs. Equity refers to protection against destitution. Social assistance programs including cash transfers and targeted food assistance serve an especially important role in supporting this goal. Opportunity refers to promotion of better health, education, skills development and helping men and women to secure better jobs. Interventions that support this goal include conditional cash transfers to incentivize health and education investments and labor market programs that provide unemployment benefits, build skills and enhance workers productivity and employability. 10