CPSCR Review Independent Evaluation Group

Transcription

1 CPSCR Review Public Disclosure Authorized 1. CPS Data Country: Guatemala CPS Year: FY09 CPS Period: FY09 FY12 CPSCR Review Period: FY09 - FY12 Date of this review: September 18, Executive Summary i. This review examines the implementation of the FY09-12 Guatemala Country Partnership Strategy (CPS) and the CPS Progress Report (CPSPR) of FY11, and evaluates the CPS Completion Report (CPSCR). The strategy was jointly implemented by IBRD and IFC, and this review covers the program of the two institutions. ii. The broad objective of the CPS was to provide Guatemala with demand-driven and focused assistance to achieve key development goals, which include reducing poverty and inequality by expanding opportunities for the poor, particularly the most vulnerable, via sustainable and broadbased growth. The CPS objectives were organized under three pillars. Under Pillar 1, the strategy sought to enhance the fundamentals by promoting macroeconomic stability and expanding the fiscal space for social expenditure and by enhancing governance and transparency. Under Pillar 2, the WBG aimed to promote sustainable growth and productivity through an improved investment climate, rural development and productivity, sustainable energy, and natural disaster risk management. Under Pillar 3, the strategy endeavored to expand the opportunities for vulnerable and at-risk groups, which included increasing the effectiveness of the Conditional Cash Transfer (CCT) program, expanding access to quality education among the poor and indigenous people and improving and expanding access to health and nutrition services, with a special focus on maternal and child health. The CPSPR reaffirmed the CPS objectives and the three pillars, but sharpened the focus on countercyclical crisis responses. iii. IEG rates the overall outcome of the Guatemala CPS as Moderately Satisfactory, the same as the CPSCR s self-rating. Under Pillar 1, the Bank s interventions in the macro area were highly relevant and effective. The Bank s resource transfers and Bank-supported tax administration reforms played an important role in helping Guatemala maintain macroeconomic stability while substantially increasing social spending. The intended progress on tax policy reforms, however, was achieved only at the end of the CPS period. The Bank s interventions in the governance and transparency area were of only modest relevance in the Guatemalan context since outcomes in these areas were determined primarily by other factors. The corresponding outcomes thus fell short of expectations. Under Pillar 2, WBG support for investment climate reforms, while relevant and instrumental to meeting the specific outcome targets, had limited impact on the overall investment climate due to its modest scope and to the lack of synergies realized. WBG interventions in rural development and productivity focused on the highly relevant issue of rural market access (including rural roads). While more rural marketing arrangements were established, there is no evidence that they have become effective in accelerating rural development and productivity. Similarly, Bank support to improve investment allocation at territorial level has resulted in a large number of local development plans, but there is no information on whether these plans have led to improved investment allocation. In natural disaster risk management, Bank support contributed to strengthening the country s risk management framework, thereby reducing vulnerability. Under Pillar 3, the Bank s CCT Institutional Establishment loan was first halved in size, then cancelled, rendering the Bank an ineffective CPSCR Reviewed by: Peer Reviewed by: CPSCR Review Coordinator Robert J. Anderson S. Ramachandran Xiaolun Sun Consultant, IEGCC Consultant, IEGCC Senior Evaluation Officer, IEGCC Surajit Goswami Consultants, IEGCC Xiaolun Sun, Senior Evaluation Officer, IEGCC

2 CPSCR Review 2 partner in helping Guatemala strengthen its CCT program. Bank interventions that focused on improving access to education and on maternal and infant health services targeted the populations that have historically been the most difficult to reach, primarily indigenous Guatemalans. They were highly relevant to expanding opportunities for the vulnerable, and contributed to improvement in education and health results at both country and target population levels. iv. IEG concurs with the lessons drawn in the CPSCR, which are grounded in WBG s implementation experience in Guatemala. The most important of these is the need for realism in the assessment of the prospects for results, with a much sharper focus on the degree of ownership of various stakeholders and the effects this is likely to have on prospects for policy reform and institutional development. The political economy factor overwhelms any of the design and implementation factors under the WBG s control. In addition, IEG notes that in spite of the strong efforts by IBRD and IFC teams to seek synergies and complementarities and the fact that IFC investments amounted to nearly half of IBRD lending and contributed, directly or indirectly, to many CPS objectives, the assigned role of the IFC in the joint strategy remained relatively marginal and most of its contributions are not captured by the results framework. 3. CPS Summary Overview of CPS Relevance: Country Context: 1. Guatemala is Central America s largest economy, accounting for 34 percent of the region s population and 27 percent of its GDP. Favorable external conditions, stable macroeconomic conditions and important reforms since the 1996 Peace Accords led to a period of steady growth averaging 3.8 percent during and to a reduction of the poverty rate to 51 percent of the population. The CPS period, however, saw a series of negative shocks, including the global financial, food and fuel crises in 2008, and a string of natural disasters (severe drought, volcano eruption and repeated tropical storms) in The authorities adopted countercyclical policies and increased social expenditures to mitigate the impact of the crisis and natural disasters. Real GDP growth declined to 0.5 percent in 2009, the lowest in two decades, before recovering to 2.9 and 3.9 percent in 2010 and 2011, respectively. The government that took office in January 2012 has expressed a commitment to prudent macroeconomic, pro-business policies. While the near-term economic outlook is generally positive, much remains to be done in advancing structural reforms to generate higher growth rates and in addressing the challenges of rising crime and violence. 2. The government s development strategy for most of the CPS period (FY 09 through the first half of FY12) is set out in the Plan De La Esperanza, whose central theme is inclusive growth and the reduction of poverty and inequality. The plan is comprehensive and envisages multi-sectoral interventions organized under four thematic objectives: solidarity (social development and municipal development); governance (security and rule of law, democratic development, and legislative policy); productivity (economic development and risk management and natural disaster prevention); and regionalism (regional integration). A new development program, Agenda del Cambio: Plan de Gobierno , was adopted by the government that took office in It has objectives broadly similar to the previous plan, but with more emphasis on the fundamental role of governance, rule of law, and security in reducing poverty. Objectives of the WBG Strategy: 3. The broad objective of the CPS was to provide Guatemala with demand-driven and focused assistance to achieve key development goals, which include reducing poverty and inequality by expanding opportunities for the poor, particularly the most vulnerable, via sustainable and broad- based growth.

3 CPSCR Review 3 4. The CPS objectives were organized under three pillars. Under Pillar 1, the strategy sought to enhance the fundamentals by promoting macroeconomic stability and expanding the fiscal space for social expenditure and by enhancing governance and transparency. Under Pillar 2, the WBG aimed to promote sustainable growth and productivity through an improved investment climate, rural development and productivity, sustainable energy, and natural disaster risk management. Under Pillar 3, the strategy endeavored to expand the opportunities for vulnerable and at-risk groups, which included increasing the effectiveness of the Conditional Cash Transfer (CCT) program, expanding access to quality education among the poor and indigenous people, as well as improving and expanding access to health and nutrition services, with a special focus on maternal and child health. 5. The CPSPR reaffirmed the CPS objectives and the three pillars, but to respond to new realities in the country, sharpened its focus on countercyclical crisis responses. Relevance of the WBG Strategy: 6. Congruence with the Country s Context and Program. The WBG strategy focused on key impediments to inclusive, poverty reducing growth in Guatemala and was well-aligned with the government s program. It encompassed sub-objectives that, while stated in very broad terms, are highly relevant in principle to the attainment of faster overall growth and to shifting the structure of this growth toward the poor. It undertook to be selective, emphasizing areas in which the Bank had some comparative advantage and to leverage its program through coordination with other partners. There are, however, some areas which prior work and/or country experience also showed to be critical to poverty reduction that were not explicitly addressed by the strategy, including high fertility rates and rapid population growth, security and justice, and environmental protection. It is unclear from the CPS whether or how these challenges were specifically addressed by Guatemala s development partners over the CPS period. 7. Relevance of Design. The design of the CPS program, including the results framework linking WBG interventions to the sub-objectives of the strategy, is of varying degrees of relevance and realism. Some interventions, such as those proposed by the WBG to promote macroeconomic stability and to expand the fiscal space for social spending, were of sufficient size and coherence to effect desired outcomes. Others, like the proposed Bank analytical and advisory assistance (AAA) and IFC technical assistance (TA), while being able to bring about the narrowly defined outcomes, had limited capacity to achieve the objective of improving Guatemala s investment climate. Strong commitment and ability to mobilize support by the government were major assumptions for all WBG interventions targeted at public sector reforms. In the area of governance and transparency, in particular, commitment of politicians and bureaucrats to improvements was limited, rendering the kinds of systems and process interventions supported by the Bank of less relevance from a practical standpoint. The CPSPR made a strong effort at adjusting the WBG program to new realities given implementation delays and political deadlocks. The adjustments increased the relevance of the program in face of unanticipated natural disasters and under-estimated political difficulties. The results matrix, which clearly showed the link between WBG activities, desired outcomes, and CPS objectives, had measurable outcome indicators. However, some of the outcome indicators resemble strongly output indicators (e.g. number of days to get a construction permit vs. relevant competitiveness index; number of supply chain partnerships and beneficiaries vs. increases in rural production and income) and could not be effectively used to measure attainment of CPS objectives. The results matrix was revised through the CPSPR to include more realistic outcome targets. On the other hand, only one IFC activity was captured by the CPS results matrix. It is thus not an adequate tool for monitoring whether and to what extent IFC investments and advisory services contributed to the achievement of CPS objectives. 8. Relevance of IFC program. Complementing the IBRD program and responding to market demand, the IFC targeted private sector growth to help reverse the downward trend of GDP growth. Given IFC s strategic focus on financial sector development, it provided countercyclical financing by making credit available, especially for trade-related transactions, at a time of global credit shortages

4 CPSCR Review 4 and recessions in Guatemala s key trading partners (US and Mexico). It is not clear, however, how IFC s investments in diverse sectors (sugar, ethanol, electricity, paints, mobile phone, and supermarkets) and its municipal advisory services (mass transit, public lighting, financial management, and business regulation) hang together as a coherent program that supported the CPS. 9. Risk Identification and Mitigation. The CPS correctly identified the major risks, including economic deceleration, political deadlock, crime and violence, natural disasters, and program performance as a result of larger, more complex operations. For each type of risk, it had a series of mitigation measures, and concluded that [t]he risks associated with implementation of the CPS are moderate. CPSPR added reversal of social gains as another risk stemming from the crisis, and proposed to coordinate closely with other donors and further expand a health sector project to counter this risk. It also considered that the economic, political, and insecurity risks were heightened, while those associated with natural disaster and program performance had decreased. As it happened, risk factors dominated the implementation of the strategy and the outcomes achieved. While the CPS correctly identified the major risks, it underestimated the probabilities that these would materialize and overestimated the efficacy of its mitigating measures. Overview of CPS Implementation: Lending and Investments: 10. IBRD delivered six loans totaling US$881.5 million. This represented 90 percent of the planned lending volume in CPSPR, with the difference explained by a cancelled development policy loan (DPL) of US$200 million and the inclusion of emergency support for social services in 2011 of US$100 million. Two DPLs (US$200 million and US$350 million) were delivered to support tax reforms and improve the quality of financial institutions. As such, a large part of WBG lending focused on helping the government maintain macroeconomic stability and social spending. Two loans amounting to US$120 million were programmed for sustainable growth and productivity in the CPSPR and delivered on schedule. Lending support in the social sectors was nearly halved to US$114.5 million in the CPSPR when Congress balked at authorizing a loan on expanding opportunities for vulnerable groups. An emergency loan of US$100 million was added and approved in FY11 to support the maintenance of social services in the wake of multiple natural disasters in In addition, eight previously approved lending projects totaling US$323.1 million were on-going during the CPS, of which five remain active at the end of the period. Trust funds provided another US$14.7 million to complement IBRD resources. 11. With the exception of FY09, between 40 to 60 percent of IBRD s Guatemala portfolio was at risk during the CPS period, which was much higher than the Latin American and world average of between 10 to 20 percent (by commitment amount). At the end of FY12, seven projects were under implementation, representing net commitments of US$400 million. This meant more than half of the portfolio was undisbursed. Disbursements of the trust fund portfolio were also lagging (65 percent undisbursed). The CPSPR and CPSCR attribute this weak performance to a number of political and institutional factors, including the need to develop new pro-poor sector strategies, the learning requirements of the new administration, high levels of turnover of government officials and Project Coordination Units (PCUs), delays in Congressional approval and in implementation due to the dynamics of the budgeting process, unresolved regulatory issues, and the need for the administration to focus on intermittent crises and disasters. 12. In terms of the quality of the operations, five IBRD projects closed during the CPS period. IEG reviewed the Implementation Completion Reports (ICRs) of these projects, and rated all of them moderately satisfactory or above. However, IEG considered that risk to development outcomes was significant in three of these projects, high in one case, and moderate in the other. Of the six on-going projects, project teams rated one to be satisfactory, three moderately satisfactory, and two moderately unsatisfactory.

5 CPSCR Review IFC committed US$422 million to 14 projects, with six partners, in a variety of sectors. Eighty percent of the investment was in financial markets, including the two largest IFC investments in Guatemala during this period which had a total value of US$250 million. These investments were committed in 2009, when private capital inflows to Guatemala was negative, to finance trade-related transactions at two banks. In addition, seven IFC investment projects were in operation at inception of the review period, for US$107 million of net commitment. 14. IEG rated four of IFC s investment projects that were active at inception of the CPS period and all were rated positively for development effectiveness. For the IFC portfolio approved during the CPS period, supervision teams reports suggest that they are enjoying business success and are also leading to positive development effectiveness. Analytic and Advisory Activities and Services 15. IBRD delivered eight pieces of economic and sector work (ESW) and five TA tasks. As the Bank s AAA activities were supposed to be flexible and adjusted yearly to meet country demands, it is not surprising that some planned activities were dropped and new ones added as new priorities emerged. Such adjustments were particularly frequent with the Bank s TA services, where four of the six planned activities were cancelled; one was combined with a lending preparation; while four nonplanned TAs were delivered. The CPSCR noted some deficiencies in the dialogue and dissemination aspects of AAA deliveries, and suggested that more efforts to stimulate debate and facilitate exchange of ideas on key priorities in the area of governance and competitiveness would enhance the impact of the Bank s AAA work. 16. IFC completed two advisory services (AS) operations that were active at inception of the CPS period and approved two new projects, one of which has been completed by the end of the CPS. There is no IEG review of these projects, but project supervision and/or completion reports showed that only the 10-year long IFC Global Environment Facility (GEF) TA on environmental issues had been successful, while the other three that focused on strengthening municipalities were either unsuccessful, or mostly unsuccessful, or on hold. In addition, IFC also started four more AS projects during this period, but none of them proceeded beyond the pre-implementation stage. IFC s lack of success in the delivery of advisory services in Guatemala was balanced to some extent by a particularly innovative and successful regional AS project addressing farmer certification and productivity in the coffee sector in several Central American countries, including Guatemala. Partnerships and Development Partner Coordination 17. The CPS, CPSPR, and CPSCR report Bank participation in a number of partnership coordination processes, such as the partnership agreement to follow up on the Paris Declaration, monthly meetings on education, health and rural development, as well as coordination for sector-wide issues. The Bank partnered with the Inter-American Development Bank in a number of operations, such as the joint technical support to the Mi Familia Progresa social program and co-financing for the Rural Economic Development Project. The Bank also has a clearly defined collaboration with the International Monetary Fund (IMF) on macro and structural policies. Safeguards and Fiduciary Issues: 18. exchanges with the Guatemala Country Team revealed two safeguards issues. The first concerned involuntary resettlement when secondary roads were being constructed under the Second Rural Roads Project. The Bank s Involuntary Resettlement Operational Policy (OP 4.12) was not included in the Loan Agreement and not followed. The closing date of the project has been extended to September 2012 to allow sufficient time to complete compensation as required by Bank policies. The second occurred with regard to access to natural resources for people living in the vicinity of a

6 CPSCR Review 6 Land Administration Project area. At appraisal, such risk was considered negligible, and OP 4.12 was not triggered. However, even though the risk did not change throughout project implementation, the application of OP 4.12 was advocated in an effort to strengthen the Borrower's capacity to understand and manage potential cases of involuntary restriction of access. The project has been restructured to conform to OP 4.12 in line with similar projects in the region. 19. The WBG Integrity Vice Presidency (INT) undertook two investigations during the period. Both were related to the Second Rural Roads project. The first investigation concluded with the recommendation to exclude two companies from bidding; the second is ongoing. 20. The CPSCR notes that the CCT program, which the Bank supported through TA, direct lending, as well as general budget support, has been criticized in Guatemala for alleged links to electoral politics, for stimulating more demand for health and education services than can be met, and for a lack of transparency in payments. The CPSCR does not evaluate these criticisms, but notes a December 2010 study that placed the country s CCT program 9 th out of 16 Latin American countries in terms of transparency and access to information. Overview of Achievement by Objective: Pillar I: Enhance Fundamentals 21. Objectives under this pillar included promoting macroeconomic stability by expanding fiscal space to fund priority social spending and strengthening financial sector supervision and risk management; and enhancing governance and transparency. 22. Promoting Macroeconomic Stability. Guatemala successfully met the macroeconomic management challenges of the CPS period. Growth, despite the sharp drop in 2009, averaged over 3 percent during this period. Inflation spiked in 2008, but returned quickly to the 5 6 percent range and shows a recent tendency to decline. External current account widened from near balance in 2009 to a deficit of 3.1 percent of GDP in But as private capital inflows recovered, the overall balance of payment remained in surplus. Central bank interventions were countercyclical and followed a rulesbased framework. The fiscal deficit increased to over 3 percent of GDP in as the government increased spending to offset the effects of the crisis and for post-disaster reconstruction. It came down to 2.4 percent in 2011 and is projected to decline further. Non-financial public sector debt increased from 20 to 25 percent of GDP during the CPS period, but IMF projections foresee debt stabilizing at 27 percent of GDP by 2017 under expected policies. The financial system has held up well during the crisis, apparently without government financial support, with sound bank capitalization and liquidity, declining non-performing loans and improving asset quality. Nevertheless, the IMF review of Guatemala s exceptional access to Fund resources under the 2009 precautionary stand-by arrangement found that the banking system s prudential and legal framework were not strengthened as envisaged, as political resistance in Congress thwarted the approval of amendments to banking legislation and regulations. 23. IBRD supported this objective with the enlarged DPL series, a financial sector TA loan, and AAAs. Net disbursements of the DPLs increased substantially from US$66 million in 2008 to US$306 million in 2009 and US$$260 million in Support would have been better synchronized with the growth slump if the Congress had approved DPL II more expeditiously. Although Bank support did not lead to expanded fiscal space as tax reforms stalled, it provided financing for increased social spending and contributed to improvements in tax administration. In the financial sector, IBRD DPLs, lending and non-lending TAs supported strengthening financial sector supervision, while IFC contributed directly to banking sector stability by providing US$78 million of capital to one bank and Tier 2 capital of US$30 million to another.

7 CPSCR Review Enhancing Governance and Transparency. Available indicators of governance and transparency provide little evidence of improved country outcomes in these dimensions. The International Country Risk Guide indicator of Bureaucratic Quality is unchanged since 2002; the Global Competitiveness Report indicator on Wastefulness of Government Spending decreased between 2008/09 and 2011/12, so did its index of Diversion of Public Funds ; the Worldwide Governance Indicators indicator of government effectiveness shows a slightly lower performance between 1996 and 2010, but its indicator for corruption suggests some improvement during the same period. 25. IBRD assistance in this area consisted of the aforementioned DPL series and the financial sector TA loan. The outcomes sought in the CPS reflected the objectives of these operations, which was to promote the development and application of budget control systems and processes to improve the transparency of government spending decisions. By and large, these were achieved. Progress was made in strengthening the government s integrated financial management system and extending its application to additional areas of public spending, and to successful pilot initiatives designed to improve the transparency of government expenditures on construction and pharmaceuticals. However, this system and process improvement has not translated into enhanced governance and transparency. Among the various obstacles, a lack of commitment by key actors to use these tools to improve governance and transparency was the binding constraint. This essentially raises the question about the relevance of a formal systems and processes approach in the previous and current Guatemalan governance context. This issue is not unique to Public Financial Management reform in Guatemala, as demonstrated in the 2011 World Development Report. 26. IEG rates the outcome of WBG assistance under Pillar I as moderately satisfactory. The Bank s interventions in the macro area were highly relevant and effective. Guatemala maintained macroeconomic stability while substantially increasing social spending. Bank resource transfers and Bank supported tax administration reforms played an important role in this outcome, although the intended progress on tax policy reforms to provide more resources for social and infrastructure spending on a sustained basis was achieved only at the end of the CPS period. The Bank s interventions in the governance and transparency area were conceptually relevant, but in the Guatemalan context of only modest relevance since outcomes in these areas were determined primarily by other factors. Governance and transparency outcomes thus fell short of expectations. Pillar II: Promote Sustainable Growth and Productivity 27. Under this pillar, strategy objectives were to improve the investment climate, promote rural development and productivity, and enhance natural disaster risk management. 28. Improving the Investment Climate. Over the CPS period, the political environment and rising crime and violence have negatively impacted the investment climate. The Global Competitiveness Index shows that although Guatemala s overall competitiveness improved somewhat between 2008/09 and 2011/12 (ranked 84 th in both periods as country coverage increased from 134 to 142), the score and ranking of its institutions dropped substantially, while that of the burden of government regulations worsened slightly. Guatemala s overall Ease of Doing Business indicator also slipped from 93 rd to 97 th place among 183 economies between 2011 and WBG support comprised an Investment Climate Assessment, a Country Economic Memorandum (CEM) that focused on small and medium enterprises (SMEs), and an IFC advisory services project on construction licensing in Guatemala City. Even though the CPS outcome indicator (reduction of number of steps to get construction license) was fully achieved, Guatemala s ranking in Doing Business Dealing with Construction Permit indicator dropped seven places during The CPSCR also notes the establishment of a customs information system that was supported by the DPL series, which, according to the ICR for the series, contributed to a reduction in customs clearance times and the enactment of legislation to permit use of movable assets as collateral to expand credit

8 CPSCR Review 8 to the private sector. These achievements were not captured by the CPS results framework, but are confirmed by relevant Doing Business indicators Guatemala ranks 8 th globally for Getting Credit, and improved its (still poor) Trading Across Border ranking by two places. A new lending operation that focuses on SME development was approved in FY11, using the CEM as input, but it awaits Congressional approval. 30. Rural Development and Productivity. The CPS results framework does not directly track progress in rural development and productivity as would be reflected in, for example, increases in rural production and income, and/or increases in total sales of rural productive supply chain partnerships (which is a Project Development Objective indicator of the Rural Economic Development Project), either at country or regional level. Instead, CPS outcomes focused on improving rural institutions (marketing arrangements, development plans) and rural roads that would contribute to rural development and productivity enhancement. At the aggregate level, agriculture s contribution to Guatemalan GDP increased from 12 to 12.9 percent between 2008 and IBRD supported this objective with two loans approved before the CPS and an ESW on land tenure. The Rural Economic Development Project (PDER) underpinned the CPS outcomes on improved farmer access to markets, increased coverage of rural telecommunication, and improved investment allocation at local level, while the Second Rural and Main Roads Project was to increase road access for targeted populations. Overall, although these projects have produced satisfactory outputs in terms of the number of marketing arrangements and development plans through a participatory approach, evidence for concrete results of increased market access and improved investment allocation is lacking. The component on rural telecommunication was cancelled at the request of the government; an Impact Evaluation is on-going to provide information on road access for targeted population. 32. IFC contributed to this objective through investments in sugar and ethanol production, creating jobs for low-income manual laborers and helping the client improve productivity and competitiveness. Through an ongoing partnership with one of the world s biggest coffee traders, a regional IFC advisory services project contributed to the development of the skills of 245 small-scale farmers in sustainable agriculture, helping them meet international certification standards, improve the sustainability and productivity of their crops, and increase their earnings. In addition, IFC investments in the commercial banks and trade guarantees contributed to this objective by increasing the range of financial services for rural producers and households. Other IFC investments, including pre-fy09 commitments in retail construction, mass transit development, as well as new commitments during FY09-11 in paints manufacturing, and mobile telephony, aimed to contribute to sustainable growth and productivity more broadly. The specific outcomes achieved, however, are unknown and not monitored by the CPS results matrix. 33. Natural Disaster Risk Management. The CPSCR reports that Guatemala has implemented the National Program for Disaster Reduction , although there is no detail on what key changes have been brought about. The CPS results framework does not directly assess Guatemala s progress in reducing its vulnerability to adverse natural events (e.g. reduction in annual expected value of damages due to natural disasters, shifts in the probability distribution of physical, social, and economic damages), but Box 1 (Building capacity to respond to national calamities) provides a broader explanation that vulnerability has in fact been reduced. The CPSCR notes that the new administration is developing a National Strategy for Natural Disaster Mitigation and that key government agencies are establishing a dialogue to define a process aimed at developing a comprehensive disaster financing strategy. 34. WBG provided a number of loans and non-lending TAs to support this objective. The Catastrophe Development Policy Loan Deferred Drawdown Option (CAT-DDO) and bankadministered trust funds supported the implementation of the National Program for Disaster Reduction , focusing on the development of an institutional framework for ex-ante planning and

9 CPSCR Review 9 mitigation and ex-post disaster response. It was also clearly the intention of the Bank to provide financing for natural disaster rehabilitation during the period of the CPS, which was achieved through the CAT-DDO, the Emergency Support to Social Services Project, the Rural Economic Development Project, and through trust funds administered by the Bank. Delivery of the Bank s emergency support in the social services area was delayed, however, by the Guatemalan Congress, which held up passage of the legislation authorizing this loan for six-months. The Bank also provided a number of TAs on damage and loss assessment. 35. IEG rates the outcome of Bank assistance under Pillar 2 as moderately satisfactory, but marginally so. WBG support for investment climate reforms, while relevant and instrumental to meeting the specific outcome targets, had very limited impact on the overall investment climate in Guatemala due to its modest scope and lack of synergy. WBG interventions in rural development and productivity focused on the highly relevant rural market access (including rural roads). While more rural marketing arrangements were established, there is no evidence that they have become effective in accelerating rural development and productivity. Similarly, Bank support to improve investment allocation at territorial level has resulted in a large number of local development plans, but there is no information on whether these plans have led to improved investment allocation. As such, both the relevance and efficacy of Bank interventions are modest. In natural disaster risk management, Bank support on ex-ante preparation and ex-post response was relevant and effective, contributing to strengthening the risk management framework and thereby reducing vulnerability. Pillar III: Expanding Opportunities (Social Cohesion Program) 36. The objectives under Pillar 3 were to increase the effectiveness of the CCT program, to expand access to quality education among the poor and indigenous people and to improve access to health and nutrition services, with a focus on maternal and infant health. 37. Increasing effectiveness of the CCT program. The initiation and expansion of the CCT program was an important aspect of the strengthening of Guatemala s social protection system. The revised CPSPR results framework, however, includes only education achievement in target localities as an outcome indicator to measure the effectiveness of this objective political realities made it impossible to advance the agenda of targeting the program on the poor indigenous population. The CPSCR reports that the program had a high conditionality compliance rate of 85 and 90 percent in 2009 for education and health, respectively; however, in the absence of baseline data, but it is not clear how much improvement these figures represent. 38. IBRD supported this objective with the Expanding Opportunities: CCT Institutional Establishment loan, which was subsequently cancelled with no disbursement. 39. Expanding Access to Quality Education among the Poor and Indigenous. The CPSCR provides substantial evidence of progress toward improved educational outcomes among targeted populations. Between 2007 and 2011, completion rates in primary schools rose from 67 to 81 percent in Guatemala s 196 poorest municipalities, slightly less than the 84 percent targeted for 2012, while the lower secondary Gross Enrollment Rate (GER) increased from 41 to 54.1 percent, exceeding the target of 51 percent by Enrollment of targeted indigenous groups in 9 th grade increased by 70 percent between 2007 and 2011, far above the targeted increase of 25 percent. The success was due in part to the opening of targeted telesecundarias (television-based delivery of instruction) in poor areas, which benefited especially indigenous communities. However, all these indicators are access indicators. There is no information about the quality of the education provided. 40. The contribution of the Bank to this objective was delivered through the Education Quality and Secondary Education Project approved in FY07 and the CCT. The former provided a mix of TA outputs, training and scholarships for low income students about 45 percent of the project cost was

10 CPSCR Review 10 for access subsidies; the latter provided demand incentives for education and health services and financing. A new loan, the Emergency Support to Social Services, provided quick disbursing financing to help maintain basic education and health care services in the country s poorest municipalities in the aftermath of the 2010 natural disasters. None of the planned non-lending services supporting this objective were delivered, although a Regional Social Services Study was completed and a TA on improving social services was active at the end of the CPS period. 41. Improving and Expanding Access to Health and Nutrition Services. The CPSCR provides substantial evidence of progress toward improved maternal and infant health outcomes among targeted populations. More broadly, World Health Organization data between 2002 and 2009 show significant improvements in child nutrition among Guatemalan children under five years of age the proportion of underweight children declined from 17.7 to 13.0 percent, of stunted growth from 54.3 to 48 percent, and of under 5 wasted from 1.8 to 1.1 percent. The UNDP Human Development Index shows a decline in under 5 mortality from 42 in 2007 to 40 in The Bank supported health objectives through the Maternal and Infant Health and Nutrition loan approved in FY06. Among the CPS outcomes, the institutional birth target has been surpassed; the indigenous child mortality target is not met, while the information on malnutrition among young children is pending a nutritional status survey. The project also achieved good results in a number of areas not tracked by the CPS results matrix, but nonetheless contributed to the objective of improving and expanding access to health services. 43. IEG rates the outcome of Bank support under Pillar 3 as satisfactory on the basis of the strong outcomes in the education and health sectors. With the CCT loan first halved, then cancelled, the Bank did not contribute directly to improving the effectiveness of the CCT program. The Bank s interventions on improving access to education and maternal and infant health services targeted the populations that have historically been the most difficult to reach, primarily indigenous Guatemalans. They were highly relevant to expanding opportunities for the vulnerable. By combining demand side components (scholarships, conditional cash transfers) and supply side components (TA, training), and adapting linguistically and culturally to the target populations, Bank support contributed to achieving significant improvement at both in country and target population levels. Objectives CPSCR Ratings IEG Rating Pillar I: Enhance Fundamentals N/A Moderately Satisfactory Pillar II: Sustainable Growth and Productivity N/A Moderately Satisfactory Pillar III: Expand Opportunities for the Vulnerable N/A Satisfactory 4. Overall IEG Assessment CPSCR Rating IEG Rating Overall Outcome: Moderately Satisfactory Moderately Satisfactory IBRD Performance: Moderately Satisfactory Moderately Satisfactory IFC Performance: Moderately Satisfactory Moderately Satisfactory Overall outcome: 44. The overall outcome of Guatemala CPS is rated moderately satisfactory. Under Pillar 1, the Bank s interventions in the macro area were highly relevant and effective. Bank resource transfers and Bank-supported tax administration reforms played an important role in helping Guatemala maintain macroeconomic stability while substantially increasing social spending. The intended progress on tax

11 CPSCR Review 11 policy reforms, however, was achieved only at the end of the CPS period. The Bank s interventions in the governance and transparency area were of only modest relevance in the Guatemalan context since outcomes in these areas were determined primarily by other factors. Governance and transparency outcomes thus fell short of expectations. Under Pillar 2, WBG support for investment climate reforms, while relevant and instrumental in meeting the specific outcome targets, had very limited impact on the overall investment climate in Guatemala due to its modest scope and lack of synergy. WBG interventions in rural development and productivity focused on the highly relevant aspect of rural market access (including rural roads). While more rural marketing arrangements were established, there is no evidence that they have become effective in accelerating rural development and productivity. Similarly, Bank support to improve investment allocation at territorial level has resulted in a large number of local development plans, but there is no information on whether these have led to improved investment allocation. In natural disaster risk management, Bank support contributed to strengthening the country s risk management framework and thereby reducing vulnerability. Under Pillar 3, the Bank s CCT Institutional Establishment loan was first halved in size, then cancelled, rendering the Bank an ineffective partner in strengthening its CCT program. The Bank s interventions on improving access to education and maternal and infant health services targeted the populations that have historically been the most difficult to reach, primarily indigenous Guatemalans. They were highly relevant to expanding opportunities for the vulnerable, and contributed to improvement in education and health results at both in country and target population levels during a time of multiple negative shocks. IBRD Performance: 45. The IBRD performance is rated moderately satisfactory. The strategy was flexible and responsive to the changing country context. However, as the CPSCR notes, the design of the strategy was not sufficiently mindful of the risks to the achievement of results, particularly in light of the Bank s track record in Guatemala. This is reflected in the varying degrees of relevance and realism across the Bank program. Interventions were highly relevant with regard to macroeconomic stability and disaster risk management objectives, but only modestly so with regard to investment climate. In general, programs under Pillar 2 and 3 were weaker than Pillar 1, as evidenced by lower program delivery rates and stop-go implementation of some interventions. Risks and the efficacy of mitigation measures were seriously underestimated. The results framework is generally appropriate and realistic, but could be improved with regard to choice of outcome indicators. Bank products and services appear to have been of satisfactory quality. However, the difficulties encountered by some lending projects suggests that there may be some design/appraisal process issues, particularly with regard to stakeholder ownership and risks in Guatemala s complex political environment. Bank supervision and management of the portfolio were proactive, but ultimately did not result in lasting improvement in the overall portfolio performance. The plan to develop a truly joint AAA program with the government and establish the WBG as a partner in the provision of knowledge services was not implemented. Coordination with other development partners appears to have been satisfactory. Attention to safeguard issues was adequate, although in retrospect the issues that arose might have been foreseen and forestalled. IFC Performance: 46. IFC performance is rated moderately satisfactory. IFC had a modest investment and advisory program. The strategy of building long-term partnerships with banks and agribusiness was consistent with Guatemala s priorities of maintaining financial sector stability and promoting private sector led growth, as well as with IFC s strategic focus on developing local financial markets and infrastructure, including the food supply chain. The strategy appears to have been very successful, as evidenced by the positive development effectiveness ratings by IEG to selected projects. However, the focus of IFC advisory services on strengthening municipalities, although a successful strategy in many other countries, did not work in Guatemala. Instead, advisory services aimed at strengthening the agricultural value chain, such as the regional program in coffee production, have met with greater success.

12 CPSCR Review Assessment of CPS Completion Report 47. The CPSCR is concise, candid, and comprehensive, critically assessing the achievement and weaknesses of the CPS program. It focuses on outcomes as they are defined in the CPS/CPSPR and summarizes WBG contributions to these outcomes. Coverage of implementation is mixed. The basic facts are there, but there is little analysis of the delays, cancellations and restructurings of some lending projects, or of possible underlying causes such as political or ownership factors. In several cases, there is little analysis of the linkage between WBG interventions and outcomes, which would have been especially useful where the outcomes are difficult to measure. The CPSCR could have dealt in more depth with areas of the program where things did not go well (such as in IFC AS projects with the municipalities) to establish the reasons for underperformance or failure to realize planned projects and to draw lessons for the future. There are also a number of undertakings in the CPS and CPSPR with regard to partnership coordination, portfolio management, and development of a joint AAA program (possibly with an element of reimbursement) that are not reported on in the CPSCR. 6. Findings and Lessons 48. The lessons drawn in the CPSCR are well-grounded in the WBG s implementation experience in Guatemala. One is of overarching importance: In considering the likelihood of achieving results, attention should go beyond formal government commitment to consider the factors that may block achieving the results, such as the difficult political economy in Guatemala. [CPSCR, para 40]. The political economy factor overwhelms any of the design and implementation factors under the WBG s control. Therefore, careful consideration of this factor is critical to the design and implementation of a WBG country partnership strategy that seeks to maximize the prospects for satisfactory development outcomes. 49. The CPS/CPSPR show strong efforts by the IBRD and IFC teams to seek synergies and complementarities. However, in spite of the fact that IFC investment amounted to nearly half of IBRD lending during the CPS period and contributed directly or indirectly to all three pillars, its role in the joint country strategy remained quite marginal, and its contributions are not captured by the results framework. For maximum development impact, it is important that the two teams collaborate to develop balanced joint country programs that give proper weight and corresponding responsibilities for the achievement of results.

13 CPSCR Review Annexes 13 Annex Table 1: Annex Table 2: Annex Table 3: Annex Table 4: Annex Table 5: Annex Table 6: Annex Table 7: Annex Table 8: Annex Table 9: Annex Table 10: Annex Table 11: Annex Table 12: Annex Table 13: Summary Achievements of CPS Objectives Actual and Planned Lending, FY09-12 Grants and Trust Funds Active in FY09-12 Planned and Actual Analytical and Advisory Work, FY09-12 IEG Project Ratings for Guatemala, FY09-12 IEG Project Ratings for Guatemala and Comparators, FY09-12 Portfolio Status for Guatemala and Comparators, FY09-12 IBRD Net Disbursement and Charges Summary Report for Guatemala (in US$ million) List of IFC s investments in Guatemala that were active during FY09-12 (US$ 000) List of IFC s Advisory Services in Guatemala, FY09-12 Total Net Disbursements of Official Development Assistance and Official Aid, (in US$ million) Economic and Social Indicators for Guatemala and Comparators, Guatemala - Millennium Development Goals

14

15 Annexes CPSCR Review Annex Table 1: Summary of Achievements of the CPS Objectives 15 CPS 09-12: Pillar 1 Enhancing Fundamentals Objectives 1. Promote Macroeconomic Stability and Expand Fiscal Space 2. Governance and Transparency Major Outcome Measures Maintain fiscal deficit below 3.0% of GDP, on average, for the CPS period as priority social spending increases. (1.4% in 2007) (a) Maintain tax collection above 11.0% of GDP, on average, over the CPS period (12.3% in 2007) (b) Tax revenues recover to at least 11.7% of GDP. Improve the supervision of financial conglomerates: (i) Complete implementation of enhanced supervision or regular supervision of the majority of risk profiles under the new manual of risk based supervision. (ii) Consolidate supervision completed in all financial groups. (i) The government has introduced the results-based budgeting methodology in at least three major sector agencies (i.e. the Ministries of Education, Health and Security) and has been able to monitor and evaluate progress and results. (ii) At least 70% of Central Government public bidding processes are published and carried out online through the new e-bidding module of GUATECOMPRAS. Actual Results (as of current month year) 1. Promote Macroeconomic Stability and Expand Fiscal Space Comments 2.7 percent ( ) Source: IMF, International Financial Statistics percent ( ) 11.8 percent in (i) Completed implementation of enhanced supervision or regular supervision of all risk profiles under the new manual of risk basked supervision. (ii) Consolidated supervision completed in 5 largest financial groups that represent about 75% of the financial assets of the system. Source: IMF, International Financial Statistics. Source: CPSCR 2. Governance and Transparency Source: GUATECOMPRAS portal and CPSCR (i) Results informed budgeting was implemented in all ministries for the formulation of 2011 budget. (ii)transactional capabilities of GUATECOMPRAS have been increased and more attributes are underway. An electronic bidding module for single source contracting modalities has been implemented; GUATECOMPRAS portal with linkages to SNIP and CDP has been enhanced; and the vendors registration module has been implemented. Refinement of indicators for the draft 2012 budget has prioritized the identification and inclusion of at least two indicators (mission critical, measurable, and linked to top government priorities) in the ministries of Interior, Education, and Health. The draft budget specifically prohibits transfers from those programs that are supporting attainment of those indicators. The process is undergoing and the Government is committed to expand this methodology to other programs and sectors in a gradual manner. Strengthen municipal governments financial management (implement simplified financial management system all municipalities) (Baseline: 256 of 333 municipalities in 2008) 100% of municipalities are using SIAF systems as of June Source: P Integrated Financial Management III - Technical Assistance Project (SIAF III) and CPSCR. SICOIN GL implemented in 303 municipalities; 5 municipalities are using SICOIN WEB; and 25 are using SIAFMUNI. SICOIN WEB used by largest municipalities to report results of financial operations tracked by more sophisticated 24 PFM systems (e.g. SAP). SIAFMUNI still used by municipalities which prefer to maintain financial records within their own servers rather than SICOIN GL, a cloud-computing approach where such data is maintained within the MOF, even though all financial transactions are reported to the MOF and CGC.