IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-78120, TF-95841) ON A LOAN IN THE AMOUNT OF US$20 MILLION TO THE REPUBLIC OF EL SALVADOR FOR A

Transcription

1 Public Disclosure Authorized Document of The World Bank Report No: ICR Public Disclosure Authorized Public Disclosure Authorized IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-78120, TF-95841) ON A LOAN IN THE AMOUNT OF US$20 MILLION TO THE REPUBLIC OF EL SALVADOR FOR A FISCAL MANAGEMENT AND PUBLIC SECTOR PERFORMANCE TECHNICAL ASSISTANCE PROJECT March 10, 2017 Public Disclosure Authorized Governance Global Practice Latin America and the Caribbean Region

2 CURRENCY EQUIVALENTS The official currency unit in El Salvador is the U.S. Dollar FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS CPS DGA DGII DGP DGT DINAFI GDP GIZ IADB ICR ICT IMF ISR IT LDSW M&E MoF MTEF OIR PAD PDO PEFA PFM PIU SAFI SCPT SIDUNEA SIRH TTL UNAC Country Partnership Strategy Customs Agency (Dirección General de Aduanas) Internal Revenue Agency (Dirección General de Impuestos Internos) General Budget Office (Dirección General de Presupuesto) Treasury Office (Dirección General de Tesorería) National Financial Administration and Innovation Office (Dirección Nacional de Administración Financiera e Innovación) Gross Domestic Product German Agency for International Cooperation (Deutsche Gesellschaft für Internationale Zusammenarbeit) Inter-American Development Bank Implementation Completion and Results Report Information and Communication Technology International Monetary Fund Implementation Status and Results Report Information Technology Locally Developed Software Solution Monitoring and Evaluation Ministry of Finance Medium-term Expenditure Framework Office of Information and Response (Oficinas de Información y Respuesta) Project Appraisal Document Project Development Objective Public Expenditure and Financial Assessment Public Financial Management Project Implementation Unit Integrated Financial Management System (Sistema de Administración Financiera Integrada) Secretariat of Citizen Participation and Transparency Automated System for Customs Data Platform (Sistema Aduanero Automatizado) Human Resource Management System Task Team Leader National Procurement Office (Unidad Normativa de Adquisiciones y Contrataciones de la Administración Publica)

3 UNCTAD USAID United Nations Conference on Trade and Development U.S. Agency for International Development Senior Global Practice Director: Deborah Wetzel Practice Manager: Arturo Herrera Project Team Leader: Maria Guadalupe Toscano Nicolas ICR Team Leader: Joanna Watkins

4 EL SALVADOR Fiscal Management and Public Sector Performance Technical Assistance Project CONTENTS Data Sheet A. Basic Information... i B. Key Dates... i C. Ratings Summary... i D. Sector and Theme Codes... ii E. Bank Staff... ii F. Results Framework Analysis... iii G. Ratings of Project Performance in ISRs... v H. Restructuring (if any)... vi 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1. Project Costs and Financing Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Bank Lending and Implementation Support/Supervision Processes Annex 5. Beneficiary Survey Results Annex 6. Stakeholder Workshop Report and Results Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Annex 9. List of Supporting Documents Annex 10. Achievement of Objectives Analysis Annex 11. El Salvador PEFA Indicators 2009 versus MAP... 45

5 A. Basic Information Country: El Salvador Project Name: Fiscal Management and Public Sector Performance Technical Assistance Loan Project ID: P L/C/TF Number(s): IBRD-78120, TF ICR Date: 03/10/2017 ICR Type: Core ICR Lending Instrument: Original Total Commitment: Revised Amount: Environmental Category: C Implementing Agencies: Ministerio de Hacienda Technical Assistance Loan Borrower: REPUBLIC OF EL SALVADOR US$20.00 million Disbursed Amount: US$15.36 million US$20.00 million Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 09/10/2009 Effectiveness: 05/24/ /24/2011 Appraisal: 10/19/2009 Restructuring(s): 10/08/2014 Approval: 11/24/2009 Midterm Review: 02/11/ /11/2013 Closing: 12/31/ /30/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Unsatisfactory Moderate Moderately Unsatisfactory Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Moderately Quality at Entry: Government: Unsatisfactory Unsatisfactory Moderately Implementing Quality of Supervision: Unsatisfactory Unsatisfactory Agency/Agencies: Overall Bank Moderately Overall Borrower Unsatisfactory Performance: Unsatisfactory Performance: i

6 C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Performance any) Potential Problem Project at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: No Yes Unsatisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Original Actual Major Sector/Sector Public Administration Public Administration - Industry and Trade Public Administration - Financial Sector Central Government (Central Agencies) Major Theme/Theme/Sub Theme Economic Policy Fiscal Policy Tax policy Public Sector Management Public Administration Administrative and Civil Service Reform Transparency, Accountability, and Good Governance Public Finance Management Domestic Revenue Administration Public Expenditure Management E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Calderon Pamela Cox Country Director: J. Humberto Lopez Laura Frigenti Practice Manager/Manager: Arturo Herrera Gutierrez Nicholas Paul Manning Project Team Leader: Maria Guadalupe Toscano Nicolas Alberto Leyton ICR Team Leader: ICR Primary Author: Joanna Alexandra Watkins Joanna Alexandra Watkins ii

7 F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To strengthen the institutional capacity of specific government processes and agencies to increase the effectiveness and efficiency of revenue and expenditure management and enhance accountability and transparency in the public sector. Revised Project Development Objectives (as approved by original approving authority) Not applicable. (a) PDO Indicator(s) Indicator Indicator 1: Value (quantitative or qualitative) Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Increased efficiency of customs through increased number of customs points with nonintrusive control methods. None of customs points use nonintrusive control methods, of a total of (6 land, 2 air, 2 sea) Date achieved 12/31/ /30/ /28/2016 Comments (including % Partially achieved. This indicator was added during the 2014 restructuring. achievement) Indicator 2: Stronger planning, budgeting, treasury, and accounting systems in place, in line with international standards The MoF concluded the development of The new SAFI II the budget system is fully formulation and SAFI does not allow for developed and budget execution program-based ready to be modules of SAFI budgeting and payments launched. It Value II. Program-based are processed through a includes program- (quantitative or budgeting is fully quasi-treasury Single based budgeting, qualitative) embedded in SAFI II Account in a separate payment as well as the MTEF. system processing through Programs are also the Treasury Single results-based and Account in a adopt the separate system logic framework methodology. Date achieved 12/31/ /30/ /28/2016 Comments (including % achievement) Partially achieved (target value). This indicator was added during the 2014 restructuring. iii

8 (b) Intermediate Outcome Indicator(s) Indicator Indicator 1: Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Hours to clear merchandise from the moment it arrives to customs terminal until it is released Value (quantitative or qualitative) Date achieved 12/31/ /31/ /30/ /28/2016 Comments Partially achieved. The targets for this indicator were revised during the 2014 (including % restructuring. achievement) Indicator 2: % increase of web-processed transactions in customs Value (quantitative or qualitative) Date achieved 12/31/ /31/ /30/ /28/2016 Comments (including % Achieved. The targets for this indicator were revised during the 2014 restructuring. achievement) Indicator 3: Development of the new SAFI II module for budget execution will include programbased budgeting The MoF concluded the development of the budget formulation and budget execution The current SAFI system modules of SAFI Value does not include proper II. Program-based (quantitative or budget execution controls budgeting is fully qualitative) and only allows for line embedded in SAFI II. item budgeting. Programs are also SAFI is able to automatically create A new module accounting reports for budget and financial execution in statistics according SAFI II that to the International includes Public Sector program-based Accounting budgeting has Standards and 2001 been developed. IMF Government Financial Statistics standards. results-based and adopt the logic framework methodology. Date achieved 12/31/ /31/ /30/ /28/2016 Comments (including % achievement) Indicator 4: Value (quantitative or qualitative) Achieved. Implementation of training and dissemination initiatives to strengthen Government transparency, citizen participation, and access to public information 22 initiatives 0 initiatives have been 50 initiatives have been implemented implemented implemented. iv

9 Institutional capacity of the Government is strengthened and knowledge about right of access to public information and Government transparency tools has increased among public servants and the general public. Date achieved 12/31/ /30/ /28/2016 Comments (including % Achieved. This indicator was added during the 2014 restructuring. achievement) Indicator 5: Increased capacity in budget planning and formulation through the new integrated financial system Value (quantitative or 0 12 training sessions 17 training sessions qualitative) Date achieved 12/31/ /30/ /28/2016 Comments (including % Achieved. This indicator was added during the 2014 restructuring. achievement) G. Ratings of Project Performance in ISRs No. Date ISR Actual Disbursements DO IP Archived (US$, millions) 1 06/04/2010 Moderately Satisfactory Moderately Satisfactory /01/2010 Moderately Unsatisfactory Moderately Unsatisfactory /11/2011 Moderately Unsatisfactory Moderately Unsatisfactory /17/2012 Moderately Unsatisfactory Moderately Unsatisfactory /01/2012 Moderately Satisfactory Moderately Satisfactory /07/2013 Moderately Satisfactory Moderately Satisfactory /03/2013 Satisfactory Moderately Satisfactory /26/2014 Moderately Satisfactory Moderately Satisfactory /30/2014 Moderately Unsatisfactory Moderately Unsatisfactory /16/2015 Moderately Unsatisfactory Moderately Unsatisfactory /17/2016 Unsatisfactory Unsatisfactory /28/2016 Unsatisfactory Unsatisfactory v

10 H. Restructuring (if any) Restructuring Date(s) Board Approved PDO Change ISR Ratings at Restructuring DO Amount Disbursed at Restructuring in US$, millions 2/16/2011 MU MU 0.00 IP Reason for Restructuring & Key Changes Made Delays in effectiveness and substantial changes to design of components. 10/08/2014 MU MU 6.53 Extension of closing date. I. Disbursement Profile vi

11 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country context. At appraisal in 2009, El Salvador was facing an economic slowdown precipitated by the 2008 global financial crisis. Declines in remittances, exports, foreign investment, and tightened access to credit linked to the slowdown in the United States led to a sudden drop in revenues, increasing the country s fiscal deficit. In 2008, economic growth slowed to 2.5 percent, while the fiscal deficit rose to 3.1 percent of the gross domestic product (GDP) from 1.9 percent in The deterioration in macroeconomic and fiscal indicators posed a tremendous challenge to the new Government. According to the 2008 Multiple Purpose Household Survey, poverty had risen to 42.3 percent in Between 2008 and 2009, unemployment increased from 5.9 percent to 7.3 percent. More than 30,000 workers had lost their jobs, and combined with the sudden decrease in family remittances, there was a risk of further exacerbating poverty levels and social problems. 2. In response to these challenges, the Government announced a comprehensive Anti-Crisis Plan to protect vulnerable populations affected by job losses or lower remittances through actions targeting revenue and expenditures to create the needed space for priority spending. These included austerity measures to reduce non-priority spending and interventions to tackle tax evasion to contain further declines in public revenues. On the expenditure side, the Government decided to embark on the upgrading of budgeting, financial management, procurement systems, and improvements in public access to Government financial and nonfinancial information. The proposed World Bank operation was designed to specifically respond to a core pillar of the Government s Anti-Crisis Plan addressing short- and medium-term challenges in public sector development and fiscal management. This operation was part of a package of lending operations designed to support El Salvador s response to the crisis. A Development Policy Lending operation and various sector lending operations accompanied this loan. At appraisal, this operation was considered critical by the Ministry of Finance (MoF) to address weaknesses in its management of revenue and expenditure. 3. The objectives of the operation were aligned to the Government s strategic priorities and the World Bank s Country Partnership Strategy (CPS). 1 The main objective of the FY CPS for El Salvador was to support the Government s focus on addressing poverty and inequality through strengthening fundamentals for economic recovery by addressing macro and institutional vulnerabilities; strengthening social service delivery; and increasing economic opportunities, particularly for the poor. Under the first sub-objective (strengthening the fundamentals for economic recovery), the CPS identified support to (a) the expansion of fiscal space, (b) improved targeting of subsidies, (c) implementation of results-based budgeting, (d) enhanced access to information, and (e) improved fiscal transparency. The proposed operation sought to address critical short-term fiscal pressures through tax measures while developing institutional capacities and systems in the areas of transparency, results-based budgeting, and public financial management (PFM). 1 World Bank El Salvador - Country Partnership Strategy Report N SV. 1

12 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 4. The PDO was to strengthen the institutional capacity of specific government processes and agencies to increase the effectiveness and efficiency of revenue and expenditure management and enhance accountability and transparency in the public sector. 5. Progress toward achievement of the PDO was to be measured using the following PDOlevel indicators: (a) Tax collection as a percent of GDP increases at least 1 percent as a result of greater effectiveness and efficiency (b) Stronger planning, budgeting, treasury, and accounting systems in place, in line with international standards (c) The introduction of multi-annual projections and programmatic structure in the budget strengthens strategic planning and the implementation of performance-based budgeting (d) Savings generated in central government purchasing are at least 10 percent by comparing the same type of items (by adopting optimizing procurement policies) 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 6. The PDO remained unchanged throughout implementation; however, key PDO indicators were revised. During the first restructuring, three of the original four PDO indicators were dropped and one new indicator was added (see table 1). This was done to retain the relevance of project activities following delays in loan approval and advances in tax reforms and multi-annual budgeting using alternative financing sources. PDO indicators Original List Tax collection as a percent of GDP increases at least 1% as a result of greater effectiveness and efficiency Stronger planning, budgeting, treasury, and accounting systems in place, in line with international standards The introduction of multi-annual projections and programmatic structure in the budget strengthens strategic planning and the implementation of performance-based budgeting Savings generated in central government purchasing are at least 10% by comparing the same type of items (by adopting optimizing procurement policies) Revised List Stronger planning, budgeting, treasury, and accounting systems in place, in line with international standards Increased efficiency of customs through increased number of customs points with nonintrusive control methods financed by the loan 2

13 1.4 Main Beneficiaries 7. The project s main beneficiaries were the MoF and its respective offices and agencies: the National Financial Administration and Innovation Office (Dirección Nacional de Administración Financiera e Innovación, DINAFI); the Customs Agency (Dirección General de Aduanas, DGA); the Internal Revenue Agency (Dirección General de Impuestos Internos, DGII); the General Budget Office (Dirección General de Presupuesto); the Treasury Office (Dirección General de Tesorería, DGT); and the National Procurement Office (Unidad Normativa de Adquisiciones y Contrataciones de la Administración Publica, UNAC). For the transparency component, the target beneficiary was the Secretariat of Citizen Participation and Transparency (SCPT) within the President s Office. After the first restructuring, the DGII and the UNAC were no longer beneficiaries of project activities. Secondary beneficiaries of the project include the 114 institutions of the central public administration and citizens, who would benefit from the reforms introduced. 1.5 Original Components 8. The project was structured around four main components: 9. Component 1: Strengthening tax collection agencies (US$7.8 million). The objective of this component was to analyze and implement options to integrate activities to strengthen revenue agencies institutional capacity to control tax evasion and smuggling. Activities included (a) strengthening coordination among the DGII, DGA, and DGT to improve information sharing practices and increase common control procedures for better tax compliance; (b) strengthening the DGII s institutional capacity; and (c) supporting the DGA by strengthening its auditing and control functions. 10. Component 2: Modernizing of public expenditures and financial management (US$7.5 million). This component sought to help the MoF modernize its budget and financial management processes and systems through four main activities: (a) introduction of a multiannual and a performance-based budgetary framework; (b) implementation of a Treasury Single Account and use of electronic payments; (c) strengthening and upgrading of the integrated financial management information system (Sistema de Administración Financiera Integrada, SAFI); and (d) modernization of the public procurement system. 11. Component 3: Enhancing and piloting information management and public sector transparency initiatives (US$3.2 million). The main purpose was to support complementary initiatives to support planning and decision-making processes and transparency and access to public information to strengthen existing accountability systems. This component was explicitly designed to be flexible and adaptable to changing conditions. 12. Component 4: Project coordination and strengthening of the MoF (US$1.1 million). This component was designed to provide support to the MoF to adequately coordinate and monitor project implementation through strengthening the capacities of DINAFI and the MoF s administrative units of financial management and procurement (UNAC). 3

14 1.6 Revised Components 13. In 2011, before project effectiveness, the team conducted a Level 2 restructuring to respond to changes in scope required by the National Assembly for their approval of the project. All components were revised to modify specific activities that would be financed with funds from other donors and the restructuring reoriented activities to focus on customs management, PFM, and transparency activities. In particular, Component 1 (strengthening tax collection agencies) was substantially reduced and reoriented to focus on customs. The restructuring dropped activities related to the provision of software, equipment, and training for the DGII. Within Component 2, which focused on modernizing public expenditure and financial management, various subcomponents were revised, with the development of a multi-annual budgeting approach dropped from this component, while activities related to the upgrading and integration of associated financial information systems (SAFI II) were significantly expanded. The update of procurement policies to increase efficiency was removed and a new subcomponent was added to support the modernization of public investment and debt management systems. Overall, this component saw the largest increase in reallocated funds (from US$7.5 million to US$13 million). Activities on piloting information management under Component 3 were also reduced as alternative funding became available. Component 4, which focused on project coordination, was also reduced. While the four components of the project remained the same, the restructuring substantially altered the subcomponents and reoriented activities to focus on custom management, SAFI II, and transparency initiatives. 1.7 Other significant changes 14. Other changes related to scope and scale resulted from the interplay of two factors. First, the 18-month delay in effectiveness meant that the Government effectively had three years to implement all activities, rather than the originally intended five years. In 2014, the team conducted a second Level 2 restructuring of the project to extend the closing date by 21 months. The justification for this change was to allow for sufficient time to develop and deliver the core of the new financial management information system known as SAFI II, which experienced a number of delays in implementation and was rated as Moderately Unsatisfactory at the time. Second, the project activities were affected by the actions of the Government to fund the fiscal and PFM reform program with other sources, including grants from bilateral donors. The fiscal reform partly financed by this project was being carried out by a donor group (the U.S. Agency for International Development (USAID), German Agency for International Cooperation (Deutsche Gesellschaft für Internationale Zusammenarbeit, GIZ), European Union, and the Inter-American Development Bank (IADB) and required significant coordination efforts. Throughout project implementation, other donors contributed financial and technical expertise to the core activities under the project s components, necessitating frequent adjustments in the activities to be financed by loan proceeds, albeit without additional formal restructuring of components and subcomponents. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design, and Quality at Entry 15. The Implementation Completion and Results Report (ICR) finds the quality at entry to be 4

15 Moderately Unsatisfactory because of the shortcomings in the project design, which included an ambitiously large number of technically complex reforms underpinned by weak diagnostics. Background Analysis 16. The project design took into account lessons identified in the program and strategic assessments under the CPS, as well as the project-specific lessons highlighted in the ICR of the recently closed El Salvador Public Sector Modernization Technical Assistance Loan Project. 2 The team specifically noted two lessons from the ICR of relevance to the design of the operation. The first was the creation and maintenance of well-staffed monitoring and evaluation (M&E) units, given that the other implementing agency staff are overwhelmed with getting things done. The Project Appraisal Document (PAD) described the need for effective coordination and included a component for project coordination that would cover these functions. A second lesson that the team incorporated was assigning project activities to appropriate agencies. The ICR for the El Salvador Public Sector Modernization Technical Assistance Loan Project advised that the project components should be located within agencies that consider them to be a strategic priority, otherwise progress will be compromised. The design of Component 3 represents a practical operationalization of this recommendation because activities were selected based on explicit demands from implementing agencies and units. 17. However, a few lessons from the previous ICR were not fully considered in the PAD and underlying diagnostics were weak in some of the reform areas. From the previous project s ICR, technical assistance for change management and monitoring were highly recommendable in supporting the Government s implementation of reforms. While the PAD included resistance to change as one of the key risks, none of the project activities contemplated change management strategies nor was there an assessment of the capacity and engagement of senior and technical staff in pursuing the PFM reform program. Rather, change management was interpreted as training for staff. In addition, the previous El Salvador Public Sector Modernization Technical Assistance Loan Project had also experienced problems with the Government combining various fund sources with bilateral grants from other donors and had gone through four closing date extensions. Finally, the depth of background analysis in some of the proposed reform areas was thin. For example, the PAD contained no assessment of the readiness of El Salvador to adopt International Public Sector Accounting Standards. 18. Assessment of the project design. The project was designed to address a number of key issues of relevance to authorities in strengthening both revenue and expenditure management functions. The team drew on recent diagnostic assessments to identify priority actions, most notably, the findings of the 2009 Public Expenditure and Financial Assessment (PEFA). The PEFA results underscored the need to improve the strategic orientation of the budget through introduction of a medium-term focus and a programmatic classification. However, the design lacked a thorough assessment of the enabling environment for upgrading the financial management information system (SAFI) and embarking on a complex integration with other systems (human resources, debt, public investment). Designing and implementing a comprehensive financial management information system project is a difficult process and must 2 El Salvador Public Sector Modernization Technical Assistance Project ( ) (P007164) of US$25 million, which was rated Satisfactory. 5

16 be carefully sequenced with other PFM reforms (such as establishing a Treasury Single Account). How the proposed reforms would be sequenced under the Government s PFM reform strategy was notably absent in the PAD. The project length was also very short, based on what the literature suggests for these types of reforms. According to a World Bank study analyzing 25 years of experience in implementing financial management information system projects, financial management information system projects take a minimum of six to seven years to complete and often require multiple operations for full integration and rollout of the system across the government. 3 The design of the project included major PFM reforms that were sequenced in parallel with automation, including improvements to the budget classification, Treasury Single Account operations, and cash management functions. 19. Adequacy of Government commitment. The project was designed in close collaboration with senior leadership of the MoF, and the Minister of Finance assumed an important leadership role in advocating for the project in the Congress. The project anticipated the need to establish a Coordination Committee under the direction of the Vice Minister of Finance with all line Directors as members of the committee. DINAFI was to serve as the Technical Secretariat for the Committee, and the Vice Minister of Finance was to play a championship role for the overall project. However, there was uneven involvement of the beneficiary units across the MoF in the design of the activities in Component 2, leading to some confusion of accountabilities and coordination once implementation began. 20. Assessment of risks. This operation was considered to pose Substantial risk due to risks at the country and operational levels. The highest risk identified was the risk related to the approval of the loan in Congress because the new administration did not have the required majority to approve World Bank loans in the National Assembly. To mitigate this risk, the Government and the World Bank engaged in a consensus-building process, including the involvement of the opposition in defining priorities and providing the opportunity to comment on project design. In addition, the loan was presented with the CPS and a package of loans to be approved by Congress, which represented a carefully selected program focused on priorities where there was broad support. Operationally, specific risks identified included limited capacity of the Government (the new administration had assumed power after 20 years of opposition rule and had limited experience in working in the Government), the lack of financial management and procurement experience of the implementing agency, and resistance to change. The proposed mitigation measures included setting realistic time frames and achievable targets in the Results Framework, a flexible approach for selecting activities to be financed through Component 3, hiring of specialized personnel for financial management and procurement, and an analysis of relevant political and economic factors that may undermine progress. 21. A number of these risks materialized during implementation. Once the package went to Congress, this loan was the last of the World Bank projects to be approved, leading to the 18- month delay in effectiveness. The operational risks proved to be high, rather than substantial, during the implementation, leading to deserted procurement processes and resistance to reform by beneficiaries. The risk of inadequate coordination between donors supporting the PFM reform was not initially identified in the PAD as a risk. Given the lessons learned from the previous 3 Dener, Cem, Joanna Watkins, and William Dorotinsky. Financial Management Information Systems: 25 years of World Bank Experience on What Works and What Doesn t. a World Bank Study,

17 project, this should have been contemplated in the risk assessment for this project. It was only during the 2011 restructuring, that an additional risk was identified regarding the implementation of complementary activities (previously under the scope of the loan) with alternative financing sources. This risk related to the timely implementation of these complementary activities and was identified as Moderate. The World Bank should have been more explicit about the risks of donor coordination at design and ensured that the activities financed by World Bank proceeds were not subject to other donor interventions. 2.2 Implementation 22. During implementation, several factors negatively affected progress. With an 18-month delay in effectiveness, implementation was off to a rocky start. It took more than a year to launch project activities, explained mainly by the number of internal and external actors to coordinate; the scope and technical complexity of the reforms (covering customs, PFM reforms and system integration, and transparency); and the technical capacity of the Project Implementation Unit (PIU). In the Implementation Status and Results Reports (ISRs), the rating for overall implementation progress oscillated between Moderately Satisfactory, Moderately Unsatisfactory, and Unsatisfactory over the course of implementation. The factors that negatively affected progress include donor coordination, weak implementation agency and PIU capacity to execute project activities, and overall governance of the fiscal and PFM reform program. 23. Donor coordination. The PFM reform program partly financed by the loan (Component 2) was carried out by a donor group (USAID, GIZ, European Union, World Bank, IADB), which required significant coordination efforts to ensure that each reform, as well as the integrated financial management information system (SAFI II), were compatible and functional. At project effectiveness, it was agreed that all donors involved, except for the World Bank, would contribute grants and/or non-reimbursable assistance to various elements of the reform program. The MoF preferred the use of non-reimbursable funds for process reforms and wanted to use loan proceeds mainly for equipment and software development of the new SAFI II. In this context, each donor was guided by distinct objectives, incentives, and deadlines without unified objectives under the reform agenda defined by the Government. Aligning these incentives proved challenging, and the Government s focus on exhausting grant funds led to delays in loan disbursements (it took 11 months for the first loan disbursement after effectiveness). Donor meetings were regularly organized to address issues, but interactions were mainly at a political/general level, even though the reform was highly technical. Notably absent from the dialogue were technical discussions on the sequencing of reform interventions and alignment of methodologies and objectives. As noted in ISR 8, donor interests were not necessarily aligned given their own internal commitments to prioritize their programs. In the development of an integrated financial management information system, loan activities substantially depended on a close sequencing and alignment with USAID interventions, which were developing the treasury and accounting modules of SAFI II, accounting reforms, and partially funding the contract of the first project manager. Despite attempts to work together, coordination remained tumultuous throughout implementation. The MoF s coordination of donor activities and funds was also very limited. Though DINAFI was the central unit responsible for coordinating donor interventions each Directorate managed its own donor projects in an independent and isolated way. At one point, the ministry had roughly 55 projects with distinct sources of financing. 7

18 24. The role of the implementing agency (DINAFI and PIU). Over the course of the project, the head of DINAFI, the unit responsible for overall project coordination and monitoring, changed three times and key staff in the PIU left. These changes in management negatively affected the execution of project activities. The PIU had not managed a World Bank project before and required time and training to understand World Bank policies and complex procurement procedures. With the exception of the Financial Management Specialist, hired with loan proceeds, the PIU was mainly formed by a small number of MoF employees (ranging from four to six persons over the course of the project) who had full-time jobs in addition to their PIU functions. This lack of full-time staff to attend to the operational, technical, and donor coordination aspects of the project contributed to delays in implementation and elevated the costs of supervision and assistance by the World Bank team. As noted in the ISRs, the Government had committed to strengthen the procurement function by hiring a Procurement Specialist, but this was not done until near project closing. The Government s justification was to not distort the internal job market (salaries would be higher for a person financed through project funds). The project management ratings oscillated between Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, and was rated Highly Unsatisfactory in the last two ISRs. 25. Governance of the overall PFM reform. While the senior management of the MoF continued to view the project as strategically important throughout implementation, a number of critical decisions related to the integration of treasury, budget, and accounting processes were decentralized to lower levels (to Directors and, sometimes, external consultants), jeopardizing the integration of modules under SAFI II. At the start of the project, a Coordination Committee was established, headed by the Vice Minister, with the task of resolving issues of coordination between the different modules with all relevant Directors present. However, the committee was not fully operational until the last year of the project. While the original intention was to meet on a bimonthly basis, in practice, meetings were held only a few times every year. The frequency increased in the last year, but it was not sufficient for a project that needed quick and definitive solutions at key points in time. Finally, as documented in the Aide Memoires, the Minister took a strategic decision to allow the information technology (IT) development of SAFI II along two parallel tracks one financed by the World Bank and led by DINAFI and another financed by USAID and led by a contractor. Unfortunately, the different teams used distinct methodologies to build the system and had different perspectives on the functional aspects of the system. Implementation of Component 1 (Customs) 26. This component required a number of large and complex procurement processes for equipment and software to improve customs control systems. A major procurement was that of the commercial nonintrusive scanners (básculas camioneras), which failed four times before it was eventually deserted. The reasons behind these repeated failures were mistakes in the technical specifications, inadequate market assessments, and difficulties in adhering to World Bank procurement procedures. To avoid another procurement failure with the scanners, at one point, DINAFI and the Customs Directorate jointly reviewed the bidding documents and carried out an on-site visit to the locations in which the weighing equipment would be placed. As a result of the visit, it was found that one of the locations was not suitable for placing a scale because of weight restrictions. The World Bank team attempted many times to support the Government in the technical definitions of the goods and services and in trainings on World Bank procurement procedures. Despite these procurement failures, the Government moved forward with the 8

19 nonintrusive customs control strategy by acquiring scanners on a concessionary basis. 27. The other major activity of this component was the upgrading of the customs software system. A contract was issued with United Nations Conference on Trade and Development (UNCTAD) to purchase their Automated System for Customs Data platform (SIDUNEA, for its Spanish acronym). This platform automates and integrates customs business processes (for example, transit, regimes, and control of deposits and guarantees). This contract suffered a number of setbacks because of delays in start-up and major differences between the Government and UNCTAD with regard to the customization of the software. There were problems with the interoperability between existing customs systems (for example, the transit system) and SIDUNEA. The Government filed a complaint with UNCTAD for the development of the system, citing problems with the low level of attention provided by the technical staff of UNCTAD to the project, the parametrization of the software, and other technical aspects to make it web accessible. Though the system was close to finalization by the time of the project s closing, unresolved contractual issues remain between UNCTAD and the Government. Implementation of Component 2 (PFM) 28. This component suffered several setbacks during implementation because of resistance to the reform by different divisions within the MoF, major donor coordination problems, weak capacity of the coordinating unit, and lack of a Project Director for the design and implementation of SAFI II. 29. Once the project became effective, it was clear that the initial design of the PFM reform program specifically, the upgrading and integration of the financial management information system was not feasible. The PFM reform strategy was revised, leading to a gradual implementation and sequencing of reforms. Initially, the design of SAFI II was to include a considerable number of modules, including public investment and human resources. In view of the accumulated delays and the challenges posed by interagency coordination and ensuring that all modules (often financed by different donors) were compatible with one another, the Government decided at the end of 2013 that it was necessary to pursue a gradual implementation approach and, as such, reduce the scope of SAFI II for the first stage. Recognizing the need for sequencing of the PFM reform, the Government modified its reform approach into three phases. The first phase of SAFI II was reduced to the financial core budget, treasury, and accounting modules. 30. In selecting the software solution, the Government decided to pursue a locally developed software (LDSW) solution rather than purchase a commercial, off-the-shelf solution. 4 The Government tried to procure a firm to implement the LDSW system. However, this procurement failed twice because of a lack of firms willing to bid on the project. The contingency plan was to contract 20 local developers because there was not enough time before project closing to conduct another bidding process. Given the size of the project and the lack of technical leadership by the Government, donors consistently called for the Government to put in place a Project Manager with solid technical background for SAFI II. Eventually, a Project Manager was selected, but 4 LDSW is usually less expensive and the client owns the rights to the software. Necessary modifications to the software can be done more easily. However, it requires substantial leadership and coordination to implement the decisions needed for LDSW. 9

20 within three months, the manager resigned. In September 2015, the World Bank financed a new Project Manager but the reception by some Government Directors was complicated. Directors had already experienced distinct approaches from various consultants and it was hard to have any agreement between World Bank consultants, USAID, and the Directors. The two main donors involved in SAFI II (the World Bank and USAID) had conceptual differences and utilized distinct methodological approaches. Within the MoF, coordination among the various divisions posed a serious challenge, and there was no clear hierarchy for resolving challenges. The situation deteriorated to the point that the Project Manager and main technical consultants resigned. At that point, DINAFI continued software development with the methodologies and approaches defined by the last Project Manager and technical consultants and finalized the development of the budget formulation, budget execution, and management of human resources structures modules with local consultants. The World Bank provided significant technical support and used project supervision missions to convene stakeholders to discuss pending issues, achievements, and next steps. Aside from this technical assistance, loan proceeds were directed to upgrading the hardware of the MoF. Detailed Aide Memoires provide evidence of the World Bank team s advice to the Government on system implementation. 31. As originally conceived, DINAFI did not play a major leadership role in ensuring that the overall PFM reform strategy was well articulated. DINAFI started to drift from the strategy of developing an integrated financial management information system (SAFI II) by financing modules with other funding sources, thereby limiting the strategic orientation across components/subcomponents. DINAFI was thrifty with loan funds, mindful that they could finance activities using grants from donors. Organizationally, DINAFI was also limited in its ability to coordinate and resolve conflicts between Directors (necessary for the integration of processes for SAFI II). Hierarchically, DINAFI is on the same level with the other directions in the ministry. Legally, DINAFI s responsibility for implementing the new system was by way of delegation from the minister through an Executive Order. This was important because in the Organic Budget Law (Ley Orgánica de Administración Financiera del Estado), DINAFI was responsible for integration and not for functional modifications. 5 As a result, the Directors of Budget, Treasury, and Accounting did not think they had to comply with DINAFI s requests. Implementation of Component 3 (Transparency) 32. Despite some initial delay in the start of activities, once the project became effective, implementation proceeded smoothly. The Law on Access to Public Information became effective in 2011 and dissemination activities commenced under the project. Initial delays under this component were because of a lack of communication between the PIU and the SCPT and delays in planning its activities in accordance with the Operations Manual. The World Bank served as a facilitator between the PIU and the SCPT to ease communication and speed up implementation, which then proceeded relatively smoothly. 2.3 Monitoring and Evaluation (M&E) Design, Implementation, and Utilization 33. The project s M&E framework is considered to be modest. The original design of the operation s M&E framework exhibited weaknesses in the logic connecting project inputs with 5 See Articles 9 and

21 outputs and outcomes. While some of the original shortcomings were overcome during the first restructuring with the dropping of some indicators and revisions to targets, weaknesses in the logical framework and the formulation of indicators proved challenging in the evaluation of the operation s impact. 34. M&E design. The Results Framework included in annex 3 of the PAD specified four project outcome indicators and 20 intermediate outcome indicators which monitored a mixture of higher-level outcomes and outputs of subcomponent activities. The original Result Framework included a number of PEFA indicators to measure performance in PFM practices and used the 2009 PEFA to establish the baseline. However, in some cases, the design of the Results Framework was not amenable to changes in the activities of subcomponents, because a number of the intermediate indicators were directly linked to outputs such as the procurement of software or number of trainings conducted (for example, The Human Resource Management System [Sistema Integral de Recursos Humanos - SIRH] is updated and interoperable with SAFI). Some of the indicators were also not well defined. For example, one of the PDO indicators was, stronger planning, budgeting, treasury, and accounting systems in place, in line with international standards. It was not clear how this would be measured. This was only partially corrected with the changes introduced in the Results Framework in the first restructuring where many indicators were dropped or revised. Such changes are indicative of elements of both a corrective and adaptive restructuring to fix design flaws in the original indicators and improve the relevance of the activities and associated Results Framework, in light of the delays to the effectiveness in the project and changes in loan-financed activities. 35. M&E implementation. DINAFI coordinated the monitoring of project activities throughout implementation; however, the PIU within DINAFI was small and a person was never hired to take on the M&E functions. Though the PAD did note the need for careful consideration of M&E arrangements (as a lesson learned from the ICR of the previous project), this was not translated into practice. Data collection relied heavily on meetings and reports by the counterparts involved in each of the subcomponents, which on occasion, led to delays in the provision of timely information. The lesson from the ICR of the previous operation on M&E was not translated into practice during implementation. Information on key project outputs and indicator progress was regularly collected as part of the World Bank s supervision efforts. In the ISRs, the ratings for M&E ranged from Moderately Satisfactory to Satisfactory, with the exception of ISR 12 when it was rated Moderately Unsatisfactory, noting the PIU has not satisfactorily monitored the project. Production of project performance information is not adequately systematized and takes significant time to be integrated and sent to the Bank. 36. M&E utilization. As originally conceived in the PAD, the MoF was to establish a Coordination Committee under the direction of the Vice Minister of Finance responsible for strategic planning and project monitoring. DINAFI was to serve as the Technical Secretariat of this committee, with specialized personnel to be hired for this purpose. Participating agencies were to be present on the committee to ensure all activities were monitored timely and substantively. However, in practice, this Coordination Committee mainly focused on activities related to PFM reform and did not take on a substantive monitoring role across all components. 11

22 2.4 Safeguard and Fiduciary Compliance 37. Safeguards. No safeguard policies were triggered by the project. 38. Financial management. Financial management performance is considered to have been Satisfactory. Some issues arose at the start of the project with the need to hire a dedicated Financial Management Specialist in the PIU. Once this issue was resolved, financial management performance was rated Satisfactory in the remaining ISRs (4 13). 39. Procurement. Procurement performance is considered to have been Moderately Unsatisfactory. In all of the ISRs, procurement performance was rated Moderately Satisfactory; however, the PIU did not hire a dedicated Procurement Specialist as was initially agreed with the World Bank until the end of the project. There were recurrent delays and failures in the largest procurements of the project, particularly in equipment and software in the customs and PFM components. Procurement plans were revised substantially during implementation and a large number of complex procurement activities were undertaken 15 International Competitive Bidding packages (US$6.9 million in total, almost half of the disbursements). There were 18 cancelled procurement packages. Staff capacity in the implementing agencies responsible for the technical specifications was also very weak. This situation could have been avoided by careful planning, bringing experienced IT experts to the team, and learning more from similar activities using available World Bank resources. 2.5 Post-completion Operation/Next Phase 40. There are no planned World Bank supported follow-up operations. The Government is committed to continue working on the remaining modules of SAFI II and will continue to receive the support of other donors, such as USAID and IADB, on parts of the PFM reform agenda. SAFI II will need to be rolled out to 118 institutions, and the latest decision of the MoF is that it will continue the rollout using internal technical expertise. Elections for Congress are scheduled for 2018 and presidential elections are scheduled for Though uncertainties naturally attend such changes in administration, it is unlikely that the activities associated with the purchasing of hardware will be undermined by the change in administration. On the other hand, there are some risks that a transition in administration will affect the transparency agenda and specifically, the main beneficiary agency, the SCPT. Secretariats are established by decrees and they could disappear in 2019 with the elections. The SCPT is aware of this and has established a far-reaching network of public servants in each of the institutions they work with to improve the sustainability of their interventions. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 41. The PDO continues to be relevant to both Government priorities and to the World Bank s strategic priorities and, as such, relevance is rated Substantial. The World Bank s Country Partnership Framework covering 2016 to 2019 sets out a strategic objective of promoting the efficiency of public spending under Pillar 2 to foster sustainability and resilience. Moreover, the 12

23 Government s Five-Year Development Plan ( ) 6 sets out an agenda on fiscal policy and PFM with the following lines of action: increase tax collection, increase and improve the execution and quality of public investments, improve the efficiency of social programs and subsidies, and generate primary savings so that finances are sustainable in the medium and long term. The 2016 Article IV mission of the International Monetary Fund (IMF) concluded that El Salvador continues to suffer from significantly lower growth than neighboring countries with GDP growth expected to be 2.4 percent in According to IMF estimates, the fiscal deficit is expected to widen further over the medium term without austerity measures, and public debt is projected to rise to above 70 percent of GDP by Wage pressures, higher interest rates, additional security costs, and public investment projects are all expected to add to demands on fiscal resources. 42. The relevance of the project s design to the objective is considered Modest. While the project was restructured and substantially adjusted over time to respond to changes in Government priorities, the PDO was not adjusted nor was the design of the components. The design of the subcomponents of the project was not fully aligned with the stated objective and this disconnect was not addressed through a Level 1 restructuring. The team faced the constraint that the project would have to go back to the National Assembly for approval if there were any changes in objective (which would require a Level 1 restructuring). The subcomponents corresponding to the revenue management objective were considerably reduced at effectiveness and reoriented to customs management. The component on PFM was also significantly altered during implementation when the World Bank stopped financing the development of the core modules of SAFI II. The PDO was not adjusted to align with the change in these activities. The Results Framework was also not well suited to measure the project objectives. A lack of internal logic connecting the project s inputs, outputs, and outcomes make the project s contributions to the development objective difficult to evaluate. 3.2 Achievement of Project Development Objectives 43. Before addressing the achievement of the PDO, it is important to clarify how the efficacy of the project is rated. In line with the ICR guidelines, the overall project performance is a weighted average of the three primary objectives as described in the PDO. Each objective is rated equally. To assess each objective, PDO and intermediate outcome indicators, as well supplementary evidence, are used to arrive at an overall assessment. Annex 10 provides a detailed analysis of this evidence base, organized by objective. 44. From the PDO, one can identify three main objectives of the project: (a) increase the effectiveness and efficiency of revenue management, (b) increase the effectiveness and efficiency of expenditure management, and (c) enhance the accountability and transparency in the public sector. 45. To analyze the first objective, one should consider whether the project strengthened the institutional capacity to increase the effectiveness and efficiency of revenue management. At design, the project components were focused on strengthening the links between tax and customs with the aim of improving revenue management. However, at effectiveness, the revenue 6 El Salvador Productivo, Educado y Seguro. Plan Quinquenal de Desarrollo ( ). 13

24 management component was substantially adjusted to focus solely on the customs aspects of keeping out undesirable goods, trade facilitation, and back-office improvements to customs systems. The role of customs management is generally to keep undesirable goods out while collecting revenue and taxes on goods that are allowed in, as well as facilitate trade. 7 The PDO indicator on increasing tax revenue was dropped. The objective of revenue management remained the same, but the project effectively de-emphasized revenue improvement activities. 46. The PDO and intermediate indicators focus on the effectiveness and efficiency of customs management. The PDO indicator was partially achieved the target was to have seven customs points with nonintrusive control methods, and at project closing, six were achieved. However, none of the six customs points nonintrusive methods were actually funded through the loan (because of procurement problems), but the strategy funded through the project did contribute to the end result. Before the project, the strategy and mechanisms in customs points did not exist. The intermediate indicator on web-processed transactions was achieved. The intermediate indicator related to reducing the hours to clear merchandise was partially achieved, reduced from 60 to 21 hours (the target was 17 hours). 47. While the indicators were partially achieved, it is not clear that there were any attributable improvements in revenue management as a result of project activities. From this assessment, it appears that the project had a negligible contribution to increasing the effectiveness and efficiency of revenue management. 48. To analyze the second objective, one should consider whether the project strengthened the institutional capacity of the MoF to increase the effectiveness and efficiency of expenditure management. The PDO indicator was stronger planning, budgeting, treasury, and accounting systems in place, in line with international standards. Given the breadth of areas that this indicator covers, the target value was defined as the SAFI II system to be fully developed and ready to be launched and to include program-based budgeting and payment processing through the Treasury Single Account. According to the ISRs, this indicator was partially achieved (because the budget formulation and execution modules funded by a combination of World Bank and USAID funds of SAFI II were completed by project closing), though a more nuanced assessment of this indicator would likely have relied on PEFA indicators, which appeared in the original Results Framework. El Salvador s two most recent PEFAs are from 2009 (at the time of project design) and A comparison of the relevant PEFA indicators between 2009 and 2013 provides no evidence of progress on the relevant indicators (see annex 11). 49. With regard to stronger budget planning, program-based budgeting, and the medium-term expenditure framework (MTEF), these reforms are embedded in the design of the new SAFI II modules. Treasury reforms, which were initially conceptually guided by the World Bank and operationalized by USAID, did achieve some results. Payment processing for Central Government payments (excluding payroll) is now being done through the Treasury Single Account. Full implementation of the Treasury Single Account (across all institutions and for the payroll) is expected once SAFI II is fully operational. Even with partial payments through the Treasury Single Account, the Government has seen a reduction in the time to pay private contractors. Before the reform, it took two weeks (15 days) for funds to arrive to providers, now 7 Gerard McLinden et al, Border Management Modernization, World Bank (2011). 14

25 it takes one day or less. This has helped resolve serious issues in paying health and education providers. The Treasury now also has real-time information on its cash balances. All of the debts are registered and finance officials can conduct a daily analysis of the operations, which is critical given the liquidity crisis facing the country. Nonetheless, project performance can only be partially attributed to these results, given the predominant role of USAID in this area. 50. Both intermediate indicators were also achieved. The SAFI II modules budget formulation, budget execution, and the Management of Human Resources Structures were developed by project closing, but they are not operational yet. The Government expects to put the modules in operation in the 2017/18 budget process. Training sessions on how to use the modules were conducted for the Government staff. At closing, SAFI II was not operational. Modules that were the responsibility of the World Bank have been developed, and other modules being developed by USAID are still in development. 51. Other achievements not captured in the Results Framework were the rollout of resultsbased budgeting and the substantial upgrading of the MoF s IT hardware. The legal framework has been modified to allow institutions in the central public administration to prepare their budgets following a programmatic classification. As of September 2016, 79 out of 84 public sector institutions (excluding 30 hospitals) have a programmatic budget structure, which is ready to be applied in the preparation of the 2018 budget through the SAFI II budget formulation module. Approximately 1,820 Government officials of the public administration have been trained on the results-based budgeting techniques. As a result of the project-financed investments in upgrading hardware, the MoF s information and communication technology (ICT) capabilities have improved. About 90 percent of the hardware of the MoF has been updated through the purchasing of new computers, servers, video conferencing equipment, critical network infrastructure, and a data warehouse. This infrastructure has contributed to improvements in the processing, storage, accessibility, and security of data which in turn has contributed to an increase in the number of online citizen/business services provided by the MoF (see section on unintended outcomes in section 3.5 for further details). 52. While there is no evidence of improvement in the efficiency and effectiveness of expenditure management as measured by comparing relevant PEFA indicators between 2009 and 2013, the project was able to introduce some of the inputs necessary for eventually realizing these changes (new systems, processes, hardware, and so on). From this assessment, it appears that the project had a negligible contribution to increasing the effectiveness and efficiency of expenditure management. 53. The third objective, as stated in the PDO, was to enhance the accountability and transparency in the public sector. No PDO indicator was established for the transparency component, but the related intermediate indicator was achieved and surpassed, with 50 training and dissemination initiatives conducted to strengthen Government transparency. However, this indicator does not provide much evidence of the impact of the range of activities financed under this component. From supplemental evidence, it appears that this component had a substantial impact. With regard to transparency initiatives, the project supported improvement and expansion of the InfoÚtil Portal (originally developed with funding from the IADB in 2012), which enables citizens to access useful day-to-day public information (public procedures and services) for health, education, economy, and finances. This portal allows citizens to obtain 15

26 updated information and submit suggestions, opinions, or complains about public management. The portal has received more than 5.3 million visits in the last three years. The project also financed a user satisfaction survey of the portal. 8 The survey results suggest that users were generally satisfied with content, design, and accessibility of the portal (see annex 10 for details on the results of the survey). The project also supported the development of a Public Jobs Vacancy Portal (portal empleos publicos) that was launched in December 2015, which consolidates vacancies from 45 public institutions. The project also contributed to the design of the proposal for open data policy and the creation of the Gobierno Abierto Portal which integrates information, requests for information, complaints, and rankings of the performance of public institutions. Evidence of the impact of these reforms are reflected in improvement to El Salvador s Open Government score. From 2014 to 2016, Open Government improved from 0.37 to With regard to accountability initiatives, this component supported the national and international dissemination of El Salvador s Law on Access to Public Information through television campaigns, brochures, and trainings in schools and public institutions. Eighty-three Offices of Information and Response (OIRs) were created within the Central Government. As of November 2013, the OIRs had received 24,021 requests for information and responded to 90 percent of the requests. 10 Higher-level evidence of the impact of these reforms reflected in improvement to El Salvador s voice and accountability scores. From 2012 to 2015, the score improved from 46.5 to For a comprehensive analysis of the results of each objective, see annex 10. For the list of outputs by component, see annex Efficiency 55. Project efficiency is considered to be negligible. Given the nature of the project, no formal economic cost-benefit analysis was performed at the time of design. While results, such as increases in the web-processing of various transactions and reductions in hours to clear customs merchandise are likely to have contributed to savings from enhanced efficiency, it is difficult to quantify such benefits ex post given the lack of concrete data. 56. The PAD s Economic and Financial Analysis listed a number of potential financial and economic impacts mainly related to efficiencies generated from improvements in revenue collection, facilitation of payments to public sector suppliers, and reductions in idle balances in commercial banks. To the extent possible, these impacts have been updated based on the results of the project and are captured in annex 3. Some of the achievements, such as systems development and capacity building, affect back-office government processes, and while it is expected that such improvements will end up contributing to improved services for citizens and businesses, such gains are difficult to measure. 57. Nevertheless, aspects of project implementation likely reduced the efficiency of some activities. In line with ICR guidelines, beyond traditional measures of efficiency, the review should take into account aspects of design and implementation that reduced efficiency. As 8 InfoÚtil, Evaluación Ciudadana del Portal Infoútil, World Justice Project. 10 Rendición de Cuentas Fondos de Cooperación Worldwide Governance Indicators. 16

27 discussed in the implementation section, the restructuring and extension of the project as a result of procurement issues and the complexity of reform reduced the overall efficiency in a number of the subcomponents. 3.4 Justification of Overall Outcome Rating Rating: Unsatisfactory 58. The overall outcome rating is considered to be Unsatisfactory (see table 2), reflecting major shortcomings in the operation s achievement of its objectives. While the objectives of the project are still relevant, the design of the project was not fully consistent with the objectives. The efficacy of project interventions was negligible in terms of revenue and expenditure management but substantial in improving transparency of the public sector. The efficiency of the project was negligible. Dimension Relevance Relevance of Objectives Relevance of Design Efficacy (equal weight across three objectives) Objective 1 (revenue management) Objective 2 (expenditure management) Objective 3 (transparency) Efficiency Overall Substantial Modest Negligible Negligible Substantial Negligible Unsatisfactory Rating 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 59. Through the dissemination and implementation of the Access to Public Information Law (signed into law in 2012) and transparency initiatives, the operation has had an impact on the accessibility of public information for citizens. While the project was not designed to have a direct impact on poverty, gender, or social development, access to information and transactional services for citizens through online portals has improved. For example, the InfoÚtil Portal was redesigned enabling citizens to access useful day-to-day public information (public procedures and services) for a range of health, education, economic, and financial services. (b) Institutional Change/Strengthening 60. This project has been an important catalyst in the institutional strengthening of both the MoF and the SCPT. From a long-term institutional development perspective, the activities begun during the first loan and continued in the second loan have strengthened the MoF s ability to perform critical functions related to cash management, program-based budgeting, medium-term budgeting, and customs management. Through this project, the World Bank has maintained a continuous policy dialogue on PFM and other important areas for the country s development with the authorities over the last decade. 61. The institutional evolution of the SCPT also suggests that the transparency and 17

28 accountability agenda has taken hold in the administration. In 2009, the Secretariat was a Sub- Secretariat of Transparency and Anti-corruption (Subsecretaría de Transpaencia y Anticorrupción) under the Secretary for Strategic Matters of the Presidency. In 2014, it was converted into the Secretary of Citizen Participation and Transparency (SCPT), as citizen participation was added to its functions. The number of staff increased from 45 in 2014 to 60 in The SCPT has also built a network of public officials across institutions to promote and incorporate tools related to transparency, citizen participation, and accountability. (c) Other Unintended Outcomes and Impacts (positive or negative) 62. As a result of project-financed investments in upgrading hardware, the MoF s ICT capabilities have improved. Approximately 90 percent of the hardware of the MoF has been updated through the purchasing of new computers, servers, video conferencing equipment, critical network infrastructure, and a data warehouse. This infrastructure has contributed to improvements in the processing, storage, accessibility, and security of data that has contributed to an increase in the number of online citizen/business services provided by the MoF. The upgraded infrastructure contributed directly to improving the capacity of the MoF to make the following citizen services available online: (a) tax refund consultations; (b) presentation of information and declarations and payment of taxes online; (c) request forms and consultations for tax solvency; and (d) requests for cancellation, receipt, or modifications of one s Tax Identification Number (Número de Identificación Tributaria) for naturalized citizens. These services represent about 20 percent of total number of services available on the MoF s online portal. Along with the infrastructure, the project also contributed to improvements in ICT knowledge of the MoF staff through trainings. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 63. No beneficiary survey or stakeholder workshops were conducted. 4. Assessment of Risk to Development Outcome Rating: Moderate 64. The progress made under the project is likely to be sustained for a number of reasons, though full implementation of SAFI II will require a concerted effort by the MoF leadership. First, the legal framework provided by laws and decrees across a number of areas (transparency, treasury, PFM), makes the sustainability of project outcomes more likely. Second, the use of transparency and open data tools by the ministries and agencies has created a demand for support from the SCPT. The tools developed through this project have been incorporated into the agencies current procedures and are unlikely to be reversed. The moderate, rather than low, risk rating for these reforms is because of potential difficulties in the full implementation of SAFI II. The budget preparation, budget execution, and human resource modules have all been developed and are expected to be operational in 2017/18. Nonetheless, critical interconnection issues remain between the Treasury, accounting, and the budget divisions of the MoF to ensure that all modules are integrated within SAFI II. Once these issues are resolved, the software modification should be relatively straightforward given improvement to the IT capacity of DINAFI staff. It appears likely that SAFI II will be implemented in the next 1 3 years given continued USAID support to the reform and the Government s expressed interest in ensuring the new system is up 18

29 and running. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 65. During project preparation, the World Bank team drew upon its previous experience in implementing the El Salvador Public Sector Modernization Technical Assistance Loan Project from (P007164), an understanding of the political economy of El Salvador, and PFM and tax diagnostic assessments to inform the design of reforms. The overall risk of the operation was rated Substantial and the risk of delays in the approval of the loan in Congress was the most significant risk identified. Other risks identified included institutional implementation capacity, procurement, and resistance to change. While these risks were acknowledged in the PAD, the calibration of activities and components to these risks could have been better reflected in the design of the reforms by reducing the scope of activities. The proposed mitigation measures were also not sufficient for reducing the high costs of coordination inherent to the design of the project. 66. Moreover, from the complex design of the project covering tax, PFM, financial management information system, and transparency reforms it is not clear if similar projects from other contexts were reviewed to anticipate implementation problems. The integrated financial management information system reform, in particular, was overly ambitious from the start. It appears that the project did not take into account the lessons of other financial management information system projects, which on average take seven years to complete, and must be appropriately sequenced (starting with core budget and treasury modules, and eventually moving to integration of additional modules such as human resources or public investment management PIM). The large and complex procurement packages were also not given enough attention at the design phase. It appears that the original bar was set too high for a number of the PFM reforms. For example, the application of International Public Sector Accounting Standards and Government Financial Statistics There was limited description of the weaknesses of the current arrangements in the PAD or attention to how to sequence the introduction of the standards. 67. Finally, while the PAD acknowledged that the new administration, which had assumed power after 20 years of opposition rule, might lack the political and technical experience needed to implement the project, the design of the project was not adequately tailored to potential problems in capacity to execute reforms. Additional support and supervision were seen as mitigating measures rather than reducing reform expectations or coordination costs across components. (b) Quality of Supervision Rating: Moderately Unsatisfactory 68. Over the course of the project, the World Bank team was somewhat inconsistent in the quality of its supervision. At times, it was proactive, organizing several missions and meetings 19

30 with the Government and donors to align incentives and priorities and strengthen the implementation capacity of the PIU. At other times, it was noticeably absent in defining the reform trajectory, using restructurings to adjust the PDO or better align activities with the PDO or was slow in responding to client requests for no-objections. 69. There were three official Task Team Leaders (TTLs) during the life of the project. This turnover contributed to different approaches in supporting implementation at times focusing on components where the TTL had technical expertise and notably absent in areas where the TTL lacked the technical expertise to engage. The World Bank was also remiss in ensuring adequate transition arrangements between TTLs, leading to some gaps in the dialogue. Importantly, a World Bank ICT Technical Specialist was not on the team for the majority of the life of the project. At times, there were difficult technical decisions that the team was not equipped to make regarding SAFI II. This situation improved toward the end of the project, but there were missed opportunities early on. 70. Transaction costs were high with the Government to keep the project moving forward and in response, the team did increase supervision and monitoring of the project. The Country Management Unit was very engaged in this project, participating in the donor roundtables and in the high-level meetings with the Government. Workshops with the PIU and Procurement and Financial Management Specialists were also scheduled to provide information and build capacity about World Bank policies and procedures. 71. Once challenges in the integration of the conceptual modules of SAFI II became apparent, the World Bank financed an evaluation of the IT system to provide recommendations on the way forward. Though the World Bank stopped financing the technological development of SAFI II, periodic video and audio conferences were scheduled to closely follow up on the conceptualization and programming of activities, and the team continued to provide advice on SAFI II development. This was done through hiring independent consultants and utilizing supervision missions to convene stakeholders and discuss achievements and pending issues. The results of these meetings were documented in detailed Aide Memoires. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory 72. Overall, World Bank performance is considered to have been Moderately Unsatisfactory. The design of the reform path was overly ambitious for the country context. The calibration of activities and components to the risks could have been better reflected in design. During supervision, the World Bank team was somewhat inconsistent in its focus across various components, though over time, supervision across all components did improve. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory 73. The Government s political commitment was highly uneven across components over the course of the project, negatively affecting the achievement of the PDO. Component 2, in particular, suffered from the absence of leadership and coordination of donors involved. 20

31 Problems in the coordination of donors led to multiple changes in activities supported by the project. While a Governance Committee was formed to support the PFM reforms, led by senior officials, many of the difficult decisions and mediation required to ensure integration of SAFI II modules were not resolved through the committee. While senior leadership expressed interest in moving forward with reform implementation, there was substantial resistance to reform at the Director level. The Government was not able to effectively coordinate Technical Directors or donors in support of its fiscal and PFM reform program, leading to serious problems in developing an integrated financial management information system. Advances in Component 3 (transparency), on the other hand, were underpinned by a clear commitment and interest of the Government in consolidating transparency and accountability reforms. (b) Implementing Agency or Agencies Performance Rating: Unsatisfactory 74. The implementing agency for the project was DINAFI, which also housed the PIU. High turnover in DINAFI and a small PIU undermined the pace and quality of implementing project activities. To save resources, the Government decided to not fully staff the PIU and resisted hiring additional procurement staff into the project. The PIU was small and for the majority of implementation the only full-time person was the Financial Management specialist. Implementation issues were not resolved in a timely fashion. No staff were hired to perform M&E functions as originally envisioned in the PAD and this delayed reports on project progress. The pace of implementation in the final year of the project (reflected by an increase in disbursements) improved as a result of more direct engagement of senior management and the hiring of a Procurement Specialist for the PIU. (c) Justification of Rating for Overall Borrower Performance Rating: Unsatisfactory 75. The overall borrower performance is Unsatisfactory because of major shortcomings in the Government s coordination of the project, the PFM reform trajectory, and issues of capacity of the implementing agency to guide project implementation. 6. Lessons Learned 76. There are several lessons that can be drawn from the design, implementation, and results of this operation of relevance for both similar projects dealing with customs, PFM, transparency reforms, and the management World Bank operations in El Salvador. These lessons are drawn from project documents and interviews with staff who worked on the project both inside and outside of the Government. Project Management 77. The lack of full-time staff working for the PIU limited the Government s support to the implementation of activities. For the majority of the project, the PIU consisted mainly of a few full-time DINAFI staff assigned to the PIU on an honorary basis, where the functions of the PIU were added to their day-to-day tasks. The only exception was the contracting of a Financial Management Specialist in 2012, as a result of pressure from the World Bank. The PIU should have been separated from the implementing agency (DINAFI) to ensure some independence and 21

32 oversight in its functions. The PIU would have benefited from the presence of Procurement, M&E, and Technical Specialists to support the implementation of activities. 78. The absence of a dedicated Project Manager to guide the development of SAFI II led to significant integration issues. While the Government attempted to hire a Project Manager on various occasions, the absence of a full-time dedicated Project Manager throughout implementation significantly slowed progress and left many conceptual and functional issues unresolved. The World Bank attempted to fill the gap through increased supervision, but ultimately this was not sufficient to manage the day-to-day issues of system integration. 79. The absence of appropriate technical expertise (in the areas of financial management information system and PFM) on the World Bank team during the first years of project implementation contributed to oversights in design and limited the ability of the World Bank to advise the Government during reform implementation. This was particularly acute at the start of project implementation. Consultants were hired to address this issue and eventually, the design of the reform strategy for SAFI II implementation was scaled back and sequenced. 80. The project suffered from a lack of engagement of senior public officials in resolving technical disagreements in the development of SAFI II. During the implementation of the activities related to SAFI II, serious concerns about the direction, pace, and underlying reforms required for integration emerged. While a Governance Committee was formed, led by senior officials, many of the difficult decisions and mediation required to ensure integration would occur was not resolved. Resistance to such reforms is common to these types of projects, and more direct senior engagement and communication could have facilitated a smoother transition. The design of the project should also have carefully considered the need for change management interventions and sought to build broader coalition for the reform at the technical level to advance reforms. 81. Standard World Bank supervision missions (two per year) were not sufficient in such a challenging reform environment. As illustrated by the numbers of weeks dedicated to supervision, these missions increased significantly between 2014 and In general, financial management information system reforms require greater technical guidance to launch the process, ensure the creation of technical conditions and coordination arrangement among all stakeholders, and follow up, given the size and characteristics of the reform and the capacity of the counterpart, for whom most of the reforms were not only complex but also new (for example, program-based budgeting, International Standards of Public Sector Accounting). A more robust capacity assessment was needed before project implementation. 82. A positive lesson from the operation was the engagement of the opposition in Congress to help get the project approved. The team adapted the project design to reflect the concerns of the opposition by strengthening the transparency component and demonstrating how such interventions would directly contribute to citizen access to information and government services. This adaptation facilitated project approval. Reform Design 22

33 83. A deeper assessment of the viability of the proposed PFM reforms should have been undertaken at effectiveness because of the substantial delay in project approval. As a result of the 18-month delay in the effectiveness of the project, the operating environment in which the project was conceived changed significantly, with donors directly engaged in many of the activities originally foreseen in the PAD. During the first restructuring, some of the components were adjusted to reflect changes in the operating environment, but no thorough assessment was conducted nor was the project extended. Major procurements were not prepared in advance and the teams were under immense pressure to proceed as quickly as possible with procurements, rushing studies and sometimes fragmenting activities to simplify the procurement process and save time. 84. Combining major PFM reforms with a move to integration and automation created reform overload. The lack of sequencing of the public financial reforms included under the project caused substantial confusion and overload of the technical directions involved. Not until the ministry adjusted its reform strategy to proceed in phases did the situation improve. A deeper understanding of the prerequisites needed for financial management information system implementation and reform sequencing should have gone into preparation. 85. Coordination of various donor interventions by the Government was weak, leading to overlapping interventions and divergent interests between donors, undermining the overall coherence of the Government s PFM reform strategy. Various sources of funds grants, loans, consultant projects from several donors were utilized by the Government to fund the implementation of PFM reforms. Donors were not adequately coordinated, resulting in different methods and approaches and conflicting reform advice to the Government. 86. More careful consideration of the implications of selecting an LDSW solution should have preceded implementation. The limited capacity of the Government to steer the reforms, the number of donors involved, and the lack of a Project Director all undermined the Government s ability to effectively implement an LDSW solution. It is not clear from project documentation that the full implications of pursuing an LDSW were considered at the onset. Procurement 87. The PIU s and the implementing agencies limited degree of experience working with World Bank procurement procedures negatively affected the rhythm of project execution. At the design phase, the project entailed the procurement of complex goods and services, requiring a substantial degree of technical expertise, a solid understanding of the market, and familiarity with the World Bank s procurement rules and procedures. The PIU did not have a full-time Procurement Specialist until 2015 but rather had to rely on the MoF staff to fulfill the functions. Moreover, the implementing agencies were unfamiliar with World Bank procurement procedures and in some cases, lacked the technical capacity to develop the required specifications and undertake market analysis. 88. The absence of a strategic approach to the planning of ICT procurement at the onset led to major failures in procurement. The limited understanding by both the Government and the World Bank of the products and services of the market for customs and ICT software included under the project contributed to major failures in procurement. On the World 23

34 Bank side, restrictive terms and conditions effectively dissuaded domestic providers from participating in the selection process and many did not know how to respond to the World Bank s call for proposals. A better understanding of the market dynamics at project design would have substantially increased the viability of the International Competitive Bidding and National Competitive Bidding procurement processes. Strategic planning of ICT procurement should be an integral part of project design and task teams can benefit from regularly updated resources (for example, financial management information system database, PFM systems and e-services, Digital Governance database) to minimize the number of complex ICT procurement packages, optimize technical design, and learn more about cost/time implications of different ICT solutions and development paths. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies Not applicable. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders Not applicable. 24

35 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$, millions equivalent) Components Appraisal Estimate Revised Estimate Actual/Latest Estimate (US$, millions) (US$, millions) (US$, millions)* Strengthening tax collection agencies Modernizing of public expenditures and financial management Enhancing and piloting information management and public sector transparency initiatives Project coordination and strengthening of the MoF Total baseline cost Physical contingencies Price contingencies Total project costs Project Preparation Fund Front-end fee IBRD Total financing required *Source: Interim Financial Report from June 30, 2016, covering the first semester of Therefore, the final numbers may vary from what is presented in this table. (b) Financing Source of Funds Type of Cofinancing Appraisal Estimate (US$, millions) Actual/Latest Estimate (US$, millions) Percentage of Appraisal Borrower International Bank for Reconstruction and Development

36 Annex 2. Outputs by Component Component 1: Strengthening tax collection agencies Indicator End Target Status at Closing Increased efficiency of customs through increased number of customs points with nonintrusive control methods financed by the loan 7 customs points Not achieved. 6 customs points* Hours to clear merchandise from the moment it arrives to custom terminal until it is released 17 hours Partially achieved. 21 hours (from baseline of 60 hours) % increase of web-processed 98% Achieved. transactions in customs Note: *Though none of the current six customs points were funded by the loan, the strategy that led to the introduction of this strengthened customs management and control mechanism was originated as part of the project. Before the project, this strategy and mechanism in customs points did not exist. Summary of Outputs Strengthening of data processing capacities for the DGA and the DGII Audit Units, the DGT tax collecting units, and users of SIDUNEA World through the acquisition of 1,237 computers (laptops and desktop). Improvement of control mechanisms for customs through the implementation of a video surveillance system with more than 100 cameras, using the corporate data network of the ministry. Customization and implementation of SIDUNEA World, as well as significant advances in its respective integration with other customs systems. Strategy to incorporate nonintrusive mechanism (that motivated the integration of scanners in six customs points). Component 2: Modernizing of public expenditures and financial management Indicator End Target Status at Closing Stronger planning, budgeting, treasury, and accounting systems in place, in line with international standards Development of the new SAFI II module for Budget Execution will include program-based budgeting. The new SAFI II system is fully developed and ready to be launched. It includes programbased budgeting and payment processing through the Treasury Single Account A new module for budget execution in SAFI II that includes program-based budgeting has been developed. Partially achieved. The MoF concluded the development of the budget formulation and budget execution modules of SAFI II. Program-based budgeting is fully embedded in SAFI II as well as the MTEF. Programs are also results-based and adopt the logic framework methodology. Achieved. The MoF concluded the development of the budget formulation and budget execution modules of SAFI II. Program-based budgeting is fully embedded in SAFI II. Programs are also 26

37 Increased capacity in budget planning and formulation through the new integrated financial system Summary of Outputs 12 training sessions results-based and adopt the logic framework methodology. Achieved. 17 training sessions financed by the group of donors with the objective of increasing capacity in managing public expenditures to take full advantage of SAFI II functionalities. Conceptual design document for SAFI II. IT development for SAFI II concluded for the modules of budget preparation, budget execution, and human resources. Updating the skills of the technical staff of the central institutional Finance Directorates in the core business practices and computer areas through training and hiring of specialized consultancies (budget, Treasury, investment, human resources, and so on). Results-based budgeting outputs: Methodological guide; guide for the validation of programs; training of 1,820 public servants; review of program classification structure for 25 Central Government institutions, 72 decentralized institutions, 4 State Owned Enterprises, and pilot in 5 entities. Updating the communications infrastructure of the MoF through the acquisition of structured cabling with a shelf life of 15 years; networking equipment for the core of the corporate network, robust enough to support current and new services for the next five years. Reinforcement of the data processing and storage infrastructure of the MoF Data Center, by implementing a private cloud, based on the virtualization platform that makes the use of computational resources more efficient and strengthens aspects of security, scalability, and availability of current and new business services for the next 10 years. Strengthening of information security mechanisms. Renovation of the client computer plant of the MoF with the acquisition of 768 computers for the central institutional Finance Directorates; extending the shelf life by five years. Component 3: Enhancing and piloting information management and public sector transparency initiatives Indicator End target Status at closing Implementation of training and 22 initiatives have been dissemination initiatives to implemented. Institutional capacity Achieved. 50 initiatives strengthen Government of the Government is strengthened implemented. transparency, citizen participation, and knowledge about right of and access to public information access to public information and 27

38 Government transparency tools has increased among public servants and the general public Summary of Outputs Promotion of the national and international dissemination of the Law on Access to Public Information through various mechanisms: o o o o 2 dissemination campaigns for three months Training of 40 leaders of nongovernmental organizations on the Law on Access to Public Information Printed materials and printing in Braille of the law Training of 1,600 leaders from the central and eastern parts of the country Technical assistance support to the development of two web portals (InfoÚtil and Open Government) Technical assistance and equipment for 14 regional OIRs and 42 OIRs in the municipalities Carrying out of the first Salvadoran Open Government hackathon to find innovative ways for the Government to provide information services (100+ youth participants, 3 apps winners) Two studies into the typology of information for public consumption Study into the public usefulness of the Law on Access to Public Information Computer equipment to promote a pilot plan of face points for the multichannel window, based on the experience of Uruguay Establishment of a universal and standardized system for the receipt of complaints by the OIRs and their analysis by those in charge of the internal audit Development of the norms for the management of archives within the framework of the Law on Access to Public Information through one university diploma (40 public servants), workshops for 80 public servants, and 1 validated national normative framework Transparency in the information on the recruitment of human resources in the Executive Body o Technical assistance to 12 autonomous institutions and methodology of monitoring compliance with the publication of information 28

39 o Workshops with 80+ public institutions, tutorial videos Support in the implementation of the Citizen Care System Exchange visit to the Brazilian Comptroller-General (Controladoria Geral da União CGU) on the implementation of efficient mechanisms for the management of requests for public information Promotion of citizen culture of transparency and accountability 29

40 Annex 3. Economic and Financial Analysis As this was a technical assistance project devoted to a range of public administration reforms, computations of rate of return and cost effectiveness are difficult to assess and do not capture the range of intended benefits from the reforms. Nevertheless, it was expected to have a positive fiscal impact and generate economic gains that were to come from more efficient use of public finances through reductions in tax evasion and smuggling, improved cash management, and increased competition in purchasing. Because the project was restructured to focus on customs, PFM, and transparency activities, table 3.1 summarizes some of the potential efficiency/fiscal and social impact of project activities in each of the components. No formal studies were carried out during project implementation to verify these results, so these gains are considered indicative. Component 1. Strengthening tax collection agencies 2. Modernizing of public expenditures and financial management Activities Strengthening of the Customs Office processes and procedures Strengthening of the Customs Office IT systems Implementation of a Treasury Single Account and electronic payments Economic Gains/Social Gains (based on Self-Reporting-Expected/Realized) Realized private sector efficiency gains from the reduction in hours to clear merchandise from the moment it arrives to the customs terminal until it is released (from 21 to 17 hours). Realized private sector efficiency and effectiveness gains from increased number of customs points with nonintrusive control methods financed by the loan. The new process is faster, more accurate, and less intrusive. (for example, the opening of packages). Realized effectiveness gains in the control of customs merchandise through the implementation of a system of video surveillance with more than 100 cameras. Realized private sector efficiency gains from the increase in web-processed transactions in customs Expected efficiency gains from time-savings and processing moving toward an integrated, secure, and web-based customs platform (SIDUNEA World) Partially realized private sector efficiency gains from the adoption of a Treasury Single Account reduction in the time required for the Government to pay private contractors. Earlier, it took 2 weeks (15 days) for the funds to arrive to providers; now it takes 1 day or less. Full implementation (all institutions and payroll expenses) is expected by fiscal Realized efficiency gains for the central public administration from the introduction of payment cards for small purchases to facilitate the processes of purchasing. Upgrade of the financial management information system (SAFI II) Expected efficiency gains from time-savings and processing of the preparation and execution of the budget in SAFI II for the central public administration Introduction of resultsbased budgeting (program-based budgeting) Expected efficacy gains in the use of a program-based budget linked to results for the central public administration 30

41 3. Enhancing and piloting information management and public sector transparency initiatives Upgrade of the MoF s IT infrastructure Dissemination and implementation of transparency initiatives Realized private sector and citizen time-savings gains from the MoF s services available online as a result of upgrades in the MoF s IT infrastructure. Services are (a) tax refund consultations; (b) presentation of information and declarations and payment of taxes online; (c) request forms and consultation of tax solvency; and (d) requests for cancellation, receipt, or modifications of one s Tax Identification Number for naturalized citizens. Realized time-savings gains for the public through the InfoÚtil portal, which provides access to useful day-to-day information on health, education, economic, and financial services. The portal has received more than 5.3 million visits in the last 3 years. Realized time-savings gains for those seeking a job in the public sector as a result of the Public Jobs Vacancy Portal, which consolidates public vacancies from 45 institutions. Realized time-savings gains for citizens to submit complaints to the Government (and improved access to information) through the creation of 83 OIRs within the Central Government, which have received more than 80,000 information requests, from which 90% have been responded to. 1. Despite some of these gains, overall efficiency of the project is rated negligible given the elapsed time and modest achievements in key project interventions. Moreover, many of the expected benefits will only be realized if the Government fully adopts the new IT systems for day-to-day operations (for example, SAFI II and SIDUNEA World). 31

42 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility Lending Alberto Leyton Task Team Leader LCC2C TTL Alejandro Alcalá Senior Counsel LEGLA Counsel Alma Hernandez Program Assistant LCCSV Administrative support Alvaro Larrea Procurement Specialist LCSPT Procurement Carmen Zuleta Consultant LCSPS SAFI II Dean Thompson Consultant LCSPS Enrique Fanta Senior Public Sector Specialist LCSPS Customs specialist Fabienne Mrozka Financial Management Specialist LCSFM Financial management Henry Forero Senior Information Officer CITPO IT specialist Jania Ibarra Operations Analyst LCSPS Operational support João Veiga Malta Senior Procurement Specialist LCSPT Procurement Karla Lopez Flores Language Program Assistant LCSPS Administrative support Manuel Rosales Consultant LCCSV Marcos Mendiburu Social Development Specialist WBIGV Transparency component Mayra Moran Consultant LCSPS Pedro Aritzi Public Sector Specialist LCSPS Supervision/ICR Alejandra Ramon Y Martinez De Alva Program Assistant LCC1C Administrative support Ana Sirani Romero Mata GGODR Angela Nieves Marques Porto Consultant GGO16 Operational support Arturo Herrera Gutierrez Practice Manager GGO16 Practice Manager Daniel Jorge Arguindegui Senior Procurement Specialist GGO04 Procurement David Santos Ruano Public Sector Specialist GGO16 IT specialist Enrique Fanta Ivanovic Senior Private Sector Specialist GTC04 Customs specialist Jania Ibarra Operations Officer LCCSV Operational support Joanna Alexandra Watkins Senior Public Sector Specialist GGO16 TTL of ICR Jose Simon Rezk Senior Financial Management Specialist GGO22 Financial management Jovana Stojanovic Operations Officer LCC2C Operational support Maria Guadalupe Toscano Nicolas Public Sector Management Specialist GGO16 TTL at closing Maryanne Sharp Country Operations Advisor LCC2C May Cabilas Olalia Senior Public Sector Specialist GGO16 Operations support Mayra Del Carmen Alfaro De Moran Senior Private Sector Specialist GTCLA Reidar Kvam Safeguards Specialist GSUSD Safeguards specialist Sandra Lisette Flores De Mixco Financial Management Analyst GGO22 Financial management (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$, thousands (including travel No. of Staff Weeks and consultant costs) Lending FY

43 Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$, thousands (including travel No. of Staff Weeks and consultant costs) FY FY FY FY Total: Supervision/ICR FY FY FY FY FY FY FY FY Total:

44 Annex 5. Beneficiary Survey Results Not applicable. 34

45 Annex 6. Stakeholder Workshop Report and Results Not applicable. 35

46 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR No comments were received from the Borrower. 36

47 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable. 37

48 Annex 9. List of Supporting Documents Dener, Cem, Joanna Watkins, and William Dorotinsky Financial Management Information Systems: 25 years of World Bank Experience on What Works and What Doesn t. World Bank Study. Decree No. 516, Ley Orgánica de Administración Financiera del Estado, El Salvador. IMF (International Monetary Fund) El Salvador: Staff Concluding Statement of the 2016 Article IV Mission. May 6, (accessed at McLinden, Gerard et al Border Management Modernization. Washington, DC: World Bank. InfoÚtil Evaluación Ciudadana del Portal Infoútil. Secretaria Técnica de Planificación. El Salvador Productivo, Educado y Seguro. Plan Quinquenal de Desarrollo ( ). World Bank, Aide Memoires, Back-to-Office Reports, Project ISRs, and Project Assessments. World Bank El Salvador - Country Partnership Strategy Report No SV. World Bank. Restructuring Paper on a Proposed Project Restructuring of Fiscal Management and Public Sector Performance Technical Assistance Loan Project. Report No SV World Bank. Restructuring Paper on a Proposed Project Restructuring of Fiscal Management and Public Sector Performance Technical Assistance Loan Project. Report No. RES14771,

49 Annex 10. Achievement of Objectives Analysis Type of Evidence/ Indicator PDO-level Results Indicators (Objective to which they correspond) Target Final Value Level of Achievement To strengthen the institutional capacity of specific government processes and agencies to increase the effectiveness and efficiency of revenue and expenditure management and enhance accountability and transparency in the public sector Component 1: Strengthening tax collection agencies PDO Indicator PDO Indicator Intermediate Indicator Intermediate Indicator Intermediate Indicator Tax collection as a percent of GDP increases at least 1% as a result of greater effectiveness and efficiency Increased efficiency of customs through increased number of customs points with nonintrusive control methods financed by the loan Hours to clear merchandise from the moment it arrives to customs terminal until it is released % increase of web-processed transactions in customs % recovery of tax arrears portfolio Development of the conceptual model for the programmatic classification in the budget Tax collection as percentage of GDP has increased at least 1% The conceptual model for the programmatic classification in the budget has been developed and the budget is structured by programs. Tax collection as a percentage of GDP was 15.8% by December 2013 Dropped 7 6 Partially achieved Partially achieved Achieved (December 2013) Dropped The conceptual model for the budgeting module was approved on October 15, 2012, and it includes the programmatic classification of the budget. Operalization of the conceptual plan in the new SAFI is still under development (December 2013). Dropped 39

50 Type of Evidence/ Indicator Supplemental evidence PDO-level Results Indicators (Objective to which they correspond) Target Final Value Level of Achievement The customization of SIDUNEA World is concluded for relevant customs business processes (declaration, transit, regimes, and control of deposits and guarantees); however, final adaptations will be done by the local team, including tests and stabilization of the solution. SIDUNEA World is expected to be launched in Launching SIDUNEA World will modernize customs management processes, with potential results in time-savings, accuracy of information, security of information, interoperability, and verification of information, accessibility, and so on. Component 2: Modernizing of public expenditure and financial management PDO Indicator PDO Indicator PDO Indicator Intermediate Indicator Stronger planning, budgeting, treasury, and accounting systems in place, in line with international standards Savings generated in central government purchasing are at least 10% by comparing the same type of items (by adopting optimizing procurement policies) The introduction of multiannual projections and programmatic structure in the budget strengthens strategic planning and the implementation of performance-based budgeting Development of the new SAFI II module for budget execution will include program-based budgeting The new SAFI II system is fully developed and ready to be launched. It includes program-based budgeting, payment processing through the Treasury Single Account The MoF concluded the development of the budget formulation and budget execution modules of SAFI II. Programbased budgeting is fully embedded in SAFI II as well as the MTEF. Programs are also results-based and adopt the logic framework methodology. n.a. n.a. Dropped Improvements in the ratings in PEFA indicator 12: Multiyear perspective in fiscal planning, expenditure policy and budgeting, and PEFA indicator 5: Classification of the budget A new module for budget execution in SAFI II that includes program-based budgeting has been developed. Several legal and methodological instruments approved to introduce multi-annual projections and program-based budgeting (December 2013) The MoF concluded the development of the budget formulation and budget execution modules of SAFI II. Programbased budgeting is fully embedded in SAFI II. Programs are also results-based and adopt the logic framework Partially achieved. Modules have been developed but are not fully operational. Dropped Achieved 40

51 Type of Evidence/ Indicator Intermediate Indicator Intermediate Indicator Intermediate Indicator Intermediate Indicator Supplemental evidence PDO-level Results Indicators (Objective to which they correspond) Increased capacity in budget planning and formulation through the new integrated financial system Availability of a new electronic purchasing system designed and implemented (COMPRASAL) Implementation of the Treasury Single Account and electronic payment module implemented in all Central Government, including the national hospitals network. The SIRH is updated and inter-operating with SAFI 12 training sessions Target A new electronic purchasing system designed and implemented in Central Government institutions (COMPRASAL) The Treasury Single Account and electronic payment module was implemented in all Central Government institutions, including the national hospitals network Budget formulation for human resources and payroll is automatic and linked to SAFI methodology. Final Value 17 training sessions financed by the group of donors with the objective of increasing capacity in managing public expenditures to take full advantage of the SAFI II functionalities COMPRASAL has been developed and is working although not 100% of the Government agencies are using the system (December 2013). Treasury Single Account is under development within the new SAFI (December 2013) SIRH is being developed under the new SAFI (December 2013) Level of Achievement Achieved Dropped Dropped Dropped Program and results-based budgeting is close to a reality. The legal framework is already set as well as the practice in the institutions. In the last two years, the institutions have been designing and integrating their programs. As of September 2016, there are 79 (out of 84, excluding 30 hospitals) institutions with a programmatic structure of the budget that are ready to start integrating Budget 2018 under a program approach. The launching of the reform was deferred for one year (from Budget 2017 to Budget 2018) to match this policy reform with the tool that would make it operational (SAFI II - budget formulation module). The MTEF is also embedded in the budget reform as well as in SAFI II and would be put in practice in Budget This is a major reform that would should improve budget credibility, critical to the current fiscal situation. The Treasury Single Account is operating for Central Government payments, except for payroll expenses. Full implementation (all institutions and payroll expenses) is expected for fiscal year 2018 once SAFI II is fully operational. The budget formulation and budget execution modules for SAFI II are fully developed and will be used in May Treasury and accounting modules are expected to be finished in the next semester. The technical platform for web-based access to SAFI II is in development. 41

52 Type of Evidence/ Indicator PDO-level Results Indicators (Objective to which they correspond) Target Final Value Level of Achievement ICT infrastructure of the MoF has been substantially upgraded. Hardware infrastructure to run the system is in place (new computers, servers, updating of critical networks, security solutions, data warehouse). About 90% of the hardware of the ministry has been updated. The ministry now has videoconferencing facilities and this has been helpful for all of the ministry s projects and activities. This infrastructure platform has also allowed for an increase in the number of services that the MoF can offer online. The upgraded infrastructure contributed directly to improving the capacity of the MoF to process, store, and increase accessibility and security of the following citizen services available online: o Tax refund consultations o Presentation of information and declarations and payment of taxes online o Request forms and consultation of tax solvency o Requests for cancellation, receipt, or modifications of the Tax Identification Number for naturalized citizens This represents about 20% of total number of services available on the MoF s portal. Additionally, the hardware upgrade supported the use of the new public procurement system (COMPRASAL II), which is being used by all the central and municipal government s contracts units and the support of the Treasury Single Account, increasing the security and efficiency of the payment processes. Transfer of ICT knowledge to the ministries staff with various consultants teaching the teams how to program will be helpful for future iterations of the software (SAFI III or IV). Component 3: Enhancing and piloting information management and public sector transparency initiatives PDO Indicator Intermediate Indicator Supplemental Evidence Implementation of training and dissemination initiatives to strengthen Government transparency, citizen participation, and access to public information 22 initiatives have been implemented. Institutional capacity of the Government is strengthened and knowledge about right of access to public information and Government transparency tools has increased among public servants and the general public. 50 initiatives implemented Achieved Expansion and improvements to the InfoÚtil Portal (originally developed with funding from the IADB), enabling citizens to access useful day-to-day public information (public procedures and services) for health, education, economy, and finances. The portal contains information on different topics health, education, economy, finances of use for citizens and business. This portal allows the citizens to obtain updated information and to submit suggestions, opinions, or complaints about public management. The portal has received more than 5.3 million visits in the last 3 years. o In 2013, the portal had 29 databases. In 2016, the portal had 53 databases. o The number of users has increased from 151,312 in 2013 to 818,020 in o Site visits have increased from 196,926 in 2013 to 1,188,964 in o As of 2016, 29 public institutions are involved in the portal such as the Ministries of Health, Education, and Labor. 42

53 Type of Evidence/ Indicator PDO-level Results Indicators (Objective to which they correspond) Target Final Value Level of Achievement Results of the InfoÚtil user satisfaction survey undertaken between 2013 and 2014: o Number of respondents: 85. o 85% found the portal to be very useful o 79% considered the website easy to access, attractive, and dynamic. o 81% of respondents encountered the information they were looking for, while 19% did not find the information. o 80% of respondents found the website easy to understand, while 20% did not. o 91% of respondents would recommend the portal to others. Support to the Public Sector Jobs Vacancy Portal that was launched in December 2015 (consolidating vacancies for 45 institutions) and the design and creation of the Social Programs Portal that was launched by the end of The project also contributed with the design of the proposal for the open data policy and the creation of the Gobierno Abierto Portal which integrated information, requests for information under way, complaints, and ranking of the performance of institutions. Different actions to disseminate the Law of Access to Public Information across the country by different means (television campaigns, brochures, trainings in schools and public institutions). 83 OIRs have been created within the Central Government. As of November 2013, they had 24,021 requests for information, of which 90% have been responded to (Source: Rendición de Cuentas Fondos de Cooperación ). The SCPT conducted a ranking of all the institutions for monitoring their compliance with the law. Creation of the app Camino a la U which provides information about universities, aptitude test results, and scholarships. Introduction and dissemination of a structured process for holding public institutions to account for results. In 2010, 37 central institutions participated in the process and by 2012, 89 central institutions and 14 Management Cabinets (Gabinete de Gestión) from each of the regional departments, and schools (approximately 5,000). 43

54 Annex 11. El Salvador PEFA Indicators 2009 versus I. CREDIBILITY OF THE BUDGET 1. Aggregate expenditure out-turn compared to original approved budget C A 2. Composition of expenditure out-turn compared to original approved budget B A 3. Aggregate revenue out-turn compared to original approved budget A A 4. Stock and monitoring of expenditure payment arrears B NR II. COMPREHENSIVENESS AND TRANSPARENCY 5. Classification of the budget C C 6. Comprehensiveness of information included in budget documentation A B 7. Extent of unreported government operations A C+ 8. Transparency of inter-governmental fiscal relations B+ B+ 9. Oversight of aggregate fiscal risk from other public sector entities C+ C 10. Public access to key fiscal information B B III. POLICY-BASED BUDGETING 11. Orderliness and participation in the annual budget process B B 12. Multi-year perspective in fiscal planning, expenditure policy and budgeting C+ C+ IV. PREDICTABILITY AND CONTROL IN BUDGET EXECUTION 13. Transparency of taxpayer obligations and liabilities B B+ 14. Effectiveness of measures for taxpayer registration and tax assessment B B 15. Effectiveness in collection of tax payments B+ D+ 16. Predictability in the availability of funds for commitment of expenditures B+ B+ 17. Recording and management of cash balances, debt and guarantees B B 18. Effectiveness of payroll controls A C+ 19. Transparency, competition and complaints mechanisms in procurement B+ B 20. Effectiveness of internal controls for non-salary expenditure B+ C+ 21. Effectiveness of internal audit C+ C+ V. ACCOUNTING, RECORDING AND REPORTING 22. Timeliness and regularity of accounts reconciliation B+ B+ 23. Availability of information on resources received by service delivery units B B 24. Quality and timeliness of in-year budget reports B+ B+ 25. Quality and timeliness of annual financial statements A B+ VI. EXTERNAL SCRUTINY AND AUDIT 26. Scope, nature and follow-up of external audit C+ D+ 27. Legislative scrutiny of the annual budget law C+ B+ 28. Legislative scrutiny of external audit reports D D DONOR PRACTICES D-1 Predictability of Direct Budget Support A D D-2 Financial information provided by donors for budgeting and reporting on project D D and program aid D-3 Proportion of aid that is managed by use of national procedures D C Source: PEFA. El Salvador Assessment Data and Report. Accessible from: 44

55 MAP 45