IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-37570) ON AN IDA CREDIT IN THE AMOUNT OF SDR5.80 MILLION (US$7.77 MILLION EQUIVALENT) TO THE

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Document of The World Bank Report No: ICR IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-37570) ON AN IDA CREDIT IN THE AMOUNT OF SDR5.80 MILLION (US$7.77 MILLION EQUIVALENT) TO THE KYRYGZ REPUBLIC FOR A GOVERNANCE TECHNICAL ASSISTANCE PROJECT June 24, 2014 Public Sector Institutional Reform Cluster Poverty Reduction and Economic Management Unit Europe and Central Asia Region 1

2 CURRENCY EQUIVALENTS (Exchange Rate Effective May 31, 2014) Currency Unit = Som Som 1.00 = US$ US$ 1.00 = Som FISCAL YEAR ABBREVIATIONS AND ACRONYMS AC CAS CC CFAA CoA CSMA DFID ECA EU-TACIS FM FMIS FSU GDP GSAC GTAC HRMIS IBRD IBS ICB IDA IDF IMF LCS M&E MOF MOU NPRS NBKR PCL PCN PDO PER PIU PMG PPP PRGF PSRMAC SDR SES SPS TGL TMIS TSA UNDP USAID Accounts Chamber Country Assistance Strategy Coordinating Council Country Financial Accountability Assessment Chart of Accounts Civil Service Management Agency Department for International Development (United Kingdom) Europe and Central Asia European Union TA program for CIS countries Financial Management Financial Management Information System Former Soviet Union Gross Domestic Product Governance Structural Adjustment Credit Governance Technical Adjustment Credit Human Resource Management Information System International Bank for Reconstruction and Development Information Business Systems International Competitive Bidding International Development Association Institutional Development Fund International Monetary Fund Law on Civil Service Monitoring and Evaluation Ministry of Finance Memorandum of Understanding National Poverty Reduction Strategy National Bank of the Kyrgyz Republic Pharm Control Laboratory Project Concept Note Project Development Indicators Public Expenditure Review Project Implementation Unit Project Management Group Purchasing Price Parity Poverty Reduction and Growth Facility Arrangement Public Sector Resource Management Adjustment Credit Special Drawing Rights Sanitary and Epidemiology Service State Personnel Service Treasury General Ledger Treasury Management Information System Treasury Single Account United Nations Development Program United States Agency for International Development Vice President: Country Director: Sector Manager: Project Team Leader: ICR Team Leader: Laura Tuck (ECAVP) Saroj Kumar Jha (ECCU8) Adrian Fozzard (ECSP4) David Nummy (ECSP4) Zachary Mills (ECSP4) 2

3 KYRGYZ REPUBLIC Governance Technical Adjustment Credit Contents Data Sheet... 4 A. Basic Information... 4 B. Key Dates... 4 C. Ratings Summary... 4 D. Sector and Theme Codes... 5 E. Bank Staff... 5 F. Results Framework Analysis... 6 G. Ratings of Project Performance in ISRs... 9 H. Restructuring (if any) I. Disbursement Profile Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1. Project Costs and Financing Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Bank Lending and Implementation Support/Supervision Processes Annex 5. Beneficiary Survey Results Annex 6. Stakeholder Workshop Report and Results Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Annex 9. List of Supporting Documents MAP

4

5 Data Sheet A. Basic Information Country: Kyrgyz Republic Project Name: Governance Technical Assistance Project Project ID: P Credit Number: IDA ICR Date: 06/04/2014 ICR Type: Core ICR Lending Instrument: TAL Borrower: Original Total Commitment: Revised Amount: XDR 4.14M Environmental Category: C THE KYRGYZ REPUBLIC XDR 5.80M Disbursed Amount: XDR 4.14M Implementing Agency: Ministry of Finance of the Kyrgyz Republic Co-financiers and Other External Partners: N/A B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 05/02/2002 Effectiveness: 08/28/ /28/2003 Appraisal: 03/26/2003 Restructurings: 05/12/ /28/ /20/ /14/ /28/ /18/2013 Approval: 05/15/2003 Mid-term Review: 02/23/ /03/2012 Closing: 12/31/ /31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Moderately Unsatisfactory High Moderately Unsatisfactory Moderately Unsatisfactory 4

6 C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Satisfactory Quality of Supervision: Moderately Moderately Implementing Agency: Unsatisfactory Unsatisfactory Overall Bank Performance: Moderately Unsatisfactory Overall Borrower Performance: Moderately Unsatisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: No Yes Moderately Unsatisfactory Quality at Entry (QEA): Quality of Supervision (QSA): Rating Highly Satisfactory None D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration Theme Code (as % of total Bank financing) Administrative and civil service reform Public expenditure, financial management and procurement Other accountability/anti-corruption 22 0 Other public sector governance 22 0 Poverty strategy, analysis and monitoring 11 0 E. Bank Staff Positions At ICR At Approval Vice President: Laura Tuck Johannes Linn Country Director: Saroj Kumar Jha Dennis de Tray Sector Manager: Adrian Fozzard Helga Muller Project Team Leader: David Michael Nummy Jit Bahadur S. Gill ICR Team Leader: Zachary Mills ICR Primary Author: Zachary Mills 5

7 F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project objectives are to strengthen the Borrower s institutional capacity to implement the measures under the program outlined in the GSAC Program Document and to assist with the Treasury modernization. Revised Project Development Objectives (as approved by original approving authority) The objectives were not changed. (a) PDO Indicators Indicator Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Indicator 1: Establishment of a modern computerized treasury system. Actual Value Achieved at Completion or Target Years Most treasury Automated and N/A Not achieved. The introduction of operations carried modernized treasury the automated treasury system out manually. system. could not be completed within the extended closing date. The Government canceled the contract with the vendor on December 16, Indicator 2: Public perception of transparency, responsiveness and integrity of public sector as measured by annual surveys has improved. No baseline surveys were conducted. (b) Intermediate Outcome Indicators Indicator Baseline Value Undefined improvement as a target value was not defined. Original Target Values (from approval documents) N/A Formally Revised Target Values Indicator 1: Number of declarations of high state officials published annually. No declarations or provision for declarations. Increase in number of declarations. Not achieved. No surveys were conducted under the project. Actual Value Achieved at Completion or Target Years Achieved. All high state officials present and publish declarations. Indicator 2: Improved access, reliability and timeliness of information about budget execution, to public officials and civil society. 6

8 Indicator Baseline Value Original Target Values (from approval documents) Greater access to and quality of budget execution reports. Formally Revised Target Values N/A Actual Value Achieved at Completion or Target Years Budget reports Achieved. Comprehensive fragmented and summary budget reports made not widely publicly available within 6 available. months of budget execution. Public procurement information more readily available. Indicator 3: Access to legal database of the Ministry of Justice available to the public online. Legal information Legal information not readily accessible to the available. public online. N/A Achieved. Online access to the centralized database of legal information is available through the website of the Ministry of Justice: Indicator 4: Government Public Information Center established and functioning. Limited accessibility of public information to Increased accessibility of Government information. N/A Achieved. Center fully established and significant number of users from the general public. (Target assumed based on private sector and formulation of indicator). civil society. Indicator 5: Number of draft laws and regulations discussed with the private sector and civil society, prior to their adoption. Limited input from private sector and civil society on Government legislation. Target value not defined. N/A Achieved. Public steering councils were established comprising representatives from civil society, think tanks, professional and sector unions, and experts to provide public opinion on formulating and implementing government policy (Target assumed based on formulation of indicator). Indicator 6: Number of public grievances addressed annually by the President's Administration, the Office of the Government and the regulatory agencies covered by the GSAC. No data is kept Target value not on the number of defined. public grievances addressed. N/A Dropped. Activity dropped from project due to availability of grant financing from other donors. Indicator 7: Coverage, quality and timeliness of audit reports of (sic) Chamber of Control. 7

9 Indicator Baseline Value Reports of the Accounts Chamber (AC) confined to compliance with regulations Original Target Values (from approval documents) Target value not defined. Formally Revised Target Values N/A Actual Value Achieved at Completion or Target Years Achieved. Improved quality of audit reports, particularly on the annual budget over the course of the project (Target assumed based on formulation of indicator). Indicator 8: Increased usefulness of the budget as a strategic policy instrument, as a result of improvements in its formulation process, content and presentation. Budget poorly presented; allocations managed on a monthly basis according to cash availability. Target value not defined. N/A Partially Achieved. Transparent budget prepared on basis of medium-term framework. Revisions to budget typically take place through due process and not on basis of cash available to the Treasury. Nonetheless budget allocations are revised at mid-year based on cash availability. (Target assumed based on formulation of indicator). Indicator 9: Improved financial accounting and compliance with laws and regulations in three ministries, due to strengthened internal control systems. Not measured. Target value not defined. N/A Dropped. Activity dropped from project due to availability of grant financing from other donors. Indicator 10: Kyrgyzstandard reorganized to separate regulatory and enforcement functions. Kyrgyzstandard carried out both regulatory and enforcement functions. Clear separation of functions. N/A Achieved. Regulatory and enforcement functions fully separated. Indicator 11: Percentage of private firms participating in public procurement. Not measured. Target value not defined. N/A Achieved. 100 percent of companies currently participate in public procurement. (Target assumed based on formulation of indicator). Indicator 12: Improved allocation, predictability, availability of resources for health and education services. Not measured. N/A Target value not defined. Achieved. New financing formulas have been agreed for both sectors and are currently applied (Target assumed based on formulation of indicator). Indicator 13: Percentage of recruitments and promotions in the Civil Service made on a competitive basis. 8

10 Indicator Baseline Value Not available as the old civil service law did not require mandatory competitive recruitments or promotions. Original Target Values (from approval documents) Target values not defined. Formally Revised Target Values N/A Actual Value Achieved at Completion or Target Years Partially Achieved. Most chief and senior positions in the government are filled through a competitive selection process. (Target assumed based on formulation of indicator). Indicator 14: Independent Civil Service Agency established and operational. Independent Civil Fully functional and N/A Achieved. State Personnel Service Agency did not exist. independent civil service agency. Service established and authorized to develop, implement and ensure the sustainable functioning of state personnel policy in both central and local government bodies. G. Ratings of Project Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD millions) 1 06/25/2003 Satisfactory Satisfactory /11/2003 Satisfactory Satisfactory /24/2004 Satisfactory Satisfactory /29/2004 Satisfactory Unsatisfactory /17/2004 Satisfactory Satisfactory /29/2004 Satisfactory Satisfactory /23/2005 Moderately Satisfactory Moderately Satisfactory /28/2006 Moderately Satisfactory Moderately Satisfactory /29/2007 Moderately Satisfactory Moderately Satisfactory /24/2008 Moderately Satisfactory Moderately Unsatisfactory /17/2009 Moderately Satisfactory Moderately Satisfactory /30/2009 Moderately Satisfactory Moderately Satisfactory /21/2010 Moderately Satisfactory Moderately Satisfactory /15/2010 Moderately Satisfactory Moderately Unsatisfactory /12/2011 Moderately Satisfactory Moderately Satisfactory /24/2011 Moderately Satisfactory Moderately Satisfactory /28/2011 Moderately Moderately Unsatisfactory Unsatisfactory /16/2012 Moderately Moderately

11 19 04/07/ /13/ /07/2014 Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory H. Restructuring (if any) Restructuring Date(s) Board Approved PDO Change ISR Ratings at Restructuring DO IP Amount Disbursed at Restructuring in USD millions Reason for Restructuring & Key Changes Made 05/12/2008 No MS MU 1.80 Closing date extension 05/28/2009 No MS MS 2.08 To align the PDO text in the PAD with that in the legal agreement (part of a Bankwide exercise). 12/20/2010 No MS MU 3.00 Closing date extension 02/14/2011 No MS MS 3.50 Closing date extension 05/28/2013 No MU MU 6.00 Closing date extension No Canceling the undisbursed 12/18/2013 MU MU 6.35 credit amount of $2.5 million due to contract termination I. Disbursement Profile 10

12

13 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal The Kyrgyz Republic is a small, land-locked country with a population of 5.6 million. Following independence in 1991, income levels declined by about 60 percent as budgetary transfers from the Soviet Union ended, traditional trading relationships were disrupted, and the central planning system collapsed. At appraisal, agriculture accounted for about 40 percent of GDP and gold and electricity were the two most important exports. The country undertook major reforms beginning in 1993 to create the basic foundations of a market-oriented economic system, including a market-based price system, substantial private ownership of key productive assets, and an open international trade regime (the Kyrgyz Republic was the first CIS country to join the World Trade Organization). A large number of small and medium enterprises emerged during this period. The regional financial crisis in 1998 halted economic recovery. It also highlighted continuing vulnerabilities caused by large financial imbalances in the energy sector, fragile public finances, inefficient safety nets, and a poorly developed banking sector. The primary purpose of the Governance Technical Assistance Credit (GTAC) approved in May 2003 was to support the implementation of the reforms included in the twotranche Governance Structural Adjustment Credit (GSAC) which was approved by the Bank at the same time. These projects were launched in an improved macroeconomic context as the country had made significant progress in attaining macroeconomic stability and restarting economic growth, averaging 4.8 percent in The fiscal deficit had fallen from about 11.3 percent of GDP in 1998 to 5.2 percent of GDP in 2002, mostly as a consequence of expenditure cuts. The current account deficit declined from an average of 18.4 percent of GDP in to 3.2 percent of GDP in As a result of the fiscal and monetary tightening, inflation fell to 2.3 percent in Despite the improving economic and fiscal situation, the Kyrgyz Republic remained the second poorest country of the Former Soviet Union (FSU) republics and one of the poorest countries in the world. In 2001, per capita income was less than US$300. Severe poverty affected 13.5 percent of the population and 47.6 percent were considered poor with incomes below US$4.30 per day (PPP). Poverty was highest in rural areas, but there were large regional disparities. Access to public services such as running water, public sewerage, reliable electricity, district heating and telephone service was very low. The country also had a heavy burden of external debt, which stood at 477 percent of Government revenues in net present value terms at the end of Civil servants were poorly paid and often sought alternative sources of income. Macroeconomic Policy Framework: A 3-year arrangement under the Poverty Reduction and Growth Facility (PRGF), approved by the Executive Board of the IMF in December 2001, provided the basic underpinnings for the macro-policy framework. The 11

14 Government s reform efforts had also been supported through adjustment lending by the World Bank. Seven prior Bank adjustment operations provided support for wide-ranging and ambitious policy and institutional reforms in the following areas: privatization and enterprise reform; agricultural privatization; improvements in the business environment; financial sector reform; public sector resource management; pension reform; and power and gas sector reform. These operations helped the Kyrgyz Republic to orient its policy and institutional framework to the principles of a free market economy. Governance. The National Poverty Reduction Strategy (NPRS) recognized that weak governance and an ineffective state were serious hurdles to achieving development goals. At the level of central and local administration, policy making capacity was extremely weak, implementation of policies was poor and often non-existent, transparency and accountability were inadequate, and Government interference in business activities was excessive. The civil service also suffered from political interference. Its professional capacity and accountability for performance were low as real salaries declined and motivated staff became demoralized and left for other opportunities. Surveys of households and firms confirmed that corruption was a systemic problem in public agencies. Moreover, the judiciary and law enforcement agencies faced serious capacity and integrity challenges that reduced their ability to uphold the law. Political parties were weak, politics was personalized and characterized by populist measures to mobilize political support and rent-seeking behavior. NPRS noted that the legal framework suffered from lack of public discussion of proposed laws and regulations and inadequate public access to normative acts. For these reasons, the NPRS accorded a high priority to measures that strengthened governance and accountability. Public Financial Management. Economic and sector work in , such as the Public Expenditure Review (PER) and the Country Financial Accountability Assessment (CFAA), identified a moderate to high degree of fiduciary risk in the use of public resources. Critical weaknesses included the absence of effective cash planning, weak institutional arrangements for assessing the effectiveness of budget implementation, weak internal controls further compounded by the absence of an internal audit function, insufficient accountability arrangements for state-owned enterprises, weak external audit, and weak capacity in the legislative body to provide effective parliamentary oversight over the executive. The Government recognized the need to modernize its treasury operations and was strongly encouraged to do so by the IMF. The country already had the requisite legal and regulatory framework for treasury operations but all treasury functions were carried out manually. Government and its external partners believed that computerization would help to address shortcomings of the manual system related to transparency, informality, control of funds and inefficiency. Rationale for Bank Involvement. The reforms included in the Government s program were urgently needed to successfully implement the poverty reduction agenda. The Bank had been actively involved in supporting reforms to key public sector institutions through various operations, including the Public Sector Resource Management Adjustment Credit (PSRMAC), projects to reform the health sector and Institutional Development Fund (IDF) grants to strengthen public procurement. GTAC s primary objective was to provide 12

15 technical assistance to design and implement the policy and institutional reforms supported by the GSAC (see Table 1). Extensive consultations with the Government and the donor community helped to build consensus around the GSAC-supported policy actions and the technical assistance activities to be provided through the GTAC. The Bank was identified as the lead partner given its global experience in implementing public sector reforms, its country knowledge and strong relationships with counterparts and stakeholders. The Bank was only one of several partners supporting a complex, multi-sector reform program and the GTAC was only one of several Bank-financed projects supporting these reforms. However, the Bank was identified as the only donor with the expertise and resources needed to support the computerization of treasury operations. Table 1. Technical Assistance for the GSAC Source of Technical GSAC Objective Assistance I. Enhancing Transparency, Openness and Accountability of the Public Sector to External Stakeholders (i) Improving Personal Accountability of Political Officials and Civil WB-GTAC Servants (ii) Strengthen Voice and Participation by Improving Access to Official WB-GTAC Information (iii) Improving the Interaction between Regulatory Agencies and Business Enterprises (iv) Strengthening External Audit II. Improving Efficiency and Effectiveness within the Public Sector (i) Strengthening Public Expenditure Management (a) Establishing a more Strategic and Transparent Budget Formulation Process (b) Strengthening Accountability and Transparency in Budget Execution (ii) Improving the Internal Control Environment (iii) Improving Transparency, Value for Money and Accountability in Public Procurement (iv) Improving Service Delivery in the Health Sector (v) Enhancing the Effectiveness of the Civil Service (vi) Streamlining the Structure of Government Ministries and Agencies USAID; WB-FINBUS; WB-Health II Project DFID DFID IMF, WB-GTAC DFID WB-GTAC, WB-IDF WB-GTAC; WB-Health II Project WB-GTAC; EU-TACIS DFID; UNDP; EU-TACIS 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The objective of the GTAC was to strengthen the Borrower s institutional capacity to implement the measures under the Program outlined in the GSAC Program Document and to assist with the Treasury modernization. The original results indicators spanned several sectors and were designed support implementation of GSAC program measures: 13

16 Number of declarations of high state officials published annually Number, quality and timeliness of key pieces of official information, specified in the GSAC Policy Matrix, published annually Number of draft laws and regulations discussed with the private sector and civil society, prior to their adoption Number of public grievances addressed annually, by the President's Administration, the Office of the Government and the regulatory agencies covered by the GSAC Coverage, quality and timeliness of audit reports of Chamber of Control (COC) Public perception about the transparency, responsiveness and integrity of the public sector Increased usefulness of the budget as a strategic policy instrument, as a result of improvements in its formulation process, content and presentation Improved access, reliability and timeliness of information about budget execution, to public officials and civil society Improved financial accounting and compliance with laws and regulations in three ministries, due to strengthened internal control systems Percentage increase in the number of private firms participating in public procurement Improved allocation, predictability, availability of resources for health & education services Percentage of recruitments and promotions in the Civil Service made on a competitive basis. Only one of these indicators was related to the second element of the PDO to assist with Treasury modernization: improved access, reliability and timeliness of information about budget execution, to public officials and civil society. Text in the table at PAD Annex 1 (Project Design Summary) identified computerized Treasury system implemented as an output-level rather than an intermediate indicator. No indicators were formulated to track progress in the implementation of Treasury modernization or the impact of these reforms. The structure of the intermediate indicators bore little relation to the relative weight of the project components, focusing on the activities supporting GSAC implementation rather than the longer-term needs of Treasury modernization. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, Although the GTAC was extended several times, no revisions were made to the PDO. Relevant project results indicators were updated to reflect new closing dates and two results indicators were dropped, because the Borrower (during implementation) gained access to grant funds from other development partners to implement the corresponding activities. No key indicators were formulated to track progress in the implementation of Treasury modernization or the impact of these reforms despite repeated project restructuring, even when the Treasury modernization became the only active project component. 14

17 1.4 Main Beneficiaries The initial project design anticipated that benefits of the project would accrue to: The people of the Kyrgyz Republic through more effective and transparent use of public sector resources and improved access to information and grievance mechanisms, both critical to the delivery of public services key to addressing poverty reduction and other developmental goals; The private sector, as a result of improvements in regulatory policies and practices and public procurement; Civil servants, on account of improvements in applicable policies, processes and actual practices; and The political executive, due to the availability of a more effective state machinery to assist in policy-making and implementation. Poverty would be reduced indirectly through improved public services resulting from better public financial management and improved business confidence resulting in increased investment and faster economic and employment growth. 1.5 Original Components GTAC was intended to support a wide range of reforms, including but not limited to those supported by GSAC program. GTAC had three components: Component 1: Technical assistance to support the design and implementation of the reforms included in the GSAC (US$1.98 million). GTAC provided TA for key reforms supported by the GSAC, complementing grant assistance provided by other donors. GTAC assistance covered a five year implementation as compared to the two years for the GSAC in order to enable institutionalization and sustained implementation of difficult public sector reforms. The key reforms to be supported by GTAC were: strengthening voice and participation by the private sector and civil society in public decision-making; increasing transparency in public procurement; improving service delivery in education, social protection and health services; strengthening the professionalism and effectiveness of the civil service by increasing its independence and merit-based personnel actions; streamlining the structure of three ministries via functional reviews; refining the NPRS and implementing a monitoring and evaluation (M&E) system to measure progress in achieving NPRS objectives. Component 2: Modernization of the treasury (US$6.37 million). This component was to finance consulting services, goods and training for the modernization of treasury operations through the implementation of a Treasury Management Information System (TMIS), restructuring for Treasury General Ledger (TGL), and implementation of a new Chart of Accounts (CoA) in full compliance with the new GFSM2001. It was also to provide assistance in (a) preparation of user requirements, conceptual design of the system, system architecture and software and hardware specifications for the TMIS; (b) selection, procurement, adaptation and implementation of an off-the-shelf software 15

18 package; (c) procurement and implementation of hardware and communication equipment; (d) development of the regulations for the implementation of the system; and (e) training of staff. It was envisaged that the IMF would be closely involved in the implementation of the TMIS and would also provide technical assistance. The system would first be piloted in the Central Treasury and two or three pilot oblasts before being replicated to all regional treasury offices. Component 3: Project management (US$ 0.66 million). This component was intended to finance consulting services, equipment and training for the Project Management Group and Project Implementation Unit. It was also intended to finance the communication of project objectives and activities to stakeholders, stakeholder surveys and project audits. GTAC combined two distinct sets of activities: public sector and institutional reforms supported through GSAC; and a treasury modernization project comprising ICT hardware and software, associated regulatory reforms and capacity building. The modest credit (SDR5.80 million) could only finance an initial set of activities within the vast public sector institutional reform agenda of the time, but it was considered prudent to proceed with a project of this size initially so that the Government could draw on the experience gained from implementing this TA project in addressing its wider TA needs for supporting public sector reform. 1.6 Revised Components There were adjustments in the structure of Component 1 during implementation. Two subcomponents were dropped in their entirety because the Government was able to access grant financing from other donors: sub-components 6, streamlining the structure of Government ministries and agencies; and sub-component 7, refining NPRS and monitor and evaluate progress. The two intermediate indicators applicable to these subcomponents were also dropped. In addition, specific activities were dropped from the subcomponents that remained under Component 1: Strengthened Voice and Participation: regulatory impact assessment of proposed laws and regulations; support to public grievances by the President's Administration, the Office of the Government and the regulatory agencies covered by the GSAC. Improving Service Delivery in Health and Education: improvement of inspection regimes of Sanitary and Epidemiology Service and Pharm Control Laboratory PCL; funding for health insurance for pensioners and the unemployed accordance with budget allocations. Improving the Internal Control Environment: improvement in financial accounting and compliance with laws and regulations in three ministries through strengthened internal control systems. The specific reasons for these adjustments in the scope of the project are not clear from the project record: some activities were addressed through complementary projects, others were dropped because priorities changed. 16

19 1.7 Other significant changes There were four extensions of the project closing date with the project finally closing without concluding project activities on December 31, 2013 five years beyond the original closing date of December 31, The final project restructuring completed on December 18, 2013 cancelled the remaining US$2.5 million of the credit. There were no significant changes in implementation arrangements. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Project preparation commenced in December Support for preparation and capacitybuilding was also partly financed with a Policy and Human Resource Development (PHRD) Grant (TF026448) for US$729,250 from the Government of Japan which was formally associated with the preparation of the GSAC program but covered preparation of several GTAC activities. These included technical assistance for: an Anticorruption Strategy (US$56,350); Public Administration Reform (US$460,600) including the preparation of an administrative and civil service reform program; a draft law on public service; an assessment of budgetary employment and remuneration; and an analysis of the institutional environment within which three to four key ministries or government institutions function; and Public Expenditure Management (US$48,550) including development of proposals for reforming budget management in the health and education sectors. GTAC was grounded in diagnostic work, including but not limited to the analysis financed by the PHRD Grant, which had supported the development of the GSAC program. There was strong national ownership of the institutional reform program, an extensive consultation process with the government and with society more broadly. The context appeared very conducive for reform. Against this background, and although the size of the credit was small, the project scope and activities were thought to be achievable. Indeed, the project scope and objective were also reviewed by a Quality At Entry (QEA) exercise which rated the project under preparation as Highly Satisfactory. However, in hindsight, the Treasury modernization component of the project was inadequately prepared and the MOF far from ready for implementation. The design of the Treasury modernization component was similar to other Bank-financed treasury modernization projects prepared in the early 2000s. A detailed review of treasury operations had been completed by a joint IMF-WB team during GTAC appraisal. The review concluded that Government was now ready for a successful transition, within a period of about three years, to a fully functional modernized treasury that employed contemporary information technology. However, this timeline proved to be unrealistic. Preparatory work on Treasury modernization, in particular the detailed specifications for 17

20 the TMIS, the business process re-engineering requirements and implementation of related institutional reforms, only began after project became effective. 2.2 Risks Assessment and Mitigation Most of the significant risks were correctly identified and recognized during preparation, and risk mitigation measures put in place, although these risk mitigation measures were not always effective. However, the risk of major political upheaval was one not identified at appraisal: indeed, such a risk in the case of the Kyrgyz Republic in 2003 would have been considered a black swan event and inconceivable at appraisal. The risks identified at appraisal, and what actually happened, are summarized in Table 2 below. The PAD failed to identify risks specifically related to the implementation of Treasury modernization. Three risks stand out, all of which subsequently materialized: first, the lack of technical and management capacity to design and manage a complex TMIS contract and the business reengineering requirements of Treasury modernization; second, internal resistance to reform from those with vested interests in the existing system and the flexibility and informality that it facilitated; and third, differences in perspectives regarding systems requirements across institutions and within the Ministry of Finance. GTAC s failure to manage these risks contributed to the failure of the project s Treasury modernization component. Risk Political commitment to implement the NPRS and improve governance and public sector performance is weakened Demand for good governance and improved public sector performance amongst the private sector, civil society and the public is not strong Civil society, the private sector and the public do not contribute to policy debates, or make optimal use of publicly disseminated official information to enforce accountability in the public sector Government commitment to implement reforms included in the GSAC is not strong. The Government is unable to overcome opposition to reforms from vested interests Table 2. Risks Identified at Appraisal and Outcome Rating at Outcome appraisal H Despite the voice, accountability and transparency gains from the implementation of GTAC, there was some backtracking on politically sensitive reforms such as the depoliticization of the civil service. S Strong demand for good governance was sustained. Weak governance, high-level and administrative corruption and an ineffective public sector were underlying factors behind the 2005 and 2010 revolutions. H Some elements of the publicly disseminated official information (e.g. income and asset declarations, budget information) have proven to be useful instruments to strengthen accountability. S Government commitment to implement GSAC reforms was strong at the outset but diminished: the GSAC ICR rated GSAC performance as MU. H This was true in the periods before the two revolutions, when vested interests stalled or pushed reforms. Government lacks the capacity to H Weak institutional capacity has been an issue, 18

21 Risk implement the reforms effectively There is a lack of coordination of reform efforts between the President s Administration, Office of the Government, MOF and concerned line agencies Relevant managers and staff of the President s Administration, Office of the Government, MOF and concerned line agencies are not actively involved in the design, development, quality control and implementation of project initiatives Donor coordination in parallel financed project activities or in implementing independently executed activities is weak Adequate allocation of funds for the project is not made in the national budget and availability of counterpart funds is inadequate Overall risk rating Rating at Outcome appraisal exacerbated by losses of trained civil servants across ministries (including the MOF) both before and after each of the revolutions. S The turnover of high-level officials led to a non-functional GSAC/GTAC Coordination Committee. Although this intensified coordination challenges, project decisions continued to be taken, to the extent possible, by Technical Committees. S Engagement of officials in design and implementation of project activities varied across agencies and was not always constructive. Vested interests opposed to greater transparency, control and accountability in the management of public finances have resisted the Treasury modernization. S In practice, donor coordination remained fairly strong even in the challenging environment. H H This was not a problem. Indeed, the opposite was the case: government spent its own funds to out in place the ICT backbone and infrastructure for the TMIS while the World Bank refused to agree to additional financing at a critical time in 2009 when this was needed to meet the unforeseen increase in the estimated cost of the TMIS/HRMIS. Risk rating H (high risk), S (substantial risk), M (modest risk), N (negligible or low risk) Source: GTAC PAD 2.2 Implementation By the time of the Mid-Term Review (MTR) in March 2008 the majority of Component 1 activities had been completed, with only a small allocation left for work on strengthening capacity in the civil service reform. The MTR assessment at that time was that the reforms envisaged by GSAC and supported by GTAC had progressed at a varying pace during different periods due to both political developments and the complex nature of the institutional reforms. Nonetheless, the MTR considered the reforms as largely successful and prospect of sustainability for these reforms to be high. Implementation of the Treasury Modernization component proved to be problematic. Procurement issues and the political events of 2005 led to delays in the first three years of the project. As a result, GTAC had only disbursed 23 percent of the credit proceeds at MTR. A project extension was needed to launch procurement of the TMIS/HRMIS software (ICB-1) and hardware (ICB-2), which only started in Although the installation of the ICT hardware (ICB-2) was successfully completed, subsequent implementation delays relating to the implementation of the TMIS/HRMIS software 19

22 (ICB-1) led to three additional extensions of the closing date, more than doubling the original implementation timeline. In the end, this component could not be satisfactorily completed. The factors that affected implementation of the Treasury modernization component are outlined below. Delays in the preparation of technical specifications for TMIS and HRMIS. The decision to submit the GTAC for Board approval alongside the GSAC operation meant that the GSAC preparation schedule was driven by the country s urgent budget support needs. Preparation and the implementation readiness of GTAC activities were a secondary consideration. Consequently, preparatory work on the Treasury modernization component was only undertaken after the project became effective in August 2003 when a consultant firm was contracted to define the functional requirements and prepare the technical specifications for the TMIS and HRMIS. Although a bidding document for this tender was issued to a shortlist of three qualified companies in February 2004, the evaluation committee became deadlocked and opted for an external evaluation of the proposals. An external expert was selected in December 2004 to evaluate the proposals, and an international firm selected. The TMIS and HRMIS design reports were finally completed at the end of 2006, more than three years after effectiveness. These delays had significant implications for procurement, contracting and contract implementation. In the absence of detailed specifications, there was no proper basis for estimation of the cost of the software and related hardware at the time of Project appraisal, and these costs turned out to have been underestimated. The specifications, scope and business re-engineering requirements of the Treasury modernization were not thoroughly reviewed during project appraisal and agreed during negotiation of GTAC financing. Lack of agreement on these details contributed to delays during implementation and ultimately to the failure of the Treasury modernization component. Deficiencies in the TMIS tender process. The TMIS as described in the Treasury modernization component of the PAD did not envisage modules for human resource management, budget preparation, debt servicing, and agency accounting on an accrual basis. However, these modules were included in the technical requirements issued to bidders. This added to the system cost and complexity. When the first international competitive bidding (ICB-1) process for the TMIS and HRMIS application software and related servers was evaluated by the Government s Evaluation Committee, the lowest evaluated bid was found to be nearly US$2 million above the budget allocation. The Evaluation Committee decided to rebid, hoping that the participation of more companies would lower the bid price. When the tender was relaunched, specifications were not updated to incorporate advances in information technology, changes in the country s pay system, the preferred structure and content of the single CoA, or the MOF s approach to adopting international accounting and reporting standards. Some of the TMIS functional requirements were not clearly defined, including terms such as preparation of a draft national budget and preparation of a medium term financial plan. The expectations of MOF s Treasury and Budget departments differed on the extent to which budget preparation functionality would be included in TMIS. The vendor agreed to produce a customized module, which it demonstrated to the Government in mid-2012, but the MOF was unwilling to accept this module. The MOF and the vendor had very different 20

23 expectations regarding the scope of system customization required to satisfy the requirements of the contract signed in These differences in expectations regarding the scope of TMIS functionality and system customization were compounded by delays on the part of the authorities in finalizing two system design elements: the CoA and the interface between the Treasury Single Account (TSA) and the banking system. Delays in the approval of the Chart of Accounts. A unified CoA was expected to provide the backbone for classification, accounting, transaction processing, reporting and control within the TMIS. However, the development of the CoA took much longer than anticipated. The revisions to the budget classification and CoA structures only started in A unified CoA was expected to be introduced in 2009 and shared immediately with the winning bidder of the ICB-1 second stage. In reality only a partial version, relating to Treasury, was ready at this time. The selected vendor stated that it could provide a unified CoA as part of its software package. After a few months, however, it became clear that the vendor s standard CoA did not meet the needs of Government operations. After a year of negotiations, the vendor eventually agreed that a new unified CoA was needed. The CoA was finally completed in September 2012, nearly two years later than scheduled. Lack of clarity with regard to the design of the Treasury Single Account and NBKR interface. In 2007, before launching bidding on TMIS/HRMIS, the MOF signed an agreement with the National Bank of Kyrgyzstan (NBKR) on managing gross payments in online mode and establishing direct access for the Treasury to the central bulk clearance arrangements. The NBKR did not have the capacity to perform transactional banking tasks for the Government and expected that the existing practice of using commercial banks, or establishing access to the interbank payments system for Treasury, would be adopted for payment processing, detailed settlements, and transfers between the Government s accounts. The NBKR maintained a simpler structure for Government accounts and had revised its CoA prior to the 2007 agreement. The NBKR was not consulted about the technical specifications for the TMIS contract. These specifications included a general provision for the design of interfaces with the NBKR and agent (commercial) banks for the transfer of information on accounts, payments, and other matters, and for the receipt of payments data to commercial banks. It also included an outline of the proposed operations of a TSA system that the Treasury expected to be open with the NBKR. After the TMIS contract was signed, disagreements arose on how the TSA should function. The Treasury and the vendor were unable to resolve the details of the TSA arrangements due to the difficulty in defining the mode of TSA operations and the lack of legal/operational framework (such as a TSA agreement or protocol) between the Treasury and the NBKR. In 2012, the vendor agreed to develop an NBKR interface to work within the bulk clearance system. Ultimately, the vendor was not able to produce this software before the contract was terminated. Delays in contract implementation. In January 2012, the Bank was informed of the Government s intention to terminate the contract with the vendor due to the continuous delays in contract implementation. At this stage, the contract was supposed to close by the end of December Following discussions with the vendor, the Government 21

24 agreed to continue with the contract. The MOF and the vendor signed an MOU on June 8, 2012 setting out explicit deliverables and timelines. However, progress in TMIS/HRMIS implementation continued to be unsatisfactory as both the MOF and vendor missed key deadlines, and the parties were not able to agree on detailed system specifications for TMIS/HRMIS software, testing and operational acceptance. In early 2013, a new action plan for full implementation was agreed between the parties with the vendor agreeing to demonstrate a fully functional module by December Delays and communication difficulties persisted, however, and the Government decided to terminate the contract with the vendor on November 27, 2013 citing the vendor s failure to complete TMIS/HRMIS implementation activities within the agreed timeline. The Government also notified the Bank on December 16, 2013 that it had elected to cancel the undisbursed amount of the credit (approximately US$2.5 million). The MoF and the vendor are expected to be in discussion over the issues in contention. 2.3 Monitoring and Evaluation Design, Implementation and Utilization The GTAC results indicators and results framework reflected the practice in the Bank in As a TA project supporting the achievement of key policy and institutional reforms under an adjustment credit, the results framework was aligned with the GSAC policy matrix. The results framework was reviewed during the Quality At Entry (QEA) exercise and considered as representing best practice. In retrospect this assessment seems unwarranted: the monitoring and evaluation framework was weak in design and implementation. The use of public perceptions of transparency, responsiveness and integrity of the public sector as the PDO indicator for Component 1 presented problems of attribution since a wide range of factors outside the project would have impacts on those perceptions. Many of the GTAC intermediate outcome indicators measured processes rather than the outputs or outcomes of reforms. Many of the indicators lacked baselines, others lacked target values. In constructing the datasheet, the ICR team has presumed target values based on assumptions of the kinds of outcomes that would be associated with the indicator. Attribution problems also arise because of the design of GSAC and GTAC as parallel operations. Component 1 indicators were monitored by joint GSAC and GTAC teams, focusing on the Government s progress in achieving GSAC triggers. Close alignment of the GSAC and GTAC results frameworks made it difficult to distinguish which results can be attributed to each of the financing instruments. A significant shortcoming of the M&E original design was the lack of an appropriate monitoring framework for Treasury modernization, the largest of the two components in terms of value. This component featured as a PDO indicator ( Establishment of a modern computerized treasury system ). However, the intermediate indicators were formulated in such a way as to assess the performance of the Treasury and Budget systems as a whole rather than specific components ( Improved access, reliability and timeliness of information about budget execution, to public officials and civil society ; Increased usefulness of the budget as a strategic policy instrument, as a result of improvements in its formulation process, content and presentation ). The intermediate indicators provided 22