Country Practice Area(Lead) Additional Financing Finance, Competitiveness and

Transcription

1 Public Disclosure Authorized Independent Evaluation Group (IEG) 1. Project Data Report Number : ICRR Public Disclosure Authorized Public Disclosure Authorized Project ID P Project Name XK Fin. Sect. Strengthen.&Market Infrast Country Practice Area(Lead) Additional Financing Finance, Competitiveness and P Kosovo Innovation L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-50060,IDA-H Jun ,000, Bank Approval Date 13-Dec-2007 Closing Date (Actual) 31-Dec-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 2,000, Revised Commitment 8,813, Actual 8,288, Prepared by Reviewed by ICR Review Coordinator Group Paul Holden Fernando Manibog Christopher David Nelson IEGFP (Unit 3) Public Disclosure Authorized 2. Project Objectives and Components a. Objectives Original Project Development Objective The original project development objective (PDO) was "to enhance the stability and development of Kosovo s financial system by supporting stronger prudential regulation and supervision by CBK for banks and non-bank financial institutions, supporting the institutional strengthening of CBK, and strengthening the banking and microfinance industry." (Financing Agreement Between and United Nations Interim Administration Mission in Kosovo, dated December 14, 2007, page 4) The PDO in the original Project Page 1 of 16

2 PHEVALUNDERTAKENLBL Independent Evaluation Group (IEG) Appraisal Document and the original Financing Agreement was identical. CBAK is the Central Bank Authority of Kosovo, the name of the central banking authority when project appraisal commenced in Following Kosovo's declaration of independence in 2008, the CBAK became the Central Bank of Kosovo (CBK). Revised Project Development Objective (after the 2011 Restructuring and Additional Financing) A major restructuring occurred in It was based on the recognition that although the original project had achieved important results, additional priorities and the need to further strengthen the financial sector remained. Kosovo had become independent, joined the World Bank Group and the IMF, and needed additional basic financial infrastructure. The revised PDO was "to (i) enhance the stability and development of the financial sector; and (ii) strengthen the financial sector s underlying market infrastructure." The PDO in the revised Project Appraisal Document and the revised Financing Agreement was identical. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 14-Jun-2011 c. Will a split evaluation be undertaken? No d. Components Original components: (Appraisal estimate US$2,000,000; actual US$735,033) The original project had 3 components, numbered in the original Financing Agreement (Page 4) as Part A, Part B, and Part C. Component 1 (Part A) : Assistance to the Central Bank Authority of Kosovo (CBAK): (Appraised estimate US$1,540,000; actual US$339,496) This component had two main subcomponents. 1. To strengthen CBAK's long-term institutional and financial capacity and sustainability, by preparing (a) a medium-term development strategy for the CBAK and a detailed assessment of the capacity and functions of the institution, and (ii) developing a phased medium-term staffing plan. In addition, the CBAK's salary Page 2 of 16

3 scales would be benchmarked to private sector financial institutions; a strategy would be developed to ensure CBAKs long-term financial sustainability; and a financial literacy campaign would be funded to educate the population on such issues as availability of financial products, concept of risk and returns, and the importance of a credit reporting system. 2. To strengthen, through technical assistance and training, banking sector supervision as well as the supervisory framework for the insurance industry and other non-bank financial intermediaries that fell under the authority of the CBAK. Component 2 (Part B): Provide assistance to the Association of Microfinance Institutions of Kosovo (AMIK): (Appraisal estimate, US$240,000; actual US$220,986) Provide assistance to the Association of Microfinance Institutions of Kosovo (AMIK): This component aimed to provide training to microfinance institutions (MFIs) to promote self-sustainability, improve risk management, improve capabilities for deposit taking, and promote the ability of the industry to analyze their impact and advocate for their importance. Component 3 (Part C): Provide assistance to the Kosovo Bankers Association (KBA): (Appraisal estimate US$220,000; actual US$179,551) This component was directed at improving the ability of the KBA trainers and bank staff to provide training on key issues such as risk management, anti-money laundering and how to conduct training needs assessments. Revised Components As part of the restructuring, Component I was increased. Over the original total funding of U$1,670,000, the restructuring provided US$130,000 additional financing. (Actual $1,353,537) Additional financing support to Central Bank of Kosovo (CBK) was added to enhance the implementation of CBK s training, as identified in the CBK s training needs analysis. Three new components were added: Component 4: Establishing a Real Time Gross Settlement System (RTGS) (Appraisal estimate, US$1,870,000; actual US$1,643,580) In 2009, the CBK had approved a strategy document and action plan to implement a national payment system including the establishment of a Real Time Gross Settlement system. This component was to enable the implementation of the action plan that would provide for the safe and efficient settlement of all systemically important payments. Component 5: Establishing a Business Continuity Center (BCC) that would support key operational functions within CBK in the event of a disaster; (Appraisal estimate, US$850,000; actual US$870,131) Page 3 of 16

4 This component was to enable the establishment of a Business Continuity Center and a Business Continuity Management Team that had been identified as a priority in World Bank technical assistance. It was also in line with the Bank for International Settlements' Core Principle 7. The technical assistance recommended the establishment of the BCC at a site outside the capital center. The appraised amount included funding the renovation of an identified building site. Component 6: Providing seed funding to the Deposit Insurance Fund of Kosovo (DIFK). (Appraisal estimate, US$4,000,000; actual US$US$3,425,277) This component provided for seed funding for the DIFK in conjunction with additional funds that were to be provided by the Government of Kosovo, premiums from financial institutions whose deposits were guaranteed, and additional grant funding of US$4.5 million from KfW. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The original project (2007) was estimated to cost US$2 million to be financed by an International Development Association (IDA) grant. The restructuring in 2011 provided an additional US$6.85 million financed by an IDA grant. KfW also contributed US$4.5 million to the seed funding of the DIFK. The restructuring agreement provided an additional US$130,000 for revised Component 1 (Part A) even though only 22% of the appraised amount had been disbursed. In any event, the total amount spent on Component I was less than the original appraised amount. Borrower Contribution: the Borrower did not contribute. Dates: The closing date of the original project, which had been set to close on 6/30/2010 was extended to 6/30/2011. The project was restructured on 6/14/2011. The closing date of the restructured project, originally scheduled for 6/30/2014 was extended to 6/30/2016; this was further extended to 12/31/ Relevance of Objectives & Design a. Relevance of Objectives Both the original and the restructured PDO were consistent with the Transitional Support Strategy for Kosovo (2004) and the Kosovo Interim Strategy Notes for FY2010 and FY2011, respectively. The original PDO, namely, to strengthen and stabilize Kosovo's financial sector, were clearly important and of high priority. Access to finance was limited with a relatively low ratio of private sector credit to GDP. Trust in the banking system needed to be increased. All aspects of the regulatory structure required strengthening. The CBAK had only limited experience in financial supervision, providing urgency in upgrading its capacity in Page 4 of 16

5 this area. Addressing these weaknesses and limitations was an important first step in establishing a foundation for long-term growth. The objectives of the original project were highly relevant. The revised PDO of the restructured project was fully consistent with the over-arching goal of the original project, while covering more financial sector institutions and diversified instruments. The restructured project provided for additional financing to further enhance the capacity of the CBK to supervise the financial system, to modernize the payment system, to provide for the ongoing functioning of the financial system in the event of disaster, and to establish a framework for deposit insurance. Rating High Revised Rating High b. Relevance of Design The ICR (page 23) notes that the original project design correctly placed emphasis on institutional strengthening, stability and development of the financial sector, capacity building and strengthening market infrastructure. It further notes that the project contributed to strengthening Kosovo's financial system and has improved its stability. The original project was designed to improve CBAK's institutional strength and ability to supervise both banks and non-bank financial institutions. The original design was expected to have a positive impact in terms of building trust in the financial system, by also strengthening the capacity of the bankers' association and microfinance institutions, i.e., KBA and the AMIK, respectively. A weak feature of the design was the failure to foresee the need for a dedicated project management unit, especially considering the weak capacity that existed in Kosovo at the time of project appraisal. The revised design of the restructured project sought to further strengthen the trust in the financial system, reduce transactions costs in the clearing of payments, and fully operationalize the deposit insurance scheme -- in addition to further strengthening the capacity of CBK (formerly the CBAK) to supervise banks and non-bank financial institutions. A negative feature of the revised project design, however, was the inclusion of a separate component to refurbish the Business Continuity Center. In many ways, this component was no more than supplementary to the higher-level goal of strengthening CBK, because it involved dealing vis-à-vis the local authorities with the legal and regulatory ramifications of renovating a building. It led to delays in the project closing. Although in the end, the component was completed, it would have been preferable to have left this in the hands of CBK. Moreover, the restructuring should have included a dedicated project management unit. It should have been apparent that a dedicated PMU was needed by the time of the restructuring, as evidenced by the fact that only 22% of the funds for component A had been disbursed. The restructuring offered an ideal time for the introduction of a dedicated PMU. IEG therefore revises the rating of design to Modest. Rating Revised Rating Page 5 of 16

6 Substantial Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective The original PDO objective was "to enhance the stability and development of Kosovo s financial system by supporting stronger prudential regulation and supervision by CBAK for banks and non-bank financial institutions, supporting the institutional strengthening of CBAK, and strengthening the banking and microfinance industry through capacity building." Rationale CBK The ICR does not discuss Efficacy directly. The ICR (page 33) states that the project outcome indicators were the output targets, which suggests that outputs and outcomes were the same and the achievement of outcomes through outputs was therefore tautological. Outputs The project's outputs were the following: Preparation of a development strategy for CBAK - Completed. Assistance to CBAK for development of a medium-term staffing and incentive plan - Completed. Assistance to CBAK in developing a new salary structure - Completed Feasibility study for establishing a training institute - Completed. Drafting of laws and regulations governing financial market supervision of the banking sector, the insurance sector and pension fund management. Four new laws and 70 by-laws were drafted as part of the restructured project by 6/30/ Completed Designing and implementing a regulatory reporting database. This was implemented but only by 6/30/ Completed Training of bank supervisory staff. This was implemented by 6/30/2016 as part of the restructured project. - Completed Regarding the strengthening of the capacity of the Association of Microfinance Institutions of Kosovo (AMIK) to provide training, to advocate for the importance of the industry, and enhance the capacity of AMIK staff: Six packages of training sessions involving 100 staff and 17 potential trainers were delivered by 6/30/ Completed Nine staff participated in study tours/workshops/conferences by 6/30/ Completed Regarding the institutional strengthening of the Kosovo Bankers Association (KBA), the output was Page 6 of 16

7 the training for 223 staff provided by the Bled School of Business or individual trainers, which was completed by 6/30/2011. Outcomes The key project outcome were the following: Development and implementation of supervision plans for CBK for each of Kosovo's main banks, which was completed by 6/30/2016. Development and implementation by CBK of an ongoing supervision plans for insurance companies and pension funds, which was completed by 6/30/2016. For AMIK, the outcomes were the same as the outputs. No evidence was provided that these outputs directly contributed to the improvement in the functioning of the Kosovo financial sector. For KBA, the outcomes were also the same as the outputs, with no further evidence provided on the direct contribution of those outputs to the achievement of the over-arching PDO. Overall assessment: The project supported and enabled the preparation of new laws on payment systems, pensions, insurance supervision, and on banks and micro-finance institutions. In addition, more than 70 regulations pertaining to the new legislation were put in place. As a result, the project played a central role in substantially improving the legal framework supporting the CBK in its supervision of the financial sector. In addition, the staff of the CBK received extensive training and the CBK installed off-site supervision software on which all staff were trained. The results of the new legislation, regulations, and training was an upgrading of the capacity of the CBK to supervise both bank and non-bank financial intermediaries. Training was provided to the AMIK to prepare the industry for accepting deposits, to enhance the capacity of the staff and to participate in study tours and to advocate for the importance of the industry to the Kosovo financial sector. To support the institutional strengthening of the Kosovo Bankers Association, training was deemed to be necessary. On this basis, the achievement of the original PDO is rated as substantial. Rating Substantial PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective Page 7 of 16

8 During the 2011 restructuring the objectives of the project were broadened to: (i) Enhance the stability and development of the financial sector; and (ii) Strengthen the financial sector's underlying market infrastructure. The rationale remained unchanged. Revised Rationale The rationale continued to be providing assistance to the CBK, in the same form as the rationale for the original project described above. The two original performance indicators were merged into one indicator. In addition, the restructured project sought to establish a Real Time Gross Settlement Payment System (RTGS), to be controlled and maintained by CBK, aimed at minimizing settlement risks and increasing confidence and reliability of commercial banks. It included an automated transfer system incorporating both RTGS and an Automated Clearing House. This output was successfully launched in July 2016 and constituted a substantial upgrade of payment systems in Kosovo that directly reduces operating costs for banks, increases the speed of clearing, minimizes payment risks and promotes confidence in the banking sector. As the main outcome, this upgrade resulted in shortening the execution times for high-value payments, minimizing the payment risk, and increasing the confidence in the banking sector. As of December 31, 2016, 100% of transaction were settled through the system in less than 15 minutes. The restructured project also sought the establishment of a fully operational Business Continuity Center (BCC). The BCC involved establishing standby site containing continuously updated real-time data from all the CBK's critical systems. The main output is the opening of the BCC. Although there were significant delays involved in the refurbishment of the building in which the BCC operates, the Center was officially opened in November The main outcome is a substantial increase in security and stability of the financial system. All transactions are recorded on the backup system. The restructured project also financed the Recipient's initial contribution to the DIFK, thus enabling the DIFK to commence operation of the Deposit Insurance Scheme. The main output is the IDA disbursement of US$4 million to provide seed funding for the DIFK, which occurred in December The key outcome is the commencement of the operation of the DIFK. The World Bank assessment in March 2014 concluded that: (i) the governance arrangements of the DIFK are satisfactory; (ii) DIFK is operating in line with the financial management procedures and standards that were found to be acceptable; (iii) DIFK and the Deposit Insurance Scheme's operations are consistent with the key elements of the law; (iv) DIFK has sufficient resources for its continued operation; and (v) the DIFK's audit reports are unqualified and are in line with the financing agreement. On the basis of this evidence, the achievement of the PDO of the restructured project is rated substantial. Revised Rating Page 8 of 16

9 Substantial PHREVISEDTBL 5. Efficiency No cost benefit analysis was undertaken at appraisal nor at restructuring. Rather the expected benefits of the project focused on improvements in the institutions underlying the financial system and on capacity building in the activities that need strengthening, in particular, the supervisory capacity of the CBAK, the clearing payments system, and the redundancy aspects of the BCC. Two issues affected the efficiency of the project negatively. Firstly, limited capacity and the lack of familiarity with World Bank procurement procedures resulted in significant delays in the early stages of the project. This was only one of several projects that the Project Management Team (PMT) were managing, which compounded issues related to procurement procedures and project efficiency. The ICR (page 31) notes that procurement training occurred in a piecemeal fashion that also contributed to delays. An additional factor was the restructuring of the project, which extended its scope through the introduction of three new components. The refurbishment of the BCC and the need to comply with extensive local regulatory requirements that this involved added to the delays. The procurement complexities in purchasing software for the establishment of the RTGS and for off-site supervision added to the time needed to complete the project. The need to retender the contract for the off-site supervision software for the CBK and concerns regarding security and conflicts of interest for the RTGS component were also critical factors in delaying the completion of the project (ICR, page 31). It became clear that the World Bank should have insisted on a fully staffed Project Implementation Unit dedicated solely to this project. Nevertheless, the energetic commitment by the World Bank task team and the CBK led to the resolution of these difficulties. However, it took nine years for the project to be completed in total, compared with an original expected timeframe of five and a half years. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) Appraisal 0 0 Not Applicable Page 9 of 16

10 ICR Estimate 0 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The original project objective was to enhance the stability and development of Kosovo's financial system by supporting stronger prudential regulation and supervision by CBAK for banks and non-bank financial institutions, supporting the institutional strengthening of CBAK, and strengthening the banking and microfinance industry through capacity building. This was highly relevant as there was only limited intermediation by the financial sector, lending was limited and there was distrust of the banks by the population at large, and domestic capacity to improve the functioning of the financial system was weak. Of the 3 original components, the second and third (providing assistance to the Association of Microfinance Institutions of Kosovo and providing assistance to the Kosovo Banking Association) were achieved. Substantial progress was made on the first component, namely preparing a detailed medium-term development strategy, capacity assessment and staffing plan for the CBAK, and strengthening the CBAK's banking sector and non-bank sector supervisory capacity. In the restructuring, component 1 was continued and provided with extra funding and 3 additional components were added, namely: to install a Real Time Gross Settlements System; to establish a Business Continuity Center; and to assist with and provide seed funding to a Deposit Insurance Fund. Despite significant delays, partly caused by the requirements surrounding the refurbishment of the building to house the BCC, all objectives were achieved, albeit over a 9-year period. However, part of the extended time required to complete the project arose from the 2011 restructuring, which expanded the scope of the project significantly. However, there were extensive delays and many of the objectives of the project were only completed in the last year of the extended closing date. Based on successful but partial achievement of the original and revised PDO, and the long time delays incurred, overall project outcome is rated moderately satisfactory. a. Outcome Rating Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating The main targeted results were successfully achieved and there is no additional risk associated with the implementation of the project. The ICR (page 33) notes the substantial political and economic uncertainties associated with Kosovo's post-conflict situation. When the project was originally designed, Kosovo was under Page 10 of 16

11 the administration of a United Nations Interim Administration Mission which implies significant uncertainties surrounding the future evolution of the country. By the time the restructured financing agreement was signed, Kosovo had declared independence and was a member of the World Bank Group and the IMF, which suggests a decline in political risk through time, although its unresolved status in relation to its neighbors remains a barrier to the country's political and economic development. The ICR (page 34) notes that the use of the Euro as Kosovo's currency reduces policy flexibility and the ability of the CBK to undertake monetary policy, although it also limits exchange-rate risk. While the deposit insurance system has been established and is functioning, it has not yet accumulated sufficient funds to deal with a major bank failure; however, the current moderately sound status of the banking system suggests that as time passes the accumulation of levies will increasingly allow it to do so. a. Risk to Development Outcome Rating Modest 8. Assessment of Bank Performance a. Quality-at-Entry The ICR (page 34) notes that the World Bank team identified correctly both when the initial project was being designed and when it was restructured, the need for institutional strengthening, developing the financial sector, building up capacity and strengthening the overall institutions supporting the sector. Both the original project and the restructured project were aligned with the World Bank Interim Strategy Note and the Kosovo Country Partnership Strategy. Negative features of the design at entry were the decision to involve the project with the refurbishment of the BCC and the failure to include a dedicated PMU to support the project, even after experiencing delays before the restructuring. The restructuring presented an ideal opportunity to rectify the omission. Overall, IEG rates Bank quality at entry as moderately satisfactory. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The ICR (page 35) notes that the Bank actively supervised project implementation. There were 2 ISR missions per year and there was active communication between the World Bank, the project management team, and the implementing agencies. A significant amount of technical assistance was provided by technical staff from the World Bank and from consultants to ensure the quality of the technical aspects of the various project components. The World Bank team was proactive in expanding the project through Page 11 of 16

12 restructuring when it became apparent that additional assistance was needed to improve financial sector development and to enhance the stability of the financial system. Over the life of the project there were 4 task team leaders but this did not affect communications and cooperation with the borrower, which was judged to be satisfactory through the whole of the project life. Project supervision costs were high, amounting to more than 10% of the total financing costs of the project and exceeded budgeted supervision expenses by a significant amount. There were several project implementation challenges associated with procurement issues because the Kosovo authorities did not realize that World Bank procurement rules superseded country procurement rules. The ICR (page 36) suggests that expenses could have been reduced if closer supervision had occurred. An alternative view is that if some of the problems associated with limited capacity and lack of knowledge of World Bank procedures on the part of the borrower had been foreseen at the project design stage, particularly regarding the need for a dedicated PMU, supervision costs could have been lower. Nevertheless, the PDOs were substantially achieved, albeit with delays. More intensive supervision on the part of the task team leader at the end of the project ensured its successful completion. Overall, IEG rates the quality of Bank supervision as satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance a. Government Performance The ICR (page 37) notes that the Ministry of Finance had only a limited role in the project because the Central Bank of Kosovo was the main counterpart. Nevertheless, the Ministry was strongly committed and accepted ownership of the project during its implementation. Both the Ministry of Finance and the Central Bank actively cooperated during the life of the project, particularly during its restructuring which required an inter-agency agreement between the Ministry of Finance on the one hand and the Central Bank on the other. The Ministry of Finance's financial management system supported the implementation of the project in accordance with the World Bank's requirements. Government Performance Rating Satisfactory b. Implementing Agency Performance Page 12 of 16

13 Under the original project the Central Bank of Kosovo was the primary implementing agency, with AMIK and KBA being partner implementing agencies. The project management team was established within the CBK and included representatives from each of these three agencies. Following restructuring and because of the way the additional financing was designed, the project management team consisted of only two members from the CBK and one member from the Deposit Insurance Fund of Kosovo. A problem with the CBK component of the project was its complexity and the lack of procurement experience and knowledge on the part of CBK personnel. Since part of the project involved procurement of software and issues related to the ownership of the selected BCC premises, this lack of experience contributed significantly to delays. Nevertheless, the ICR notes that the CBK was always fully committed to the project, demonstrated strong ownership, and achieved all targets. Implementing Agency Performance Rating Satisfactory Overall Borrower Performance Rating Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design Numeric indicators were not used in the original project design to monitor progress. Rather, the itemized deliverables that were included in the matrix of the results framework were used as indicators to monitor the implementation of the project. The restructured project included a small number of numerical indicators for the 3 additional components. Monitoring of project results was done by supervision missions. Some additional indicators were added to comply with World Bank reporting requirements for MSMEs these were not used to either monitor or to evaluate project performance. Many of the outcome indicators were output indicators so that if the outputs were achieved, the outcomes were automatically achieved. This was a weakness in M&E design. b. M&E Implementation The Project Management Team coordinated with the respective agencies or line ministries to monitor and implementation of the project, both under the original design and under the restructured project. Page 13 of 16

14 c. M&E Utilization The M&E system was useful in providing feedback regarding project progress and how implementation was proceeding. Although the indicators were qualitative, the project management team used them as a basis for decision-making and making specific recommendations regarding what was needed to support implementation. The ICR (page 20) notes that at the close of the project, all the PDO indicators were either achieved or surpassed despite the significant challenges and delays that had occurred during project implementation. However, note the point on the design of output and outcome indicators made in the previous section. M&E Quality Rating Modest 11. Other Issues a. Safeguards There were no safeguard policy issues associated with the original project nor were there any associated with the additional financing. b. Fiduciary Compliance All annual audits under the project were conducted by qualified accounting and auditing firms and there were no qualifications attached to the financial statements associated with the project. Reviews of accounting records from them to be consistently accurate with adequate controls to ensure the accuracy of all financial records and reports associated with the project. However, fiduciary compliance efficiency could have been improved by having a more dedicated staff to deal with coordination procurement and financial management issues. This was particularly the case with the complicated procurement processes associated with the Real Time Gross Settlements system software. c. Unintended impacts (Positive or Negative) There were no unintended impacts associated with the project. d. Other Page 14 of 16

15 Ratings Ratings ICR IEG Outcome Risk to Development Outcome Moderately Satisfactory Moderately Satisfactory --- Modest Modest --- Bank Performance Moderately Satisfactory Moderately Satisfactory --- Borrower Performance Satisfactory Satisfactory --- Quality of ICR Substantial --- Reason for Disagreements/Comment Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR (page 38) notes several lessons: It is important to assess the capacity of the implementing agency and its ability to undertake the project as a vital component of project design. There is a need for a dedicated Project Implementation Unit in circumstances where local capacity is limited and there is unfamiliarity with World Bank processes and procedures. Complexity can lead to delays -- by designing a project where there were numerous small procurement processes -- which could be overcome by consolidating the small-scale activities In IEG'asinto a larger procurement package. In IEG's assessment, the ICR could have gone further in some areas: An important issue is that providing assistance in negotiating complicated procurement processes should have been foreseen at the outset of the project. Conversations with the TTL revealed that even towards the end of the project, Kosovo counterparts did not fully comprehend the need to use World Bank procurement rules rather than local procurement rules. A specific assessment of supervision requirements at the outset of the project and as part of the restructuring in 2011 could have led to more cost efficient supervisory processes, with perhaps a Page 15 of 16

16 recommendation that the TTL should have been resident in Kosovo for at least part of the project. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR describes effectively the issues and challenges faced by the project, as well as its shortcomings and results, which were achieved in a difficult political and institutional environment. It describes clearly the evolution of the project and the restructuring. Its analysis is candid and well thought out. It clearly describes the implementation difficulties experienced by the project and the reasons for the delays. At the same time, the ICR is unclear and somewhat confusing in certain sections and there is no section on Efficacy. Parts of the ICR could have benefited from further editing. a. Quality of ICR Rating Substantial Page 16 of 16