L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD Jul ,330,316.00

Transcription

1 Public Disclosure Authorized 1. Project Data Report Number : ICRR Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Project ID P Country Macedonia, former Yugoslav Republic of Project Name REGIONAL AND LOCAL RDS PROG SUPPORT PROJ Practice Area(Lead) Transport & ICT L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD Jul ,330, Bank Approval Date 13-May-2008 Closing Date (Actual) 31-Dec-2015 IBRD/IDA (USD) Grants (USD) Original Commitment 105,200, Revised Commitment 105,200, Actual 91,512, Prepared by Reviewed by ICR Review Coordinator Group Houqi Hong J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Pellekaan 2. Project Objectives and Components a. Objectives The project development objectives (PDO) as stated in the Project Appraisal Document of April 10, 2008 (the PAD) for the Regional and Local Roads Program Support Project was "reduced cost of access to markets and services for communities served by regional and local roads." The PDO stated in the Loan Agreement dated July 21, 2008 was "to reduce the cost of access to markets and services for communities served by regional and local roads." The PDO was revised as part of the restructuring in The revised PDO, as stated in the Loan Assignment and Assumption Agreement and the Assumed and Amended Loan Agreement, both dated July Page 1 of 15

2 PHEVALUNDERTAKENLBL 12, 2013, was: "to reduce the cost of safe access to markets and services for communities served by regional and local roads in the Guarantor's territory, and to improve institutional capacity for investment planning and road safety." The Guarantor was the Former Yugoslav Republic of Macedonia. The project's achievements will be assessed against the objectives in the orginal and amended Loan Agreements. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 11-Jun-2013 c. Will a split evaluation be undertaken? Yes d. Components The project had the following three components (their appraisal and actual costs in US dollars are based, respectively, on the EUR to USD exchange rate 1.52 at project appraisal, as specified in the PAD, and the EUR to USD exchange rate 1.09 on the project completion date (December 31, 2015), obtained from the European Central Bank website ( Component 1: Rehabilitation and Periodic Maintenance of Regional Roads [appraisal cost: EUR34.00 million (US$51.68); actual cost: EUR34.85 million (US$37.99)]. Provision of financing to cover the rehabilitation and maintenance of about 330 km paved road sections identified through a European Commission (EC) - financed study, representing around 8.7% of all Regional roads, for the period between 2008 and Component 2: Rehabilitation and Periodic Maintenance of Local Roads [appraisal cost: EUR34.00 million (US$51.68 million); actual cost: EUR32.35 million (US$35.26 million)]. Provide funding for rehabilitation and maintenance of about 420 km paved and unpaved local roads (around 5% of local roads) selected by municipalities over the period, including preparation of bidding documents through provision of technical assistance. Component 3: Institutional Support [appraisal cost: EUR1.83 million (US$2.78 million); actual cost: EUR1.75 million (US$1.91 million)]. Strengthen the Government s capacity to manage and maintain the road network of the country through the Fund for National and Regional Roads (FNRR) and through provision of various types of institutional support, including (i) an institutional analysis to identify weaknesses in FNRR and define actions to overcome the weaknesses; (ii) support for the implementation of the new Road Law and the National Road Transport Strategy; (iii) support for financial and technical audits; (iv) provision of training and office and IT equipment; and (v) other activities that may be identified later. Revised Components. During the level I restructuring in 2013 the description of Component 3 was adjusted to reflect a new lender/borrower relationship (the Borrower changed from the Government to the Page 2 of 15

3 Public Enterprise for State Roads (PESR), with the Former Yugoslav Government of Macedonia acting as guarantor for the loan) and identification of some new specific activities. The description of Component 3 was changed to: "Strengthening the Guarantor's and the Borrower's capacity to manage and maintain the road network through the provision of: (i) advisory services to the Borrower and the Ministry of Transport and Communications (MTC); (ii) training staff of the Borrower and the MTC; and (iii) office and information and technology equipment, and vehicles" (the Assumed and Amended Loan Agreement dated July 12, 2013, Schedule 1, page 6). In addition, under Component 1 as part of level I restructuring the number of kilometers of the regional roads to be financed was updated from 330km to 284km to reflect the actual work completed and some road safety activities were added, including installation of traffic signs, road markings, and guardrails, rehabilitation of regional roads prone to landslides, and repair of bridges (the Restructuring Paper dated May 28, 2013, page 12). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates 1. Project cost The project s total cost was estimated at EUR70.00 million (US$106.4 million) at appraisal, which was higher than the sum of the component costs due to inclusion of IBRD front-end fee. Actual project cost was EUR69.12 million (US$75.34 million) at completion. 2. Financing The project was financed by EUR70.00 million (US$106.4 million) from the IBRD loan and EUR12.6 (US$19.15) from the Government in the form of value-added tax exemption. The Bank loan was 98.75% disbursed. 3. Key dates The project was approved on May 13, The original closing date was July 31, It was extended twice for a total of 29 months to December 31, 2015, with the actual implementation period being 7 years and 7 months, compared to the original implementation period of 5 years and 2 months. The reasons for the extensions were related to project implementation delays, including delay due to the annual cap on loan disbursement imposed by the Government to reflect its worsened fiscal conditions caused by the financial crisis affecting Europe and delay of civil works due to unusually heavy rainfall (ICR, para. 35, page 8, and Table 2.B.2, pages 33 and 34). Mid-term review was March 7, 2011, about three months behind the schedule set at appraisal. 4. Restructurings There were one level I restructuring and one level II restructuring. (a) Level I restructuring: June 11, 2013: a level I restructuring was conducted to assign the loan from the Government as the borrower to a public enterprise named the Public Enterprise for State Roads (PESR) as the new borrower. The Government became the guarantor for the loan because of fiscal constraints on the Government resulting from the financial crisis that affected Europe at the time. PESR was transformed from the original project implementing agency, namely the Agency for State Roads (ASR), and became ASR s legal successor in December PESR had certain autonomy and was not dependent on the central budget (ICR, para. 35). Some project performance indicators were added and some dropped to reflect the new legal and institutional arrangements. The PDO was revised to reflect the newly identified activities on road safety and investment planning capacity. The loan closing date was extended from July Page 3 of 15

4 31, 2013 to July 31, 2015 due to implementation delays (ICR, page 33, Table 2. B. 2). (b) Level II restructuring: April 9, 2015: a Level II restructuring which extended the loan closing date again from July 31, 2015 to December 31, As a result of the amendment of the PDO as part of the Level I Restructuring, this Review will assess the extent to which this project achieved its original and revised objectives and estimating an average of them weighted by disbursements before and after the restructuring. 3. Relevance of Objectives & Design a. Relevance of Objectives The original PDO stated in the Loan Agreement dated July 21, 2008 was "to reduce the cost of access to markets and services for communities served by regional and local roads." This was to be achieved by improving the condition and quality of a part of the network of regional and local roads." The PDO was revised during the 2013 level I restructuring. The revised PDO, as stated in the Loan Assignment and Assumption Agreement and the Assumed and Amended Loan Agreement, both dated July 12, 2013, was: "to reduce the cost of safe access to markets and services for communities served by regional and local roads in the Guarantor's territory, and to improve institutional capacity for investment planning and road safety." The Guarantor was the Former Yugoslav Republic (FYR) of Macedonia. At approval, the original PDO was consistent with the Program of the Government of the Republic of Macedonia ( ), one of the priorities of which was to integrate the main highway network with other regional and local roads throughout the country. The FYR of Macedonia s National Transport Strategy adopted in 2007 confirmed the need for investments in roads that connecting the regional centers within the country and a focus on maintenance and enhancing the functionality of the existing road network. The original PDO was also aligned with the Bank s Country Partnership Strategy for Macedonia (CPS) for the period FY2007-FY2010, which had a strong focus on management and maintenance of roads, reducing economic distance to markets, and on feeder roads and improved access from the farm gate / local communities to the main transport system. The relevance of the original project objective was rated substantial. The revised PDO was also consistent with Bank and country strategic priorities at project completion. The Bank s CPS FY2015-FY2018 identified that local road networks were underdeveloped and of low quality, and their maintenance was a problem. The CPS s Pillar 1 on growth and competitiveness put a lot of emphasis on the key role of road infrastructure in overcoming FYR of Macedonia s disadvantage as a small landlocked economy, and on the need to strengthen road safety and improve the capacity for evidence-based investment choices. Pillar 1 of the Government s program for the period from 2014 to 2018 focused on improving the living standards of the citizens through better road and utility infrastructure. One focus of Pillar 2 of the Government program was on rehabilitation of regional and local roads. The revised PDO s emphasis on improved institutional capacity for investment planning was consistent with Pillar 7 of the Government program, which was focused on transparent and efficient work of the Government and public administration. The relevance of the revised project objective was also rated substantial. Page 4 of 15

5 Rating Revised Rating b. Relevance of Design The project s original activities were relevant to the achievement of the original PDO. They focused on rehabilitation and periodic maintenance of regional and local roads, mostly civil works related to localized repairs, replacement, asphalt resurfacing or re-gravelling, and upgrading. This, combined with the institutional support component on strengthening of the government s capacity to manage and maintain the road network, would contribute to improved conditions and quality of the targeted regional and local roads, to be identified through a European Commission (EC) - financed study and by municipalities through a participatory approach. The improved road conditions and quality would reduce vehicle operating cost, thus reducing the cost of access to markets and services by communities served by these regional and local roads. The results framework was logical and had a set of sound indicators that were measurable and relevant to the PDO. The relevance of design was rated substantial. The slightly revised project design introduced some new activities on road safety improvement for regional roads as mentioned in Section 2 above and on the establishment of an asset management system that would contribute to improved investment planning capacity (the Loan Assignment and Assumption Agreement 2013, Schedule 1). It didn't include any specific activities on road safety for local roads or on road safety capacity building. However, typical safety measures were already embedded in both components (ICR, paras. 73 and 86). On the whole, the revised project activities were substantially consistent with the revised PDO which had an added focus on road safety and institutional capacity for investment planning. There was no results chain presented in the revised results matrix except a listing of objectives and indicators, but in combination with the project's revised activities, a logical results chain can be inferred from the matrix. The relevance of the amended design was rated substantial. Rating Revised Rating 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To reduce the cost of access to markets and services for communities served by regional and local roads. Rationale The project's most important outputs and outcomes relevant to the project's first objective above can be summarized below, based on evidence from Section 3.2 of the ICR, except as mentioned otherwise. Outputs: Page 5 of 15

6 Rehabilitation/reconstruction of about 290 km of regional roads selected through a European Commissionfinanced study conducted by the consulting firm Hydroplan (compared with a target was 330 km). Regular technical audits and field inspections confirmed that road works complied with contractual specifications and were of good quality, and that traffic signs, road markings, and lighting were installed on critical sections after the Level I Restructuring. Rehabilitation of 443 km local paved and unpaved roads selected by municipalities through a participatory approach (Target was 420 km). Regular technical audits and field inspections have confirmed that road works complied with contractual specifications and were of good quality. And typical safety measures were embedded in the design of these local roads (ICR, para. 73). The roughness of the local roads rehabilitated under the project, as measured by the International Roughness Index (IRI), was estimated to have on average been reduced by 80 percent or more (Many of these roads were unpaved before the project). There were some issues related to vulnerable road slopes but there is no information in the ICR on whether these issues were addressed, although the ICR mentioned that the Bank project team provided advice on measures to address these issues (see more discussion of this matter under Safeguards in Section 11, of this Review). Outcomes: An average decrease in Vehicle Operating Costs (VOCs) of 11 percent (target was 10 percent), from Euro 1.18 per vehicle-km before the project to Euro 1.05 per vehicle km at project completion. A survey found that the project improved access to health care, social and educational facilities, other municipal services, and markets, though the ICR provided no quantitative evidence (ICR, Annex 5, page 40). Rating PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective Following the Level I Restructuring in 2013 the project s objective was amended to the combination of two sub-objectives: (a) "to reduce the cost of safe access to markets and services for communities served by regional and local roads in the Guarantor's territory" (Objective 1 Revision 1); and (b) "to improve institutional capacity for investment planning and road safety" (Objective 1 Revision 2). The extent to which Objective 1 Revision 1 was achieved is assessed below. Revised Rationale The project's most important outputs and outcomes relevant to Objective 1 Revision 1 can be summarized as follows, based on evidence from Section 3.2 of the ICR, except as mentioned otherwise. Outputs: Rehabilitation/reconstruction of about 290 km of regional roads selected through a European Commissionfinanced study conducted by the consulting firm Hydroplan (target was 284 km). Regular technical audits and field inspections confirmed that road works complied with contractual specifications and were of good quality. Page 6 of 15

7 Completion and implementation of a design on traffic signs and road markings on regional roads rehabilitated under the project, in accordance with the EU guidelines on road signs and equipment (ICR, para. 78). Rehabilitation of 443 km local paved and unpaved roads selected by municipalities through a participatory approach (Target was 420 km). Regular technical audits and field inspections have confirmed that road works complied with contractual specifications and were of good quality. And typical safety measures were embedded in the design of these local roads (ICR, para. 73). The roughness of these local roads, as measured by the International Roughness Index (IRI), was estimated to have on average been reduced by 80 percent or more (many of these roads were unpaved before the project). There were some issues related to vulnerable road slopes but there is no information in the ICR on whether these issues were addressed, although the Bank team provided advice on measures to address these issues (see more discussion of this matter under Safeguards in Section 11 of this Review). Outcomes: An average decrease in Vehicle Operating Costs (VOCs) of 11 percent (target was 10 percent), from Euro 1.18 per vehicle-km before the project to Euro 1.05 per vehicle km at project completion. The beneficiary survey found that the project improved access to health care, social and educational facilities, other municipal services, and markets, though no quantitative evidence was provided (ICR, Annex 5, page 40). The beneficiary survey also indicated that in some communities, the road rehabilitation resulted in safer roads and that the Demonstration Road Safety Project improved the road safety on the 10 km long demonstration regional road. In sum, the project reduced the cost of safe access to services and markets by communities served by the regional and local roads constructed under the project. Although the survey does not show that in all communities the road rehabilitation resulted in safer roads, traffic signs, road markings, and lighting were installed on critical sections of the regional roads rehabilitated under the project in accordance with EU guidelines and thus should have contributed to road safety. Revised Rating PHINNERREVISEDTBL Objective 1 Revision 2 Revised Objective To improve institutional capacity for investment planning and road safety. The extent to which this Objective 1 Revision 2 was achieved is assessed below. Revised Rationale The project's most important outputs and outcomes relevant to Objective 1 Revision 2 can be summarized as follows, based on evidence from Section 3.2 of the ICR, except as mentioned otherwise. Outputs: Excel-based road database established on the basis of a condition survey of core road Page 7 of 15

8 network, containing key information on nearly all motorways, national roads and 80% of regional roads, and a highway database management software was installed and the staff of the Public Enterprise for State Roads (PESR) were trained on the usage of the software, for the purpose of establishing the Road Asset Management System. A Financial Module of Enterprise Resource Planning System was established and operational in PESR, accommodating detailed financial records by source of funding and automation of report generation while improving the capacity to control revenue collection. Preparation of a report on socio-economic impact assessment of local roads and the National Environmental Road Handbook. Delivery of a training on environmental management (ICR, Annex 2). Completion of a road rehabilitation and strengthening design study. Preparation of the guidelines for the identification and design of improvements to local roads. Completion and implementation of a design on traffic signs and road markings on regional roads rehabilitated under the project, in accordance with the EU guidelines on road signs and equipment (ICR, para. 78). Outcomes: The Road Asset Management System was established and operational, and has been used by the Public Enterprise for State Roads (PESR), with assistance from consultants, to produce Five-Year Rolling Strategic Program, meeting the target set in the Restructuring Paper dated May 28, Completion and implementation of the European Union (EU) Guidelines on road signs and equipment - based design on traffic signs and road markings for regional roads rehabilitated under the Project (met the target) (ICR, Section 3.2 and Annex 2). Although the installation of safety structures at first was inadequate in regional roads they were ultimately installed following collaboration of PESR (ICR, para. 39). In sum, the establishment and operationalization of the Road Asset Management System and the development of relevant operational handbook and guidelines indicate that the institutional capacity for investment planning is improved. The completion and implementation of the EU Guidelines-based design on traffic signs and road markings is to some extent an indication of improvement of institutional capacity for road safety, although the evidence could be further strengthened. Revised Rating PHREVISEDTBL 5. Efficiency At project completion an ERR analysis was conducted for all 11 regional roads rehabilitated under the project. The ERR for each regional road ranged from 13% to 124%, substantially higher than the discount rate of 8% adopted in the PAD and the ICR (Annex 3, para 1). The overall ERR of all the 11 regional roads was 48%, lower than the ERR of 74% estimated at appraisal but much higher than the adopted discount rate of 8%. The total cost of the 11 regional roads was about 50% of total project cost at completion. There was no economic analysis for the local roads component at appraisal. An ex-post ERR analysis at the completion of the project was conducted for the around 440 km of local roads upgraded under the project. Page 8 of 15

9 The ERR analysis was based on the average characteristics of these roads. The overall ERR of these local roads was 28%, significantly higher than the adopted discount rate of 8%. The total cost of these local roads represented 47% of actual total project cost. The overall ERR of both the regional and local roads was 38% at project completion. The project experienced some delay mainly due to global financial crisis and delay of civil works due to unusually heavy rainfall, as mentioned in Section 2 above. The ERR analysis could have been strengthened if a sensitivity analysis had been conducted. Nevertheless, the ERRs were high and sensitivity analysis may not have revealed any additional useful information. The two ERRs quoted in the table below are the estimates for regional roads at appraisal and at project completion respectively. Efficiency Rating a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) Appraisal ICR Estimate Not Applicable Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of both the project's original and revised objectives is rated substantial. The relevance of design is also substantial both before and after the change in the PDO. The efficacy for either the original or the revised objectives is, and both objectives were achieved with substantial efficiency. The outcomes for the original and the revised objectives are thus both rated. Considering the project s disbursement at the time of the PDO revision was about 69 percent of total disbursement, the project s overall outcome is rated as, based on a disbursement-weighted average of the outcome ratings for the original and the revised objectives. a. Outcome Rating 7. Rationale for Risk to Development Outcome Rating Page 9 of 15

10 A major risk to the sustainability of the development outcome of the project is maintenance of roads and road safety structures. The Road Asset Management System (RAMS) established under the project will provide guidance to the Public Enterprise for State Roads (PESR) on maintenance planning and implementation. And the ongoing National and Regional Roads Rehabilitation Project (NRRRP) is supporting the further expansion of the RAMS system and the monitoring of PESR's financial management and financial capacity. However, the availability of necessary political will and financial resources for the funding of these activities is uncertain, because there are no formal institutional arrangements to ensure that adequate budget allocations will be made to the maintenance, while PESR's financial capacity to adequately maintain the regional roads could be restricted starting in 2019 "when the repayment of a loan from China starts" (ICR, para 90). a. Risk to Development Outcome Rating 8. Assessment of Bank Performance a. Quality-at-Entry The project design was based on a strategically relevant objective and a set of project activities logically linked to the project objective. A consultative process was followed in the identification of the local roads component and in related fund allocation among municipalities, which were also charged with the responsibility for implementing the local roads component to promote their design capacity. The project's M&E indicators were clear and measurable. An intensive project supervision strategy was developed at appraisal in view of a varying project preparation capacity among municipalities. Safeguards management mechanism and fiduciary arrangements were set up properly. Risk assessment and implementation and institutional arrangements were sound. Some minor shortcomings include: (a) Detailed design for regional roads was not prepared before implementation, in order to accelerate the process to meet the momentum of on-going sectoral reforms and the launch of the Program on Regional and Local Roads by the government. But this led to prolonged implementation, and (b) Road maintenance was not adequately incorporated in the original design although addressed later during the level I restructuring through support for the establishment of the Road Asset Management System (ICR, Sections 2.1 and 5.1). Quality-at-Entry Rating b. Quality of supervision According to the ICR, the team conducted regular and comprehensive supervisions, identified key implementation issues, and provided solutions on a timely basis. It responded to project agency and government's request for advice and assistance effectively. In particular, supervision efforts were instrumental in identifying a series of gaps affecting implementation and achievement of the development outcome, such as lack of road safety consideration, landslide issues, and the need for strengthening road maintenance arrangements, and in informing dialogue with the government on the need for a system-wide approach to Page 10 of 15

11 address the above mentioned challenges. Such a system-wide approach was adopted in a follow-on Bank project in the country. The ICR shows that during the implementation process the Bank team also proactively involved in relevant institutional reforms leading to the establishment of the new implementing agency: the Public Enterprise for State Roads (PESR), and ensuring PESR's legal and managerial autonomy and its major role in developing strategic programs for government's adoption, thereby promoting evidence-based investment planning in the road sector. The team was also instrumental in promoting PESR's leading advising role to the Ministry of Transport and Communications (MOTC) on technical issues, including road maintenance and road safety issues. The Bank team monitored closely the implementation of the safeguards mechanisms and fiduciary arrangements. Quality of Supervision Rating Overall Bank Performance Rating 9. Assessment of Borrower Performance a. Government Performance The Government was very committed to the project and relevant sector reform. It had a well-developed program on regional and local roads and road sector reform that informed the project design well. In addition, the ICR reports in Section 5.2 that the government initiated reform to reduce maintenance backlog, and amended the existing law on public roads to make planning and management of road investments more reliable and transparent. These activities were directly relevant to the achievement of the project development objective. The Government was timely in establishing the implementing agency the Public Enterprise for State Roads (PESR) which served as the new Borrower while the Government changed its role from Borrower to Guarantor in response to the global financial crisis that affected Europe and constrained the Government s fiscal conditions, and equipped the PESR with sufficient staff though with some delay. It actively supported the consultation at the municipal level, which was essential for the identification and design of the local roads. The Government also supported multiple meetings with beneficiaries to identify the needs and constraints of all types of beneficiaries for improving the design of the project (ICR, Section 5.2). Government Performance Rating b. Implementing Agency Performance The implementing agency of the project was mainly the Agency of State Roads (ASR) between May 2008 (when the project was approved) and December 2012, and was the Public Enterprise for State Roads (PESR) from December 2012 to December 2015 when the project implementation was completed. PESR was the legal successor of ASR and took over all of ASR's responsibilities as the implementing agency in Page 11 of 15

12 December The implementing agencies were very committed to the project's implementation despite a shortage of staff. They managed a large volume of civil works and ensured the quality of these works, were proactive in improving and maximizing municipal participation in the consultation process for the design of the local roads component, implemented the Bank s recommendations on addressing project implementation issues in a timely manner, and were committed to staff capacity building. Performance on procurement and M&E by both ASR and PESR was satisfactory, although there was some initial delay in monitoring road works' compliance with contractual specifications due to delayed hiring of technical audit consultants. However, performance on financial management had some shortcomings. The PESR audit reports for 2013 and 2014 had two qualified opinions suggesting that there were several deviations from requirements in terms of the method of revaluation of plant, property and equipment, and accounting treatment of constructions in progress, although the matters had little or no effect on the reliability of financial management for the IBRD-financed projects and PESR developed action plans to correct them (ICR, para. 61). Implementing Agency Performance Rating Overall Borrower Performance Rating 10. M&E Design, Implementation, & Utilization a. M&E Design The implementing agency had an existing M&E system to monitor regular activities related to national and regional roads. The M&E of the regional roads under the project was automatically covered by this existing system. The system was then expanded to cover the M&E of the local roads under the project (PAD, para. 33). The main PDO level indicators were clear, measurable, and attributable, and had no major shortcomings in terms of adequacy to measure the achievement of the PDO. For example, the use of vehicle operating costs (VOCs) as a measure of the cost of road access was consistent with the best practice. The VOCs can be calculated based on the International Roughness Index (IRI) and using the World Bank s Highway Development and Management Model (HDM-4). The intermediate outcome indicators were causally linked to the PDO indicators and reflected the outputs of the project substantially. Baseline data were adequate and data collection methods were clear. The monitoring indicators were appropriately updated to reflect the level 1 restructuring in There were a couple of areas where further improvement could have been made. The PDO indicator "EU guideline on road signs and equipment being applied", which was the only indicator used to measure improvement in institutional capacity for road safety, could not fully measure whether such improvement was achieved nor was it evidence of improved safety. And there was a lack of indicators measuring the number of beneficiaries of the regional and local roads supported by the project and whether these roads served to connect these beneficiaries to markets and services. Page 12 of 15

13 b. M&E Implementation In general key M&E data were collected in a timely manner, and were of good quality. The implementing agency relied on reports from construction supervision teams and municipalities to collect data on road works completion and their quality. A technical audit was not conducted until October 2011, about three years after the start of the implementation process (the effectiveness date), due to the delay in hiring technical audit consultants. However, subsequently audit reports were produced every six months. c. M&E Utilization The M&E process identified a series of gaps affecting implementation and achievement of the development outcome, such as lack of road safety consideration, landslide issues, and the need for strengthening of road maintenance arrangements. This informed the restructuring of the project to address these issues. The M&E process also informed the Bank's dialogue with the Government on the need for a system-wide approach to address the above mentioned challenges. Such a system-wide approach was adopted in a follow-on Bank project (ICR, Section 2.3). M&E Quality Rating 11. Other Issues a. Safeguards The following safeguards were triggered in the project: OP 4.01 Environmental Assessment, OP 4.04 Natural Habitats, OP 4.11 Physical Cultural Resources, and OP/BP 4.12 Involuntary Resettlement. Compliance with OP/BP 4.12 Involuntary Resettlement was satisfactory. No information was provided in the ICR on compliance with OP 4.04 Natural Habitats and OP 4.11 Physical Cultural Resources. During project implementation there were some issues in terms of compliance with OP 4.01 Environmental Assessment, mainly related to poorly maintained road sides, vulnerable slopes, and poor environmental compliance of some of the suppliers that provided construction materials to the contractors at the project sites. The Bank team advised the borrower and the implementing agency on measures to correct these issues, although no information was provided in the ICR about whether these issues were adequately addressed. b. Fiduciary Compliance Procurement: The project's performance on procurement was satisfactory, although there was some initial delay in hiring technical audit consultants to inspect the completed works for the implementing agency. Financial Management. Performance on financial management had some shortcomings that PESR Page 13 of 15

14 (the borrower and the implementing agency) acknowledged; it proposed steps to deal with these issues. The audit reports for PESR for 2013 and 2014 had two qualified opinions suggesting that there were several deviations from requirements in terms of the method of revaluation of plant, property and equipment, and accounting treatment of construction in progress although these matters had little or no effect on the reliability of financial management for the IBRD-financed projects. According to the ICR the PESR developed action plans to correct them (para. 61). c. Unintended impacts (Positive or Negative) --- d. Other Ratings Ratings ICR IEG Reason for Disagreements/Comment IEG rated the outcome against both the original and the Outcome revised objectives over the Moderately entire duration of the project. The disbursement-weighted average of these outcomes was satisfactory. Risk to Development Outcome --- Bank Performance --- Borrower Performance --- Quality of ICR --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR presented four lessons but most were either not noteworthy or narrowly focused on this project. The one that had some broader application is, with some adaption, presented below. Flexibility in the scope of institutional support should be balanced against relevant risk mitigation. In Page 14 of 15

15 this project the scope of institutional development activities was kept flexible to allow for adjustment based on the needs of relevant agencies and the sector at large, considering that the role of the initial implementing agency FNRR (PESR s predecessor) was still evolving and its needs for capacity building were not fully known. This arrangement facilitates a more targeted support for a sector that is undergoing major reforms. A caveat is that there are risks that as a result of too much flexibility some important activities, such as capacity building for road safety enhancements in this project, may end up not being included in the component designs. Appropriate measures should be adopted to ensure that such risks are addressed. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR was concise and candid. The section on the project's strategic context was well written. While the discussion of the relevance of the PDOs and relevance of design should have followed the OPCS Guidelines for the preparation of ICRs more closely, the ICR for this project nevertheless provided an in-depth and comprehensive assessment of project design. The ICR presented a substantial body of evidence on the project's achievements, including a solid analysis on efficiency. Some discussions, such as those on the participation process by municipalities for local roads component, could have been be more detailed. a. Quality of ICR Rating Page 15 of 15