Viet Nam: Northern Power Transmission Expansion Sector Project

Transcription

1 Validation Report Reference Number: PVR-485 Project Number: Loan Number: 2225 December 2016 Viet Nam: Northern Power Transmission Expansion Sector Project Independent Evaluation Department

2 ABBREVIATIONS ADB Asian Development Bank CPPMB Central Power Project Management Board EIRR economic internal rate of return EVN Electricity of Viet Nam FIRR financial internal rate of return GDP gross domestic product km kilometer kv kilovolt kwh kilowatt-hour MW megawatt NPPMB Northern Power Project Management Board NPT National Power Transmission Corporation PCR project completion report TWh terawatt-hour WACC weighted average cost of capital NOTE (i) (ii) In this report, $ refers to US dollars. The fiscal year (FY) of the government ends on 31 December. Key Words adb, asian development bank, energy sector, lessons, performance evaluation, power sector, project completion report, transmission, validation, vietnam The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. The final ratings are the ratings of IED and may or may not coincide with those originally proposed by the consultant engaged for this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

3 PROJECT BASIC DATA Project Number PCR Circulation Date 30 Jun 2016 Loan Number 2225 PCR Validation Date Dec 2016 Project Name Northern Power Transmission Expansion Sector Project Sector and Subsector Energy Electricity transmission and distribution Theme and Subtheme Sustainable economic growth Fostering physical infrastructure development Safeguard Categories Environment B Involuntary Resettlement A Indigenous Peoples B Country Viet Nam Approved Actual ADB Financing ($ million) ($ million) ($ million) ADF: 0.00 Total Project Costs OCR: Loan Borrower Beneficiaries Others Cofinancier Total Cofinancing Approval Date 21 Dec 2005 Effectiveness Date 20 Nov Nov 2006 Signing Date 22 Aug 2006 Closing Date 31 Dec Oct 2015 Project Officers IED Review Director A. Jude D. Bui X. Humbert P. Hattle A. Wanniachchi Location ADB headquarters ADB headquarters ADB headquarters ADB headquarters ADB headquarters B. Nguyen, Officer-in-Charge, IED2 From Dec 2005 Mar 2007 Sep 2007 Mar 2011 Nov 2011 To Feb 2007 Aug 2007 Dec 2010 Nov 2011 Oct 2015 Team Members L. Hauck, Senior Evaluation Specialist, IED2 K. Thukral, Principal Evaluation Specialist, IED2 F. D. De Guzman, Senior Evaluation Officer, IED2 P. Choynowski, Consultant ADB = Asian Development Bank; ADF = Asian Development Fund; IED2 = Independent Evaluation Department, Division 2; OCR = ordinary capital resources; PCR = project completion report. A. Rationale I. PROJECT DESCRIPTION 1. In 2004, Viet Nam s economic growth reached 7.5%. The Government of Viet Nam s draft 5-year development plan for targeted an annual growth of 7.5% 8.0% in gross domestic product to create jobs and reduce poverty. An efficient and stable power sector was essential to achieving the government targets. Electricity consumption during had increased more than threefold from 11.2 terawatt-hours (TWh) to 39.9 TWh and system losses fell from 21.7% to 13.5%. Electricity demand was forecast to increase by 15% 16% per year between 2004 and 2010, and 12% per year thereafter. The revised Fifth Power Development Master Plan for prepared by the Electricity of Viet Nam (EVN) forecasted electricity generation to reach TWh in To meet the growth in electricity demand, EVN expected to increase its installed generating capacity from 2005 to 2010 by about 2,000 megawatts (MW) per year from 10,871 MW in 2004 to 22,600 MW in 2010.

4 2 Rapid growth in the manufacturing sector drove the increase in industrial electricity usage with demand growing by 18.5% during An increase in household access also drove electricity demand. Between 1995 and 2004, about 30 million people, representing 37% of the population, were connected for the first time. However, end-use consumption in 2004 was only about 500 kilowatt-hours (kwh) per capita in 2004, compared with an average of about 1,300 kwh per capita per year in low- and middle-income countries. 2. The government s Comprehensive Poverty Reduction and Growth Strategy 1 acknowledged the importance of large-scale infrastructure in achieving poverty reduction through growth. Expansion of the transmission system was also consistent with Viet Nam s commitments under the Greater Mekong Subregion Power Sector Cooperation Program. Viet Nam was financing a 220-kilovolt (kv) transmission line linking with Cambodia and the People s Republic of China and was planning power transmission interconnections with other countries in the Greater Mekong Subregion. Viet Nam ratified the intergovernmental agreement on Greater Mekong Subregion power trading and was a signatory to the memorandum of understanding on initiating power trading among Greater Mekong Subregion countries. 3. In November 2005, the Board of Directors of the Asian Development Bank (ADB) approved a $360 million 25-year sector loan to the government for on-lending to EVN to finance the $452.7 million project for strengthening the northern transmission network. 2 The project had two major components: Part A for the construction expansion and upgrading of the transmission lines; and Part B for the construction, expansion, and upgrading of substations. The sections of the transmission lines and each of the substations were subprojects, some of which were confirmed at appraisal, others eligible for financing subject to agreed criteria. The projects confirmed at appraisal were as follows: (i) 269 kilometers (km) of double circuit 500-kV transmission lines covering Son La Hoa Binh Nho Quan; (ii) 152 km of double circuit 220-kV transmission lines covering Thanh Hoa Nghi Son Vinh; and (iii) expansion of the Van Tri 220-kilovolt substation. The projects identified but still subject to confirmation at appraisal were as follows: (i) double circuit 500-kV line from Son La Soc Son (260 km); (ii) 500-kV line from Quan Ning Mong Duong (25 km); (iii) Ha Tinh Tach Keh (20 km); (iv) a new substation at Soc Son; and (v) an upgraded transformer at Thuong Tin. In all, eight Part A and Part B subprojects were planned. EVN was the executing agency, and the Northern Power Project Management Board (NPPMB) and Central Power Project Management Board (CPPMB) were the implementing agencies for the subprojects. The project continued ADB s energy sector work with the government and EVN, which began in A project completion report (PCR) was prepared in June B. Expected Impacts, Outcomes, and Outputs 4. The project s expected impact was improved and sustainable livelihood for people in northern Viet Nam. The envisaged project outcome was access of households and industries in northern Viet Nam to reliable supply of electricity with four outcome indicators: (i) northern region per capita consumption of electricity increased from 130 kwh in 2005 to 250 kwh in 2010; (ii) power outages reduced during dry season from 8 hours per day to 2 hours per day; (iii) transmission losses reduced from 14.7% in 2005 to 13.8% in 2010 and 11% in 2015; and (iv) sustainable livelihoods and adequate compensation for all people adversely affected by power projects. The project had four intended outputs: (i) five fully operational transmission 1 Government of Viet Nam Comprehensive Poverty Reduction and Growth Strategy. Ha Noi. 2 ADB Report and Recommendation of the President to the Board of Directors: Proposed Loan for the Northern Power Transmission Expansion Sector Project in Viet Nam. Manila. 3 ADB Completion Report: Northern Power Transmission Expansion Sector Project in Viet Nam. Manila.

5 3 lines, (ii) one new substation fully operating, (iii) two existing substations upgraded, and (iv) an environmentally and socially acceptable power system in northern Viet Nam. C. Provision of Inputs 5. The project was approved on 21 December 2005 and became effective on 24 November 2006 (the loan agreement s target effective date was 20 November 2006). Project completion was achieved in 23 October 2015, 4.9 years later than the 30 June 2010 closing date in the loan agreement. The delay was a result of (i) loan signing and effectiveness that took 11 months (December 2005 November 2006) because of the government s lengthy approval process for the loan; (ii) the finalization of the initial procurement plan that took 1 year (April 2007 March 2008) because of inadequate procurement packaging (ADB advised EVN to formulate large procurement packages and EVN negotiated to proceed with small packages); (iii) the Quang Ninh Mong Duong 500-kV line and the Ha Tinh Tach Keh 220-kV line that were replaced with three new subprojects in 2010; (iv) the delayed selection of the last three subprojects because of a lapse in assessing unspent loan proceeds by EVN, the National Power Transmission Corporation (NPT), and ADB; (v) delayed preparation, approval, and implementation of resettlement plans because of lack of capacity at NPPMB and delayed engagement of safeguards consultants; (vi) many small contracts (96) resulting in numerous procurement transactions requiring additional time; and (vii) the NPT s involvement since 2008, which added another layer of project administration and approval time (PCR, para. 19). 4 The processing of some land acquisition applications was also delayed by local authorities due to changes in government laws; these matters were beyond the control of EVN and the NPT. 6. At completion, the actual total project cost was $447 million (foreign cost of $193.8 million and local cost of $253.2 million). ADB financed $351.7 million (less than the approved amount of $360 million) of which $3 million was canceled on 1 August 2014, and $5.28 million was canceled at loan closing. EVN counterpart financing was $95.3 million, which was slightly higher than $92.7 million at appraisal. Savings of $13.7 million were realized due to lower interest and commitment charges than estimated at appraisal. Part of the savings was utilized to procure spare parts. 7. Although the final project cost was slightly less than the anticipated cost at appraisal of $452.7 million, the project delivered 12 subprojects, 4 more than the 8 subprojects planned at appraisal. The Quang Ninh Mong Duong 500-kV transmission line was excluded because its scheduled completion was later than the original loan closing date (2010). The Ha Tinh Thach Khe 220-kV transmission line was dropped because its purpose was to supply an iron mining development area, which was not developed as planned. Three substation subprojects replaced these two transmission line projects: (i) expansion of the Nho Quan 500-kV substation, (ii) a new 220-kV substation at Tuyen Quant, and (iii) the Bim Son 220-kV new substation. The substations were needed to improve supply reliability and reduce losses. Three additional subprojects were added to utilize the balance of funds after the canceled transmission lines and the three replacement projects: (i) a new 220-kV substation at Hai Duong, (ii) a new 220-kV substation at Kim Dong, and (iii) a new 220-kV transmission line from Thuong Tin to Kim Dong. All 12 subprojects were completed by the extended loan closing date of 31 December 2014, except for several transmission towers for the Thanh Hoa Nghi Son Vinh transmission line, 4 This validation notes that point (vii) appears to be inconsistent with PCR paragraph 8, which states: The implementation arrangements and ownership were not materially affected after the government separated the transmission functions from EVN and establishing the NPT in December 2008; and PCR paragraph 39, which states Project implementation was more efficient during the later years ( ).

6 4 which were completed in August 2015 after the resolution of resettlement issues. All other projects planned at appraisal were completed. 8. EVN was to engage and finance about 300 person-months of domestic consultants for detailed technical design, bid preparation and procurement, construction supervision, implementation of the environmental and social management plans, socioeconomic impact assessment, and project performance monitoring. The ADB loan was to finance two international consultants for a total of 10 person-months to prepare resettlement plans; guide and supervise resettlement committees; train the staff of NPPMB, CPPMB, and resettlement committees; and assist the NPPMB in preparing the initial environmental assessment and environmental management plans. In addition, ADB was to finance 14 person-months of domestic consultants to monitor compliance with agreed safeguards plans. The PCR stated that VICA Consultants International Limited was engaged by the NPPMB, and three local implementation consultants were engaged for independent monitoring and reporting. The PCR did not state the number of person-months of consulting services used, although the total cost for consultant services was provided. The PCR did not comment on the use of consultants engaged directly by EVN. 9. The project was classified as category B under ADB s environmental categorization requirements; no category A subprojects were implemented under the project. A series of public consultations was carried out in each province to avoid environmentally sensitive areas and to mitigate environmental impacts. The project s environmental management complied with ADB s environmental safeguards policy 2002 and the government s Environmental Protection Law Most environmental impacts were localized, reversible, and associated with civil works during construction and there are no residual environmental issues to be resolved. 10. The project was classified as category A for involuntary resettlement. A total of 9,409 households were affected (8,239 marginally affected, 831 relocated, and 339 severely impacted by losing more than 10% of productive lands), which led to the establishment of 23 resettlement sites. These sites were well integrated into existing communities the houses were larger and better quality than the former houses. All affected households were compensated for losses at replacement cost and livelihoods were restored to maintain or improve living standards and income generation capacity. Compensation to affected households totaled D540 billion ($25.2 million). There were delays in implementing the resettlement plans because of the significant burden on the executing and implementing agencies to comply with ADB s safeguards requirements. ADB guided the NPT and NPPMB and helped prepare action plans. The project was classified under indigenous people s category B 1,826 ethnic minority households were affected by the project, primarily under the Son La Nho, Quan Hoa Binh, and Son La Hiep Hoa 500-kV transmission lines. Stand-alone ethnic minority development plans were not required by the resettlement framework. However, specific actions were developed in the resettlement plans to address the impacts on the ethnic minority people (PCR, Appendix 8, para. 7). D. Implementation Arrangements 11. As planned at appraisal, EVN was the executing agency, although EVN was restructured in July 2008, resulting in the creation of the NPT. EVN remained the executing agency along with the NPT, and was responsible for overall project implementation and for approving subprojects. The NPPMB and CPPMB were the implementing agencies as planned. 12. Of 52 loan covenants, 3 were not complied with and 5 were partly complied with. EVN was in breach of the debt service coverage ratio for , which was required to be at

7 5 least 1.5 times. It also did not comply with the self-financing ratio covenant of at least 25% during and the debt equity ratio covenant of not more than 70:30 in EVN complied partly with the internal audit function covenant. It submitted management letters together with audited project accounts and audited financial statements each year from FY2011 to FY2014. For FY2009 and FY2010, EVN did not submit management letters. It also partly complied with the covenant requiring quarterly progress reports, which were only submitted from Of four land acquisition and resettlement covenants, three were partly complied with. Partial compliance was due to some resettlement plans not submitted to ADB for approval before these were disclosed to affected people and for starting construction works in some areas while land acquisition and resettlement activities were not complete. EVN was to manage performance monitoring, which was to be based on the performance indicators in the design and monitoring framework. II. EVALUATION OF PERFORMANCE AND RATINGS A. Relevance of Design and Formulation 14. The PCR rated the project highly relevant. The project was part of an economically beneficial and financially sound investment program that helped the government achieve the goals of sustained high economic growth and poverty reduction consistent with the government s Comprehensive Poverty Reduction and Growth Strategy, (footnote 1). The project was also aligned with the government s power sector objective of strengthening northern Viet Nam s power system by improving system reliability, removing transmission bottlenecks, reducing transmission losses, and utilizing planned power plants efficiently. The sector loan modality provided flexibility in selecting priority projects, with two originally planned projects replaced with new projects and three additional projects developed to fully utilize loan proceeds to develop the needed infrastructure. 15. The project was consistent with ADB s country strategy and program at time of approval and the country partnership strategy, at project completion. This validation assesses that the project was in line with the strategies of both the government and ADB. Likewise, the PCR indicated that the use of the sector loan modality provided flexibility in selecting priority projects. All subprojects were in accordance with updated national power development plans. 16. Despite project implementation delays (para. 5), this validation recognizes the importance of the sector loan modality in allowing the project and the agencies to construct the needed transmission lines and substations and in increasing their capacity to manage future projects. Thus, the project is assessed highly relevant. B. Effectiveness in Achieving Project Outcomes and Outputs 17. The PCR rated the project highly effective because project outcomes were fully achieved. Households and industries in northern Viet Nam had better access to more reliable electricity. Three outcome targets surpassed the design targets for 2010 and The northern region s per capita consumption of electricity increased to 740 kwh in 2010 and 1,300 kwh in 2015, compared with the expected 250 kwh in When the project was completed, there were no outages during the dry season because the reserve margin was sufficient (35% in 2014). System losses were reduced to 10.5% in 2010 and 8% in 2015, compared with the targets of 13.8% in 2010 and 11% in On the fourth outcome, all project-affected households were compensated, resettled, and rehabilitated according to approved and updated resettlement plans and due diligence reports. Allowances for improving

8 6 livelihoods were also paid to people who were entitled to them in accordance with the plans. ADB also supported the Northern Power Transmission Corporation in procuring urgent spare parts and equipment. Thus, outcomes achieved were higher than those anticipated during project design. 18. In outputs, the project delivered 12 subprojects compared with the 8 originally planned (para. 7). This validation notes that of the four outcome indicators, three surpassed the outcome targets. In the case of outputs, there were additional substations that were either built or expanded. Thus, the project is assessed highly effective. C. Efficiency of Resource Use in Achieving Outcomes and Outputs 19. The PCR rated the project efficient. The economic internal rate of return (EIRR) confirmed the substantial economic benefits of the power sector investment program. The EIRR calculation was based on a time-slice approach that combined capital expenditures in the power generation, transmission, and distribution subsectors in the northern region. The EIRR calculation compared two scenarios with and without additional investment in the northern region. The ex post EIRR of 41.8% exceeded the minimum economic hurdle rate of 12%. The ex post EIRR was also higher than the 23% EIRR at appraisal. The PCR discussed the financial internal rate of return (FIRR) as well in the section, Efficiency in Achieving Outcomes and Outputs. The FIRR is a measure of sustainability, not efficiency, and is discussed under the section, Preliminary Assessment of Sustainability, of this validation. 20. EVN did not agree to use turnkey contracts for subprojects and implemented many contracts initially, which required more staff resources from the Northern Power Transmission Corporation, from the Northern and Central Power project management boards, and from ADB than anticipated. However, procurement packaging of the last three subprojects was efficient. EVN and the Northern Power Transmission Corporation did not manage the monitoring of the social safeguard measures efficiently, but all compensation complied with ADB s safeguard policy and there were no residual issues. The effect of delays in outputs was not significant as projects were synchronized as generation was added. 21. In its review of the EIRR calculation, this validation notes that the PCR stated: US dollar values were converted to Dong, a tradable portion of costs where estimated, and a shadow exchange rate factor of 1.05 was applied based on ADB guidelines. Labor costs were not calculated separately, so no shadow wage rate factor is applied. Subsequent advice provided by the PCR team clarified the matter as follows: The engineers estimated tradable portion of the costs with careful assessments which includes domestically produced equipment and materials. We indicated traded (imported) in the PCR and recognize the word imported is misleading. The PCR team also advised that the project s labor component may have included some local unskilled laborers, but all labor was assumed to have been paid at economic costs. 22. Paragraph 10 of Appendix 10 in the PCR stated that non-incremental benefits resulting from increased consumption are calculated using the additional consumer rent, applying the demand curve and elasticity. The PCR did not explain the meaning of consumer rent or why it is relevant. The PCR team did confirm that the reference to non-incremental benefits in the PCR was an error and that these incremental benefits were valued in terms of resource cost savings. The PCR also appears to have included the financial rather than the economic costs of land in the EIRR calculation, which were discussed in the Impact section of the PCR.

9 7 23. Although this validation is not able to confirm the 41.8% EIRR calculation in the PCR, there seems little probability that the EIRR was less than 12%. Thus, the project is assessed efficient. D. Preliminary Assessment of Sustainability 24. The PCR rated the project likely sustainable. Transmission lines and substations are static assets with long economic lives. The assets are owned and operated by NPT. Project facilities visited during loan review missions and the project completion mission were well maintained and output targets surpassed targets at appraisal. The NPT used extra high voltage transmission technology and had sufficient in-house technical capability for project design management and implementation and operation and maintenance. The likelihood that project outcomes and outputs will be maintained over the economic life of the project is high. 25. The PCR stated that the project s FIRR was re-estimated at 15% at completion. This was above the recalculated weighted average cost of capital (WACC) of 2.5% and comparable to the FIRR of 14.1% expected during appraisal. The higher FIRR was a result of the increase in the transmission tariff of 25% in The move to market orientation increased the challenge for EVN to remain financially sustainable; although the tariff was increased in 2015, the PCR states that an annual tariff increase of 8% is required over the next 5 years ( ) to restore EVN s financial sustainability. The financial performance of EVN and the NPT declined because of restructuring, which increased debt and EVN is unable to sufficiently increase tariffs to pass on increases in its costs. EVN engaged in active cash management and capital investment planning to control use of funds within acceptable limits. At the time of the PCR, various reform actions were ongoing to improve the financial performance of EVN. 26. This validation notes that although the FIRRs at appraisal and project completion are comparable, at appraisal the WACC was calculated to be 6.86%. This validation notes that a clear presentation of the components for the recalculated WACC was provided in the PCR, but a specific comparison with the WACC at appraisal, given the significant decrease, would provide more support to the changed WACC. The PCR also did not provide financial statements for EVN 27. On the whole, the project is assessed as likely sustainable. E. Impact 28. The PCR rated the project impact significant. Four impact performance targets were to be achieved 5 years after project completion: (i) an average economic growth rate of 9% 10% in in the northern midland and mountainous area; 5 (ii) growth of industry and consumption in gross domestic product (GDP) by 10%; (iii) per capita GDP to reach $520 by 2010; and (iv) reduction in the poverty rate from about 43% in 2005 to about 28% in 2010 in the northern upland region. 29. Viet Nam s economy grew more than 6% per year during Average economic growth in the northern midlands and mountainous areas was 9%, in line with the impact target in the design and monitoring framework. The industry sector grew at an average rate of 6.3% during and the industry sector s contribution to GDP was 39% in Per capita GDP more than doubled the target, at $1,333 in The government made efforts to improve 5 The impact target did not specify the growth to be measured; with the discussion provided in the PCR about economic growth, this validation has added the word economic to the indicator.

10 8 access to electricity and provide lifeline tariffs so that the poor could afford and benefit from electricity supply. Access to electricity was 87% in 2004, increasing to 97.4% in 2010 and 98.4% in The project expanded the power transmission system and enabled transmission of power to distribution networks to supply new consumers (mainly the poor in remote and mountainous areas). Thus, the project contributed to poverty reduction with the incidence of poverty in the northern uplands of Viet Nam falling from 43% in 2005 to 22.5% in In this validation s view, the impact is assessed satisfactory. 6 III. OTHER PERFORMANCE ASSESSMENTS A. Performance of the Borrower and Executing Agency 30. The PCR rated the performance of EVN, NPT, NPPMB, and CPPMB satisfactory. The PCR did not rate the performance of the borrower: the government. The NPPMB was weak in implementing safeguard measures and providing quality monitoring reports. Because of their poor quality, reports had to be revised and resubmitted to ADB. ADB was aware at appraisal that the NPPMB had limited experience, so it included provisions for engaging resettlement consultants, whose recruitment was delayed. In addition, services provided by the two resettlement monitoring local consultants were insufficient for timely resolution of safeguard requirements. This delayed several subprojects and placed additional administrative burdens on ADB project staff. The project assisted in developing the NPPMB s capacity and improved its ability to address safeguard requirements. NPT s involvement from 2008 caused additional delays, as new staff was not familiar with ADB policies and procedures and the NPT s chain of administrative approvals was cumbersome and time-consuming, although project implementation became more efficient in the later years ( ). 31. This validation also considers the decisions made during the procurement process, which placed considerable strain on implementation activities. However, the various agencies worked together to identify projects needed to optimize the infrastructure being developed in accordance with ADB requirements. Thus the project assessed the performance of the borrower and the executing agency at the low end of satisfactory. B. Performance of the Asian Development Bank 32. The PCR rated the performance of ADB satisfactory. The PCR stated that the ADB project team could have been more efficient if high turnover of ADB project officers had been avoided. ADB overlooked fielding of a midterm loan review mission; but review missions that were fielded reviewed all aspects of the project thoroughly. ADB supported the executing and implementing agencies in project implementation and guided them to comply with ADB requirements. ADB used its own resources and engaged consultants to assist in the procurement at the beginning of implementation and developed satisfactory resettlement plans during implementation. ADB fielded additional missions to guide the implementing agencies in carrying out resettlement and environmental management plans. Many ADB staff activities during implementation involved addressing the substandard plans and reports received from the NPPMB. ADB was in continuous discussions with the NPPMB over to submit revised resettlement plans in accordance with ADB standards. Delays were also due to insufficient detail provided in bid evaluation reports submitted to ADB, requiring requests for clarification. Advisory and training services provided by ADB included (i) procurement principles 6 Beginning May 2016, the Independent Evaluation Department adopts the ratings terminology of the April 2016 Guidelines for the Evaluation of Public Sector Operations on development impacts. In this terminology, a satisfactory rating coincides with the significant rating that was used before.

11 9 and bid evaluation, (ii) project reporting, and (iii) safeguards policy. On the whole, ADB performance on the project is assessed satisfactory. IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS A. Overall Assessment and Ratings 33. The PCR rated the project successful. This validation also assessed the project successful. Re-evaluation of economic assessment confirmed that the project was beneficial economically. The project is likewise likely sustainable given the high FIRR. Overall Ratings Criteria PCR IED Review Reason for Disagreement and/or Comments Relevance Highly relevant Highly relevant Effectiveness in achieving Highly effective Highly effective outcome Efficiency in achieving Efficient Efficient outcome and outputs Preliminary assessment of Likely sustainable Likely sustainable sustainability Overall assessment Successful Successful Impact Significant Satisfactory Borrower and executing Satisfactory Satisfactory agency Performance of ADB Satisfactory Satisfactory Quality of PCR Satisfactory Refer to para. 37. ADB = Asian Development Bank, IED = Independent Evaluation Department, PCR = project completion report. Note: This report uses the ratings terminology of the April 2016 Guidelines for the Evaluation of Public Sector Operations. Source: ADB Independent Evaluation Department. B. Lessons 34. The PCR identified five lessons. First, the PCR suggested that ADB could have anticipated the change in executing agency during project implementation. In this validation s view and given the experience in other countries, it is difficult to predict the speed at which major restructuring of a sector will take place. Second, large contract packages or turnkey contracts should be encouraged in future projects to avoid large numbers of smaller procurement packages that place needless burdens on the executing and implementing agencies and on ADB for approval. Third, ADB could have anticipated the need for early strengthening of the capacity of the executing and implementing agencies. Fourth, monitoring of safeguards implementation should be more systematic and projects should ensure that such arrangements are in place from project design through implementation, completion, and postproject monitoring. Fifth, ADB should conduct financial analyses to evaluate whether ongoing reforms will have positive impact on the financial strength of the operating entity. This validation finds the second to fifth lessons appropriate and has no other lesson to offer. C. Recommendations for Follow-Up 35. The PCR gave eight recommendations: (i) the PCR recommended assessing the operation and maintenance capacity of the NPT under ongoing and future projects and including covenants in the loan and/or project agreements of future projects; (ii) safeguard and financial

12 10 covenants in the project design were appropriate and should be maintained in future projects; (iii) financial management assessment and financial analysis should be conducted on NPT in future projects to improve financial sustainability, including reform of the tariff regime; (iv) more time and resources are needed in the preparation of projects, including getting technical designs and costs, procurement packages, and bidding documents right to improve procurement procedures and reduce implementation delays; (v) for future projects, due diligence should focus on forming project management units with adequate staff; (vi) high turnover of ADB project officers should be avoided and ADB should deploy competent staff with knowledge of the power sector and utility practices; (vii) ADB needs to establish much stricter criteria on granting project extensions; (viii) there is a need to incorporate capacity enhancement in project designs, familiarize implementation units with ADB and government procedures, and field a critical mass of ADB staff in key sectors at the Viet Nam Resident Mission. This validation finds the recommendations sound and has no other recommendation to offer. V. OTHER CONSIDERATIONS AND FOLLOW-UP A. Monitoring and Evaluation Design, Implementation, and Utilization 36. The report and recommendation of the President stated that EVN will handle performance monitoring, which was to be based on performance indicators and/or targets stipulated in the design and monitoring framework. However, the requirement for a project performance management system was not covenanted and there was no evidence provided in the PCR that such a system was established. Therefore, it was not possible to assess the monitoring and evaluation design, implementation, and utilization of the project performance management system. B. Comments on Project Completion Report Quality 37. The PCR provided clear information about the project s outputs, outcomes, and impacts; and useful and thoughtful lessons and recommendations. One drawback is that the FIRR was incorrectly discussed in the efficiency section of the PCR, instead of in the sustainability section. The PCR quality is assessed satisfactory. C. Data Sources for Validation 38. Data sources included the report and recommendation of the President, the PCR, and loan review mission reports. D. Recommendation for Independent Evaluation Department Follow-Up 39. The PCR recommended that the project performance evaluation be scheduled some time during the fourth quarter of 2016 to the fourth quarter of This validation supports this recommendation to complete a project performance evaluation if further review of the project s efficiency and sustainability is needed.