Asian Development Bank. 6 ADB Avenue, Mandaluyong City, 1550 Metro Manila, Philippines Tel +63 2 632 4444; Fax +63 2 636 2163; evaluation@adb.org; www.adb.org/evaluation Evaluation Approach Paper Project Performance Evaluation Report for the Greater Mekong Subregion: Northern Economic Corridor Project (Loan 1989-LAO) February 2014 Team Leader: Srinivasan Palle Venkata, Evaluation Specialist (email: pvsrinivasan@adb.org) Contact: evaluation@adb.org A. Introduction 1. This evaluation approach paper (EAP) presents the background, issues to be addressed and the proposed approach and schedule for the evaluation of the Greater Mekong Subregion (GMS) Northern Economic Corridor Project (Loan 1989). The resulting project performance evaluation report (PPER) will be used as input to the ongoing Thematic Evaluation Study (TES) on Regional Cooperation and Integration (RCI). The PPER has been scheduled more than 3 years after the completion of the project completion report (PCR) in December 2010. 1 This provides sufficient time for the outputs and outcomes to be re assessed and the impacts to become apparent. B. Background 1. Country/Region and Strategic Context 2. The Lao People s Democratic Republic is landlocked by five countries: Myanmar, People s Republic of China, Viet Nam, Cambodia and Thailand. In the absence of direct access to the sea and the lack of a railway system, depends primarily on road transport and, to less extent, on river (via the Mekong River and its tributaries) and air transport. In 2012, the Lao PDR s total road network of about 43,604 kilometers (km) carried approximately 92% of the total passenger traffic. 2 The Lao PDR however, lies strategically at the heart of the six-nation GMS and plays an increasingly pivotal role as a transnational road link, facilitating rising cross-border trade, tourism, and investment. 3 3. The government recognized that strengthening the country s role as land link at the center of the GMS could help narrow the gap between Lao PDR and other regional economies. At the time of project formulation, regional trade had gained importance in Lao PDR, in view of its increasing overall share of total exports and imports with neighboring countries such as Thailand. ADB had supported strengthening the road network in Lao PDR to enhance its 1 ADB. 2010. Project Completion Report: Greater Mekong Subregion-Northern Economic Corridor Project (Lao People s Democratic Republic (Loan 1989). Manila. 2 Passenger traffic by land accounts for 2,618.9 million-persons per kilometer against the total passenger traffic of 2,849.6 million persons per kilometer. Lao Statistics Bureau Website. Data accessed online on October 2013 at: http://www.nsc.gov.la/index2.php?option=com_content&view=article&id=42&itemid=44 3 ADB. 2010. Development Effectiveness Brief: Lao People s Democratic Republic - At the Crossroads of Change.
2 linkages with its neighbors, e.g., the East-West Corridor Project, linking central Lao PDR with northern Thailand and central Viet Nam. 4 4. The GMS: Northern Economic Corridor Project (the project) was identified as a highpriority regional road at the 1994 Hanoi Meeting of the Transport Ministers of the GMS Program. It included improving the existing road in the Lao PDR linking two of its remote provinces with Yunnan Province in the People s Republic of China (PRC) and Chiang Rai in Thailand, thus creating an international north south corridor in the GMS. ADB had earlier provided regional technical assistance to examine the viability of a project linking Chiang Rai in Thailand to Kunming, Yunnan Province via the Lao PDR. 5 2. Description of the Project 5. The project was envisaged to create an international north-south corridor linking the People's Republic of China (PRC) and Thailand (see project map below). It comprised three main components project road, area development and capacity building. The road component consisted of reconstruction and upgrading to international expressway standards, the 228 km of roads from Houayxay in Bokeo province to Boten in Louang Namtha province, including bridges, a new bypass road, upgrade of ferry facilities, and consulting services for design and supervision. The road works were divided into three sections, the northern section of approximately 69 km to be funded by PRC, the middle section of 74 km to be funded by ADB, and the southern section of approximately 85 km funded by Thailand. 6. The area development plan included a resettlement plan and a social action plan involving, among other things, training of health care workers, awareness raising workshops on hygiene and sanitation, preventing HIV/AIDS, and construction of community rural access roads. The capacity building component was intended to enable, at the provincial and sector levels, independent third-party monitoring of the project s environmental aspects including sensitive issues such as protected areas and wildlife trafficking. 7. The project s main objective was to strengthen regional infrastructure linkages between GMS members through more efficient infrastructure networks. The expected impact of the project was to accelerate regional development at the subregional level and to reduce poverty at the national level. A direct link between the PRC and Thailand via the Lao PDR was intended to reduce transport costs, increase efficiency in the movement of vehicles, goods, and passengers, increase volume of interregional trade via this route, and increase the amount of investments in the project influence area. 4 ADB. 2002. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to Lao PDR for the Greater Mekong Subregion: Northern Economic Corridor Project. Manila. 5 ADB. 1997. Regional Technical Assistance for Chiang Rai-Kunming Road Improvement Project via Lao PDR. Manila.
3 Source: ADB. 2010. Project Completion Report of the Greater Mekong Subregion Northern Economic Corridor Project (Loan 1989 to the Lao People s Democratic Republic. Manila. 8. Within the Lao PDR, the project linked two of Lao s remote and poorest provinces, Louang Namtha and Bokeo, to Chiang Rai in Thailand and Yunnan province in the PRC. The road link is expected to help accelerate development process in these provinces by providing better access to markets, agricultural support services and social services (e.g., health and educational facilities), increasing income and employment opportunities and reducing poverty. 9. The project was classified as environmental category A and as a poverty intervention project as the project road passes through national biodiversity conservation area and areas with significant ethnic minority populations. According to the project documents adequate loan covenants, system of performance bonds and third-party monitoring were said to have been built into the project design and cost estimates to cover risk related to environmental and social safeguard policies during implementation period.
4 3. Major Findings and Recommendations of Previous Evaluations 6 10. The 2010 PCR gave the project a rating score just above the minimum required to be rated as successful. It rated the project highly relevant as it was consistent with the respective strategies of the government and ADB. The PCR also rated the project to be effective in achieving outcomes as travel time and vehicle operating costs were reduced and traffic grew at a rate higher than forecasted. However, it rated the project less than efficient in achieving outcome and outputs due to implementation delays and cost overruns. The project was financially closed 2 years later than planned at appraisal and total project cost increased by $33.3 million from the appraisal estimate. This was attributed partly due to the fact that ADB had no leverage over consultant supervision and contractors in the bilaterally funded sections. Although under protocol 2 of the GMS Cross Border Trade Agreement, Lao PDR could collect tolls on cross border traffic, it has been reluctant to do so for diplomatic reasons. Lack of toll revenues from PRC and Thailand meant that the project was less than likely to be sustainable as there were no alternative revenue collection mechanisms for the required maintenance of the project roads. 11. The PCR Validation undertaken in 2012, agreed with most of the PCR s assessments, but gave a lower rating to project relevance. It rated the project relevant as opposed to the highly relevant given by the PCR due to poor initial design of implementation procedures. The weakness in project s design resulted from lack of ADB s leverage over supervision of consultants and contractors of the bilaterally funded sections of the road affecting construction quality. Based on the findings noted in the PCR, the validation rated the project s impact as significant. Between the years 2004 and 2008, per capita incomes in Bokeo and Louang Namtha provinces increased by 11.5% and 14.1% respectively compared to 7.6% increase in GDP. Anecdotal evidence also suggested increase in commodity trading and individual purchasing power since the completion of the project. 12. The PCR identified the following lessons. Undue delays can occur in triggering ADB loan effectiveness if the borrower does not ensure that commitments under ADB s loan covenants are included in the bilateral loan agreements (i.e., with the PRC and Thailand, respectively). Bilateral agencies commitment to adhere to the specified ADB conditions can be complied with if it is effectively communicated to their contractors. When additional financing through bilateral loans is involved, a common understanding on implementation arrangements is needed for ADB to have influence over consultant supervision and contractors in the bilaterally funded sections. For sustainability of regional projects, it is important to ensure that any implementation arrangements agreed to, such as international transit and cross-border agreements, are actually implemented. 13. The validation report identified a few other lessons. All parties involved should agree on uniform standards (e.g. vehicular movements across international boundaries) in the early stages of project preparation to avoid enforcement issues on permissible axle load limits. Moreover, the project design and validation could have been further strengthened by a wellformulated monitoring component. Data collection for vehicle traffic should have included origin destination survey results to back up traffic forecasts on subregional flows and axle load surveys for overloaded vehicles. 6 Sources: PCR (2010) and Validation (2012).
5 14. The PCR and its validation made the following recommendations. More frequent monitoring of traffic, apprehension of overloaded vehicles and budgetary allocation to project road maintenance is needed to avoid faster road deterioration. Given ADB s strong relationship with GMS countries, ADB could assist the GMS as a whole to agree on a uniform standard for vehicle weights, axle loads, and dimensions in the GMS Cross Border Transport Agreement. ADB should also try to get an agreement between member countries on implementation of tolls for cross-border traffic which is permitted under protocol 2 of the GMS Cross Border Transport Agreement. C. Issues to be Addressed by the Independent Evaluation Mission 15. The Independent Evaluation Mission (IEM) will review the PCR s findings and expand on the PCR validation report by assessing project performance using the criteria relevance, effectiveness and efficiency in achieving outputs and outcomes, sustainability of outcomes achieved to date and impact. An indicative evaluation framework is presented in Appendix 1. In addition, the evaluation will identify issues related to social and environmental impacts and address specific concerns, omissions and limitations identified in the PCR. Specifically, it will focus on the following major issues: (i) (ii) (iii) (iv) Vehicle traffic flows. The extent of growth in volume of cross border passenger and cargo traffic, the type of vehicles (axle loads) and their origins and destinations, and the extent of traffic volume on community roads. Socio economic impacts. The extent of impact on incomes, access to markets, employment opportunities and other public services in the two provinces Louang Namtha and Bokeo. Development of economic corridor and constraints. The extent of domestic and foreign investments registered in the corridor provinces and the existing incentives and barriers to conversion of the transport corridor to an economic corridor. Operations and maintenance of road infrastructure. The progress in cross border transport agreement with border countries for collection of tolls and the use of vehicles with appropriate axle loads. D. PPER Approach and Schedule 16. In preparation for the IEM, the assigned team leader (S. Palle Venkata) has reviewed the pertinent project files and has familiarized himself with the main issues associated with the formulation and implementation of the project. It is envisaged that the consulting services of an international transport/evaluation consultant will be engaged for about 30 working days while the 2 in-country national consultants will be needed for a total of 40 working days. The international transport/evaluation consultant should have experience in evaluation of transport operations, including economic corridor development and related institutional issues. Familiarity with the economies in the GMS is desired for both international and national consultants. 17. The following proposed schedule is subject to the Government of Lao PDR s clearance of the mission as well as the engagement of suitable consultants:
6 Milestone Tentative Date Approval of evaluation approach paper III November 2013 Independent Evaluation Mission III February 2014 IV February 2014 Draft PPER for Internal Review IV March 2014 Draft PPER Interdepartmental Review II April 2014 Draft to Editor III April 2014 Submission to Director, IED2 III May 2014 Submission to Director General IV May 2014 18. Knowledge Dissemination Plan. The PPER will be available to the public after the Director General, IED, approves it. The report will be uploaded on ADB s external and internal websites. 19. Toshiyuki Yokota (Senior Evaluation Specialist) and Renato Lumain (Senior Evaluation Officer) will be the peer reviewers for this evaluation. Appendix: Evaluation Framework
7 Evaluation Criteria Relevance Evaluation Questions Was the project relevant to Lao PDR s national priorities? Was the project aligned with ADB s policies and strategies? Evaluation Framework Indicators/Information Required Lao PDR socio economic background Government development policies ADB s country operational strategy and sector strategy Source of Information RRP, PCR, PVR and BTORs Minutes of Board meetings, Staff review committee meetings, Management review meetings Methods/Analysis to be used Desk review Key informant interviews Analysis of program design indicators Was the project design appropriate in addressing the envisaged impact, outcome, and outputs? Did the project inputs, outputs and outcomes follow the logical results chain to achieve the project s objective? Project design and monitoring framework Relevant government documents government officials and other key stakeholders Were the relevant lessons learned from similar interventions earlier included in the design of the project? How was ownership and participation of beneficiaries included in the project design? Effectiveness To what extent were project outputs and outcomes achieved as indicated in the DMF? What are the factors that contributed to the achievement or non-achievement of expected outputs and outcomes? Was there adequate ownership and participation of beneficiaries in various project activities? Realized project outputs and outcomes in relation to the targets set Project monitoring framework Implementation process Issues and challenges related to achieving outputs and outcomes RRP, PCR, PVR and BTORs government officials and other key stakeholders Monitoring reports of outputs and outcomes Desk review Key informant interviews Analysis of program design indicators To what extent has travel time and
8 Evaluation Criteria Evaluation Questions transportation cost of goods and passenger traffic reduced? Indicators/Information Required Source of Information Methods/Analysis to be used Efficiency How well were the project s resources used in achieving the expected outcomes? Were there implementation delays? If so what were the reasons? Loan disbursement and fund utilization data Implementation and procurement arrangements Monitoring data on inputs and outputs Economic rate of return (EIRR) Project documents government officials and ADB procurement staff Traffic count, origin destination surveys Desk review Key informant interviews Cost-Benefit Analysis Sustainability What is the likelihood that project benefits would be sustained beyond the life of the project? Are there provisions for generating adequate revenue for operations and maintenance (e.g. agreement on user charges for transit traffic)? Assessment of revenue generating capacity and activities Assessment of progress in agreements between countries in the region Financial rate of return (FIRR) government officials Key informant interviews Has the government been able to effectively implement axle load limits on the road? Is there adequate progress in cross border trade agreements and trade facilitation measures at border crossings to maintain the momentum of cross border trade and investment? Impact What are the socio economic impacts of the project including on gender? To what extent employment opportunities increased and Information on beneficiaries (e.g. extent of access to services, livelihoods and income earning opportunities) Government reports and statistics government Analysis of data from focus group discussions/ small household surveys Direct observation
9 Evaluation Criteria Evaluation Questions poverty reduced? To what extent have envisaged project impacts such as access to markets, education and health services been achieved? Indicators/Information Required Source of Information officials, stakeholders Focus group discussions/ small household surveys Methods/Analysis to be used through field visits To what extent has there been an increase in national and transit trade passing through the corridor? To what extent has there been an increase in investment including FDI in the road influence area? Did the project have any unintended or adverse impact on the society and economy?