Viet Nam: Health Care in the Central Highlands Project

Validation Report Reference Number: PVR-344 Project Number: 37115 Loan Number: 2076 (SF) October 2014 Viet Nam: Health Care in the Central Highlands Project Independent Evaluation Department

ABBREVIATIONS ADB Asian Development Bank CPMU central project management unit DALY disability life-adjusted year DHC district health center DMF design and monitoring framework HCFP Health Care Fund for the Poor IEC information, education, and communication IMR infant mortality rate M&E monitoring and evaluation MMR maternal mortality ratio MOH Ministry of Health PCR project completion report PPMU provincial project management unit PSC project steering committee RRP report and recommendation of the President NOTE In this report, $ refers to US dollars. Key Words asian development bank, central highlands, disability-adjusted life years, daly, ethnic minorities, health care funds, independent evaluation, ministry of health, primary health care, project completion report, validation, viet nam The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

PROJECT BASIC DATA Project Number: 37115 PCR Circulation Date: Jan 2012 Loan Number: 2076 (SF) PCR Validation Date: Oct 2014 Project Name: Health Care in the Central Highlands Project Country: Viet Nam Approved ($ million) Actual ($ million) Sector: Health and social Total Project Costs: 30.58 27.94 protection ADB Financing: ($ million) ADF: 20.00 Loan: (SDR equivalent, million) 20.00 13.97 18.54 12.11 Borrower: 5.00 3.86 OCR: 0.00 Beneficiaries: 0.00 0.00 Others: 0.00 0.00 Cofinancier: Sida Total Cofinancing: 5.58 5.54 Approval Date: 9 Jan 2004 Effectiveness Date: 8 Jul 2004 30 Aug 2004 Signing Date: 8 Apr 2004 Closing Date: 30 Jun 2010 4 Dec 2011 Project Officers: Location: From: To: I. Bushan E. Bloom L. Studdert L. Studdert E. Honda N. N. Thuyen V. P. de Wit ADB headquarters ADB headquarters ADB headquarters Viet Nam Resident Mission Viet Nam Resident Mission Viet Nam Resident Mission Viet Nam Resident Mission Jan 2004 Aug 2004 Sep 2005 Mar 2006 Apr 2009 Oct 2009 Dec 2009 Jul 2004 Aug 2005 Feb 2006 Mar 2009 Sep 2009 Nov 2009 Dec 2011 Validator: K. Hardjanti, Consultant Peer Reviewer: H. Son, Principal Evaluation Specialist, IED1 Quality Reviewer: E. Gozali, Principal Evaluation Specialist, IED1 Director: W. Kolkma, IED1 ADB = Asian Development Bank; ADF = Asian Development Fund; IED1 = Independent Evaluation Department, Division 1; OCR = ordinary capital resources; PCR = project completion report; SDR = special drawing rights; SF = special fund; Sida = Swedish International Development Cooperation Agency. A. Rationale I. PROJECT DESCRIPTION 1. This health care project in Viet Nam was to support the implementation of a new government initiative the Prime Minister s Decision 139 that aimed to establish the Health Care Fund for the Poor (HCFP). 1 It emphasized the capacity development of the provincial health departments for planning, financing, and managing health services. 2. The Central Highlands was chosen as the project site. It was one of Viet Nam s poorest regions with relatively poor health indicators mainly due to inadequate access to quality and culturally compatible health services. Poor access was associated with (i) lack of skills among health care personnel, (ii) lack of public health care facilities, and (iii) limited affordability of health services. There was a great need to upgrade and strengthen health infrastructure and the skills of health care providers. Thus, significant capacity development was considered necessary to effectively implement Decision 139. 1 ADB. 2003. Report and Recommendation of the President to the Board of Directors: Proposed Loan to the Socialist Republic of Viet Nam for the Health Care in the Central Highlands Project. Manila.

2 B. Expected Impact 3. It was envisaged that the project would have a significant impact on the health status and well-being of the Central Highlands population, which would eventually help to improve incomes through higher productivity because of a healthier population. 4. The project s impact or goal had the following performance indicators: (i) infant mortality rate (IMR) reduced by 30% from 64 to 40 per 1,000 live births, (ii) maternal mortality ratio (MMR) reduced from 170 to 110 per 100,000 live births, and (iii) decreased burden of disease by 15% of current disability life-adjusted years (DALYs). C. Objectives or Expected Outcome 5. The project s objectives were to (i) improve the availability of quality health services, (ii) improve the affordability and utilization of health services, and (iii) strengthen the capacity of the health system to address community health needs. 6. The expected outcomes were (i) improved quality of health services at district health centers (DHCs) through improved facilities, equipment, and human resources development; (ii) strong capacity to design and implement culturally compatible information, education, and communication(iec) programs; (iii) strong training capacity of the health system at the provincial level through strengthened secondary medical schools; and (iv) effective preventive health care system through upgraded provincial preventive health centers and trainings provided. D. Outputs 7. The project was expected to have the following outputs under the following activities: (i) Upgraded facilities and equipment: (a) upgrade 15 DHCs, (b) provide equipment for 16 DHCs and Dak Lak provincial hospital, and (c) strengthen IEC centers in four provinces. (ii) Human resources developed: (a) train doctors, nurses, and health workers; (b) upgrade all preventive health centers (including centers for social diseases); and (c) strengthen the secondary medical schools in four provinces. (iii) Financing and management of services strengthened: (a) support the HCFP in all four provinces, and (b) strengthen management and supervision capacity. E. Provision of Inputs 8. The project s cost was estimated and approved at $30.6 million: (i) $20.0 million financed by the Asian Development Bank (ADB) from its special funds resources, (ii) $5.6 million financed by the Government of Sweden, and (iii) $5.0 million provided by the Government of Viet Nam. The actual total project cost was $27.9 million. ADB provided $18.5 from its Asian Development Fund, the Government of Sweden through the Swedish International Development Cooperation Agency (Sida) supported with $5.5 million, and the Government of Viet Nam supported with $3.9 million. F. Implementation Arrangements 9. The Ministry of Health (MOH) was the executing agency. Implementation was partly decentralized through five provincial project management units (PPMUs) and a central project management unit (CPMU). The PPMUs implemented the project; while the CPMU provided

3 technical, coordination, and logistical support. The project steering committee (PSC) in the MOH guided the CPMU. The PSC secretary is also the project director, and the committee was chaired by a vice minister of the MOH with members from representatives of key departments and Sida. II. EVALUATION OF PERFORMANCE AND RATINGS A. Relevance of Design and Formulation 10. The project completion report (PCR) rated the project relevant. 2 The project design and formulation were highly supportive of the government s socioeconomic development plans for 2001 2005 and 2006 2010, 3 which aimed to increase life expectancy and reduce IMR and MMR. There was greater attention to the needs of the poor and disadvantaged by emphasizing on disadvantaged regions and targeting grassroots health systems that are used more by the poor. The government s efforts to decentralize the management and provision of health care were also factored in. The project design supported the ADB country operational strategies and the ADB Strategy 2020, 4 and its complementary operational plan for health. It was based on the ADB Health Policy, 5 which has the following principles: (i) focus on improving the health of the poor, women, children, and indigenous peoples; (ii) increase public investment in primary health care; and (iii) strengthen the health sector s managerial capacity. 11. Viet Nam has made significant progress toward achieving the Millennium Development Goal of reducing mortality rate among children under 5 years old. However, regional inequities in health status and utilization of health services still persist. Although the IMR has declined in most disadvantaged regions, large gaps remain between these regions and more socioeconomically advantaged regions (i.e., Southeast and Red River Delta). 6 The Central Highlands is one of the poorest regions in the country. 7 Rates of gross poverty are much higher than the nationwide average of 13.4%, with Kon Tum having a poverty prevalence of nearly 27%. These provinces are characterized by large numbers of ethnic minorities living in scattered and remote communities. 12. After project approval, a new province Dak Nong was split from one of the project provinces, Dak Lak. Dak Nong was also included in the project. After the midterm review, there was a change in the project s scope that expanded support for access to health insurance, thereby allowing greater access for the poor to health services. These scope changes kept the project objectives consistent with the strategies of the government and that of ADB. 13. If the project s rating was based only on para. 10, it would have been rated highly relevant. However, the rating for relevance also underscores the relevance of project design and formulation. The project s implementation experiences suggested there were design issues at entry. These included overly ambitious outcome targets and lack of data to monitor these (para. 14). Work delays due to inadequate cost estimates (para. 29) also suggested issues at the design stage. Therefore, this validation concurs with the PCR to rate the project relevant. 2 ADB. 2012. Completion Report: Health Care in the Central Highlands Project in the Socialist Republic of Viet Nam. Manila. 3 Government of Viet Nam. 2006. Socio-Economic Development Plan, 2006 2010. Hanoi. 4 ADB. 2008. Strategy 2020: The Long-Term Strategic Framework of the Asian Development Bank, 2008 2020. Manila. 5 ADB. 1999. Policy for the Health Sector. Manila. 6 MOH, Government of Viet Nam. 2010. Joint Annual Health Review. Hanoi. 7 The Central Highlands region comprises Kon Tum, Dak Lak, Dak Nong, Gia Lai, and Lam Dong provinces.

4 B. Effectiveness in Achieving Project Outcome and Outputs 14. The PCR rated the project effective. The design and monitoring framework (DMF) as presented in the report and recommendation of the President (RRP) and PCR was adjusted for the purpose of validation. The DMF goals, purposes, and outcomes are recognized as impact, outcomes, and outputs, respectively. The DMF outputs and/or activities are recognized by this validation as simply activities. Of the six target indicators for purposes and/or outcomes in the DMF (PCR, Appendix 1), four were achieved at project completion. Two were considered ambitious targets: (i) to increase the utilization rates of DHCs by the poorest quintile by 100%, and (ii) to reduce private out-of-pocket expenditure for health services by the poorest quintile by 50%. These could not be assessed because these metrics were not monitored. The PCR offered other proxy indicators, which were proportions of the poor seeking inpatient and outpatient care in government hospitals and quintile 5 (poorest) population using health insurance. Viet Nam s household living standard surveys in 2004, 2006, and 2008 showed that the proportion of the poor that used outpatient care increased by 37% compared with 22% for non-project provinces, 8 most likely as a result of the subsidized services for the poor. 9 The PCR (Appendix 1) reported that the quintile 5 (poorest) population using health insurance rose by 158%. 15. The project achieved the following outputs: (i) 12 of the expected 15 DHCs were completed to MOH standard, (ii) eligible and registered poor people in target regions received funding from the HCFP, (iii) four IEC centers were completed to MOH standards, (iv) medical schools were constructed in Gia Lai and Dak Lak provinces (Dak Lak Secondary Medical School has achieved regional accreditation), and (v) two preventive health centers were completed to MOH standards. Of the five main output indicators, four were achieved. A shortfall was noted in the upgrading of the DHCs as only 12 were upgraded instead of 15 as planned. 16. Achievements in the project area also included the following: (i) improved preventive health service capacity due to enhancement of laboratories for preventive health and the building of IEC and social disease centers; (ii) successful village health worker training, capacity strengthening for IEC, and the IEC campaigns; and (iii) improved annual health and hospital operating plans that led to better resource generation and allocation as observed by the Department of Planning and Finance of the MOH during its annual provincial plans review. However, the proportion of minority population using government hospitals for outpatient and inpatient services remained lower than the overall population during 2004 2008. The proportion of the overall population using hospitals also decreased because more patients used commune health stations and regional polyclinics. 8 The difference in health status indicators and health-seeking behavior is compared for 2004, 2006, and 2008 between project and control provinces. The approach follows the one used in the Health Policy Institute s evaluation of the Rural Health Project. Control provinces are non-project provinces with the Millennium Development Goal indexes within the range of the project provinces (i.e., from 0.2920 for Gia Lai to 0.4915 for Lam Dong), thus, eliminating any provinces that are much more disadvantaged or much more advantaged than the set of project provinces. Millennium Development Goal indexes for each province are found in the report United Nations in Viet Nam. 2003. Millennium Development Goals: Closing the Millennium Gaps. Hanoi. Changes observed in health-seeking behavior occurring in project provinces during 2004 2008 that are different from changes in control provinces are considered as project impact. Household sample survey data from three nationally representative surveys conducted by the General Statistics Office the household living standards survey of 2006, and the household living standards survey of 2008, to compare with baseline data of the household living standards survey of 2004. 9 The Viet Nam household living standard surveys in 2004, 2006, and 2008 compared the project provinces with a set of control provinces. Control provinces had similar United Nations Development Programme human development index scores to those of the project provinces just prior to project implementation.

5 17. Overall, this validation rates the project effective. This rating was based on the project s achievements of its expected outcomes in (i) improving access to health services, and (ii) strengthening health systems to address the health needs of the population. C. Efficiency of Resource Use in Achieving Outputs and Outcomes 18. The project is rated efficient by the PCR based on its process efficiency, such as the project s (i) pace of disbursement, (ii) completion of all project activities, and (iii) timely achievement of its quantitative and qualitative results. Initially, the implementation was slow, causing delays in disbursement. The main causes of the slow project start-up were (i) the CPMU could not cope with the task of administering this and another major project (Preventive Health Services Support Project) during the initial stage of project implementation, 10 and (ii) there was delay in recruiting national consultants for the HCFP component. The latter was due to the limited expertise and planning capability available in the provinces, low salaries, and possibly limited perceived benefits of these consulting inputs by counterparts. Another reason cited for delays was the lengthy and time-consuming procedures for procurement. The midterm review mission found that the overall project implementation progress by the third quarter of 2007 was 32% as compared to the elapsed loan period of 55%, which was mainly caused by the slow project start-up. However, as the project progressed and the issues hampering project implementation were solved, the pace became faster, targets were achieved, and reports were received on time. 19. The PCR did not include an economic internal rate of return and no justification was made on why this was not done. It mentioned that typically, efficiency is assessed using costeffectiveness measures, or qualitative description of the efficiency of process. For example, the efficiency with which executing and implementing agencies managed the project, and the efficiency of ADB support, supervision, and administration can be considered. However, in addition to process efficiency, a cost-effectiveness analysis was conducted as part of the effort to increase DALYs in the project area. The cost per DALY averted was estimated to be $43. 11 The World Health Organization considers health intervention to be cost effective if it averts a DALY at a cost below three times the per capita gross income of the country and/or area where an investment is being made. In 2008, the per capita gross income in Viet Nam was about $900 nationally and $600 in the Central Highlands. With a DALY of only $43, the project is highly cost-effective as far as averting DALY is concerned. 20. Based on the discussions above, this validation concurs with the PCR and rates the project efficient. D. Preliminary Assessment of Sustainability 21. The PCR assessed the project outcome partly sustainable. The project has invested in the provision of additional trained staff, health facilities, and equipment. These require ongoing recurrent budget for maintenance; even additional staff are required moving forward to match increases in curative and preventive capacity. The estimated annual recurrent budget is $0.8 million per annum. An analysis of provincial health accounts showed that 3% of the local budget is used for maintenance. This amounted to only $0.2 million per year for Dak Nong province and $0.5 million per year for Dak Lak, as compared to the annual requirement of 10 ADB. 2005. Report and Recommendation of the President to the Board of Directors: Proposed Loan and Asian Development Fund Grant to the Socialist Republic of Viet Nam for the Preventive Health Services Support Project. Manila. 11 DALY is an indicator of the time a person lived with a disability and the time lost due to premature mortality.

6 $0.8 million. Funds available are not sufficient to maintain the achieved project outputs and outcomes, therefore, this validation rates the project outcome less than likely sustainable. E. Impact 22. The PCR did not rate the project s impact. The impact envisaged was improved health status of the people, especially the poor and ethnic minorities in the central highland provinces. The DMF in the PCR and para. 15 in this validation reported that most of the impact targets (as project goals) were largely achieved. The IMR was reduced to 41.6 per 1,000 live births in 2009 (marginally above the target of 40), under-5 mortality was decreased from 43.7 to 41.6 in 2009, and illness was reduced by 10% (target is 15%) in project provinces. The original impact indicators of MMR (to decline from 170 to 110 as target) were not reported in the PCR. 23. The PCR also stated that at appraisal, the Central Highlands recorded about 280 DALYs per 1,000 persons were lost, compared with the national average of 180 DALYs per 1,000. 12 As described in the PCR, it appears that the project has contributed in minimizing the DALYs at the project areas through its (i) investments in health infrastructure, (ii) quality improvement of health staff, (iii) provision of funds for the poor to increase their access to health services, 13 and (iv) reduced IMR. The household living standards survey revealed that the decrease in illness was 10% more in project provinces when compared with non-project provinces. 24. An increase in the access to preventive and primary health services was noted, 14 and other development partners and the government have adopted the concept of the HCFP. The number of workdays lost to disease decreased by more than 20% for the total population and the poorer two quintiles. 15 The government has changed its health care finance and insurance laws to ensure that the poor people continue to be subsidized starting 2011. Based on these, this validation rates the potential impact of the project significant. 25. Covenants to the project loans were mostly complied with. These were (i) implementation of the Prime Minister s Decision 139; (ii) requirements that IEC documents explaining interventions under the HCFP component were developed in the languages of the indigenous peoples, and that 22% of all trainees under the project were from ethnic minority groups; (iii) the government followed the gender strategy prepared during appraisal; (iv) new DHCs were designed to be gender-friendly, with separate washrooms and consultation areas for women; (v) for the MOH to ensure that health staff and construction workers involved in project civil works had awareness of HIV/AIDS prevention through training programs and IEC activities; (vi) the ADB 2002 Environment Policy and MOH policies and procedures on medical waste management were complied with; and (vii) project monitoring was done during semiannual review missions. An evaluation of the project was conducted during the PCR mission and a report was prepared after project closing. 12 The higher rate reflects the substantial burden of disease in the Central Highlands due to low health status of the poor population. 13 The household living standards survey measured increased access to health services by measuring the decrease of illness in the last month. 14 From 2006 onward, there have been reductions in the burden of disease in project provinces, as measured by the prevalence of illness in the 4 weeks preceding the Viet Nam household living standards survey. IMR in the project area declined from 60 to 41 per 1,000 population. The reduction is demonstrated in the number of days lost to illness in project and non-project provinces during 2004, 2006, and 2008. 15 This is due to the project s contribution in minimizing DALY, which is a measure of overall disease burden, expressed as the number of years lost due to ill-health, disability, or early death. For example, one DALY is equal to 1 year of healthy life lost.

7 III. OTHER PERFORMANCE ASSESSMENTS A. Performance of the Borrower and Executing Agency 26. The PSC was set up within 3 months of loan effectiveness. In general, the PPMUs and CPMU were satisfactory in performing their functions and reporting on the project progress and issues. The reports on project implementation progress, issues, and plans were submitted quarterly. Although most CPMU staff had no prior experience with ADB funded operations and had to familiarize themselves with ADB procedures and processes, the implementation progress, especially of the HCFP component, improved over time. This especially started after the appointment of a project deputy director in 2007. Consultations held during the project completion review mission revealed that the provincial staff members were satisfied with CPMU s leadership and support. An audit firm was engaged to conduct annual audits of all the project activities at the CPMU and five PPMUs. The auditing report was submitted to ADB and MOH on time according to the government s and ADB s regulations. The PCR rated MOH s performance satisfactory. This validation rates the performance of the borrower and executing agency satisfactory. B. Performance of the Asian Development Bank 27. Throughout the project s life, ADB conducted 12 missions 1 loan inception mission, 10 review missions (including one midterm review mission), and 1 one project completion mission. On average, ADB undertook two missions per year. From March 2006 until the actual project closing date of December 2011, the project team leaders were posted in Hanoi at the Viet Nam Resident Mission of ADB. 16 This way, the project team leader and the resident mission staff could provide more regular support during project implementation. ADB demonstrated adequate flexibility on when to adapt and change the project scope and budget allocations as required, and this was appreciated by the government. During ADB review missions, issues were raised on (i) project implementation delay and the need to enhance the capacity of the project management unit in getting familiar with ADB procurement system, and (ii) the need to train staff in health centers in administrative and financial procedures to facilitate the smoother flow of funds and accountability during implementation, and these were supported. Overall, this validation concurs with the PCR rating and rates ADB performance satisfactory. IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS A. Overall Assessment and Ratings 28. This validation rates the project successful based on assessments that the project was (i) relevant in its design and formulation, (ii) effective in achieving project outputs and outcomes, (iii) efficient in implementation processes and cost effectiveness, and (iv) less than likely sustainable in terms of sustainability. 16 The project team leaders posted at the Viet Nam Resident Mission from March 2006 until December 2011 were as follows: (i) L. Studdert March 2006 March 2009, (ii) E. Honda April 2009 September 2009, (iii) N.N. Thuyen October November 2009, and (iv) V.P.de Wit December 2009 December 2011 (Project Basic Data table).

8 Overall Ratings Criteria PCR IED Review Reason for Disagreement and/or Comments Relevance Relevant Relevant Effectiveness in Effective Effective achieving project outcome and outputs Efficiency in achieving Efficient Efficient outcome and outputs Preliminary assessment of sustainability Partly sustainable Less than likely sustainable Overall assessment Successful Successful Impact Not rated Significant Refer to para. 24. Borrower and executing Satisfactory Satisfactory agency Performance of ADB Satisfactory Satisfactory Quality of PCR Satisfactory Refer to para. 35. ADB = Asian Development Bank, IED = Independent Evaluation Department, PCR = project completion report. Note: From May 2012, IED views the PCR rating terminology of "partly" or "less" as equivalent to "less than" and uses this terminology for its own rating categories to improve clarity. Source: ADB Independent Evaluation Department. B. Lessons 29. The PCR derived five major lessons from the project. First, complex and multi-activity projects such as this are better done in adjoining areas (e.g., provinces) to ensure implementation efficiency and more uniformity of issues being addressed. Second, continuing stakeholder training is important to implementation success. Third, there is a need to pay closer attention to accurate cost estimates and provisioning for price inflation in Viet Nam. Work delays under the project were associated with price increases and compounded by unfamiliarity with procurement regulations of both ADB and the government. Fourth, more attention is needed to monitoring outcomes. Given ADB s results-based approach to project management, outcome monitoring would have been a priority. Fifth, future projects should consider investment size that takes into account the limited capacity of the central and local governments to cover required recurrent expenditures and the need of the projects to use appropriate technology with manageable cost implications. This validation considers these lessons to be appropriate. 30. This validation would like to suggest an additional lesson on the importance of establishing a baseline data at an early stage of project implementation, or during project preparation, to enable the accurate evaluation of the project s results. This is a recurring issue that warrants serious attention. C. Recommendations for Follow-Up 31. The PCR included a number of recommendations, seven of which were project related and one was general in nature. This validation considers these recommendations appropriate for future projects. The general recommendation was to prevent project delays by minimizing the complexity of ADB and government procurement procedures to enable a more timely and efficient implementation. This validation suggests that simplifying procurement processes be done as a national policy, by aligning country procurement systems to that of ADB, while strengthening the country s procurement system that is linked, at the same time, with improved

9 procurement outcomes. The danger of oversimplifying procurement processes is the risk of reducing safeguards that are in place to ensure successful bids, and to purchases that are accurate and create value for money. 32. Other recommendations were project specific, as follows: (i) investing in lower-level health facilities and preventive health network is a priority of ADB and the government and this should be clearly articulated to health sector leaders, provincial health departments, and planners formulating designs for any future ADB support in Viet Nam; (ii) resource allocation between provinces should be transparent and should be based on transparent indicators, such as poverty prevalence, mortality, population, or a combination of these and other factors using simple, weighted criteria; (iii) improved health sector planning is required and project resources should be targeted to localities and facilities that generate tangible results; (iv) assurances for counterpart funding should be expanded to encompass enabling factors, such as potable water, electricity, or road infrastructure being provided by local authorities to ensure that project investments can be built and can function properly upon completion; (v) provinces should have long-term training plans for health workers so there is sufficient, qualified health staff at all levels; (vi) educational background and sociocultural barriers of ethnic minorities must be addressed for them to fully benefit from project support; and (vii) monitoring and evaluation (M&E) should be conducted regularly throughout project implementation and should be geared to facilitate final evaluation. V. OTHER CONSIDERATIONS AND FOLLOW-UP A. Monitoring and Evaluation Design, Implementation, and Utilization 33. It is mandatory for every RRP and PCR to attach the project s DMF as Appendix 1, thereby underlining the importance of M&E during implementation, which was done in this project. During project implementation, ADB adequately conducted review missions (para. 27). The missions, particularly the midterm review mission, were crucial in providing guidance and finding solutions to problems or issues faced by the project that may have obstructed its implementation. 34. However, as stated in the PCR, minimal attention was paid to project outcomes in the DMF and indicators for the training programs outcomes were limited. The project s baseline data was never established. Data relating to achievements of project outcomes, such as improvements in staff capacity as a result of training, or utilization of project-developed facilities, were not regularly monitored and available for final evaluation. Given ADB s results-based approach to project management, both the MOH and ADB should have given a greater focus to M&E, and this should have been thoroughly conducted throughout project implementation. Overall, M&E design, implementation, and utilization were not adequate. B. Comments on Project Completion Report Quality 35. The PCR was prepared in line with Project Administration Instructions 6.07A guidelines of ADB. Its assessment of the project was comprehensive and included the necessary details. It appropriately analyzed the strength and weaknesses of project preparation and implementation. The PCR included assessments of the performance of ADB, the borrower, and the executing agency. It provided a list of useful lessons and recommendations for future projects. The PCR is commended for presenting sufficient quantitative and qualitative data, and a cost-effectiveness analysis of the project that calculated the change of DALYs in the project

10 area. It also used and compared data from the 2004, 2006, and 2008 Viet Nam household living standard surveys. This validation rates the overall PCR quality satisfactory. C. Data Sources for Validation 36. Data sources for this validation were derived from the (i) PCR; (ii) RRP; (iii) back-tooffice reports of the loan review mission in 2006, midterm review mission in 2007, and project completion mission in 2011; and (iv) country partnership strategy for Viet Nam. D. Recommendation for Independent Evaluation Department Follow-Up 37. No further follow-up action is recommended for independent evaluation or study.