Maldives: Information Technology Development Project

Validation Report Reference Number: PVR-272 Project Number: 34276 Loan Number: 1882 December 2013 Maldives: Information Technology Development Project Independent Evaluation Department

ABBREVIATIONS ADB Asian Development Bank BME benefit monitoring and evaluation EIRR economic internal rate of return ICT information and communication technology IT information technology MOFT Ministry of Finance and Treasury MOTC Ministry of Transport and Communications NCC National Computer Center NCIT National Center for Information Technology PCR project completion report NOTE In this report, $ refers to US dollars. Key Words adb, information and communication technology, ict, independent evaluation department, lessons, maldives, performance indicators, project completion report, it development The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

PROJECT BASIC DATA Project Number: 34276 PCR Circulation Date: Dec 2011 Loan Number: 1882 PCR Validation Date: Dec 2013 Project Name: Information Technology Development Project Country: Maldives Approved ($ million) Actual ($ million) Sector: Transport and ICT Total Project Costs: 12.00 12.73 ADB Financing: ($ million) ADF: 9.50 Loan: (SDR equivalent, million) 9.50 7.51 10.32 6.78 Borrower: 2.50 2.41 OCR: 0.00 Beneficiaries: 0.00 0.00 Others: 0.00 0.00 Cofinancier: Total Cofinancing: 0.00 0.00 Approval Date: 17 Dec 2001 Effectiveness Date: 26 Jul 2002 25 Jul 2002 Signing Date: 26 Apr 2002 Closing Date: 30 Jun 2005 31 Dec 2009 Project Officers: Location: From To D. S. Pyo H. Yamaguchi N. Zhang L. Weidner N. Zhang T. Nishimura V. Lisack Mar 2002 Aug 2003 Apr 2006 Sep 2006 Apr 2007 May 2008 Jul 2009 Jul 2003 Jul 2003 Mar 2006 Aug 2006 Mar 2007 Apr 2008 Dec 2009 Validators: Quality Reviewers: J. Supangco, Consultant F. D. De Guzman, Evaluation Officer, IED2 C. Kim, Principal Evaluation Specialist, IED2 R. Vasudevan Evaluation Specialist, IED1 Peer Reviewer: Director: E. Kwon, Principal Evaluation Specialist, IED2 H. Hettige, IED2 ADB = Asian Development Bank; ADF = Asian Development Fund; ICT = information and communication technology; IED1 = Independent Evaluation Department, Division 1; IED2 = Independent Evaluation Department, Division 2; OCR = ordinary capital resources; PCR = project completion report, SDR = special drawing right. A. Rationale I. PROJECT DESCRIPTION 1. During the project preparation stage, the population of the Maldives was estimated at 280,000, spread among the country s 200 inhabited islands. Of these, 24% or about 67,000 people lived in the capital island of Malé. Over half of the inhabited islands had a population of less than 1,000; 39% of the islands had a population ranging from 500 to 1,000; and only 2% of the islands had a population of over 5,000. The wide and uneven distribution of the population posed many challenges, including the high unit cost of providing social and economic services and infrastructure. In addition, the difficulties of access and the inherent fragility of the environment combined to create one of the most vulnerable communities in the world. 2. Migration from the outer islands into Malé was prevalent as people sought employment and better education. The Vulnerability and Poverty Assessment 1998 estimated per capita

2 household income for the atoll population at only Rf7300, equivalent to $647. 1 Up to 42% of Maldivians lived on less than Rf15 daily ($1.17), barely over the international poverty line of $1.0 a day, with 22% subsisting on Rf10 ($0.83). The scattered population resulted in infrastructure and service costs that were 4 5 times higher than normal, with a subsequent lack of access. Opportunities for pursuing even secondary school education were limited often, children must move from their home islands if they wish to study beyond the primary level. 2 Those outside of Malé are engaged in fishing, agriculture, and other primary industry-related activities for livelihood. Many of these families depend on remittances from family members employed in Malé or in the tourist resorts. The redistribution of economic activities from Malé to other islands was constrained by inadequate basic infrastructure and poor interisland transport. Because of its geographic structure, the Maldives particularly inhabitants in the outer islands who could gain affordable access to public services as those in Malé had the potential to benefit from a nationwide information and communication technology (ICT) infrastructure. 3. In the Maldives, high tariffs for internet and data transmission services were regarded as major obstacles for expanding internet penetration. With the given tariff structure, the government s efforts to bring the public closer to government information and public services through government networking were considered fruitless. Harnessing modern information technology (IT) for a wide range of electronic activities could expand the range of possible economic activities. Malé had a high rate of access to telecommunication although most inhabited outer islands have only one public telephone booth. Only 10 islands had full network facilities, and telecommunication charges in the Maldives were several times higher than in neighboring countries. The local telecommunication monopoly, while an example of a profitable public private partnership, did not spread access to the internet or other modern communication facilities. Making telecommunication more accessible and affordable in the atolls would facilitate the introduction of a wide range of possible electronic activities. 3 4. The government identified ICT as a prime tool to promote sustainable economic growth. Telecommunication sector reform was deemed an essential step that could come promote ICT. The project was needed to encourage the government to use ICT in its pursuit of making its work more efficient, transparent, and accountable, and for delivering better public services. 4 B. Expected Impact 5. The expected impact of the project in the framework was the improvement of telecommunication services by lowering tariffs. The identified performance indicator was internet tariff reduced to internationally comparable level (footnote 4). C. Objectives or Expected Outcome 6. The project s envisaged outcomes were improvement of efficiency, transparency, and accountability of public sector management. Identified performance indicators to measure the project s attainment of its objectives were (i) increased access to government portals by 10% per annum for 5 years after project completion, and (ii) increased ratio of government services delivered online compared with over-the-counter by 10% per annum for 5 years after project 1 Government of the Republic of the Maldives, Ministry of Planning and National Development and United Nations Development Programme. 1998. Vulnerability and Poverty Assessment 1998. Malé. 2 United Nations Development Programme. 2002. United Nations Development Assistance Framework for the Republic of Maldives, 2003 2007. Malé. 3 ADB. 2002. Country Strategy and Program: Republic of the Maldives, 2002 2004. Manila. 4 ADB. 2001. Report and Recommendation of the President to the Board of Directors: Proposed Loan to the Republic of the Maldives for the Information Technology Development Project. Manila.

3 completion. Other outcome targets were increased ratio of intergovernment agencies work carried out online by 10% per annum for 5 years after project completion; and growth in government employment by 1.5% for 5 years after project completion and 1.0% thereafter (footnote 4). D. Components and Outputs 7. At appraisal, the project was envisaged to produce four outputs. The first was the networking of government ministries and parastatal organizations in Malé accomplished by installing a fiber optic cable connecting 20 outer atolls relying on carrier service offerings and extended to other atolls in a phased manner, and establishing a transactional portal site for individual citizens and businesses, which would offer a range of services. The second was the establishment of the National Computer Center (NCC) to centrally coordinate ICT policies, standards, and practices for government work. The project was to provide the building to accommodate the NCC and house the facilities and equipment needed for networking government agencies and delivering public services through the web. The third was the provision of internet kiosks to give the public access to the government network. The project would provide computers, modems, furnishings, and workstations; appropriate accommodations to house the kiosks; and initial training for personnel (footnote 4). 8. The fourth output was the provision of consulting services for capacity strengthening of the regulatory authority and preparation of the telecommunication law. This was to ensure that reform measures were implemented during the agreed time. Reforms included separation of the existing regulatory body s policy-making and regulation functions, establishment of an independent regulatory body, enactment of a telecommunication law, opening up of markets to competition, and tariff restructuring for internet services. The project would also assist the government by providing consulting services for preparation of bid documents, bid evaluation for provision of networking and applications, and conducting awareness campaigns and external audit for project accounts. E. Provision of Inputs 9. At appraisal, the estimated project cost was $12.0 million equivalent, comprising $9.7 million or 81% foreign exchange cost and $2.3 million or 19% local currency cost. The estimate included physical and price contingencies, taxes and duties, and interest during construction. The total project cost at completion was $12.73 million, about $0.72 million (6.1%) higher than estimated at appraisal. This was because of the replacement of the applications contractor and implementation delays resulting in the cost at completion of $3.98 million, 59.0% higher than the $2.50 million estimated at appraisal. The cost of the internet kiosks increased by 22.1% because of the higher prices of materials. The costs for constructing the National Center for Information Technology (NCIT) building decreased by 27.6% and consulting services by 29.4%, because of government efforts to self-finance the NCIT construction and minimize consulting services. 5 10. At appraisal, the Asian Development Bank (ADB) was to provide an ADF loan of $9.5 million equivalent to finance 79.2% of project costs, comprising 95.9% foreign exchange costs and 8.7% local currency costs. The remaining costs were to be financed by the government. At completion, ADB s loan of $10.32 million equivalent was used to cover 81.1% of project costs, comprising 99.5% foreign exchange costs and 6.4% local currency costs. The government financed the remaining 18.9% ($2.41 million equivalent) of the project cost. 5 ADB. 2011. Completion Report: Information Technology Development Project in the Republic of the Maldives. Manila.

4 F. Implementation Arrangements 11. The project was implemented in accordance with the institutional arrangements envisaged at appraisal. The Ministry of Finance and Treasury (MOFT) was the executing agency with the Ministry of Transport and Communications (MOTC) as the implementing agency. A project steering committee, chaired by the MOTC, was established to direct the project and assist with interagency coordination. The NCIT acted as the project management unit with the NCIT head designated as project director. Consultants were contracted to assist in project implementation, design, and supervision. Due to the limited capacity of IT personnel at the MOTC, ADB granted permission to hire ICT experts to assist during the application-testing period. 12. The four sector covenants, the financial covenant, and seven other covenants were complied with. The performance evaluation covenant was partially complied with, as the MOFT and the MOTC did not provide any report on benefit monitoring and evaluation (BME). II. EVALUATION OF PERFORMANCE AND RATINGS A. Relevance of Design and Formulation 13. The project completion report (PCR) rated the project highly relevant and consistent with the government s development strategy and ADB s country partnership strategy. In addition, it noted that the intended project objectives of promoting regional development, creating a conducive environment for private sector development, and developing an information and communication network in the atolls, were highly relevant. The validation noted that these stated project objectives were not the same as those indicated in the project framework. 14. The validation reviewed various documents pertinent for the period. There were large regional disparities in incomes and access to and availability of essential services in the Maldives. The government s Sixth National Development Plan and Vision 2020 aimed to address these issues. ADB s country strategy and program shifted from the 1995 priority of addressing macroeconomic imbalances to promoting regional development aimed at equitable development among the atolls on one hand and promotion of economic growth for alternative centers to Malé on the other. It recognized the need to assist the government in strengthening the Maldives' economic sustainability. 15. ADB s operational strategy for the Maldives supported poverty reduction through a combination of pro-poor growth, social development, and good governance initiatives. To foster pro-poor growth, ADB assistance was directed, among others, to developing the region, including the provision of basic infrastructure and capacity building and developing an information and communication network in the atolls. To support good governance, ADB would improve information access and public sector connectivity, and build the capacity of key public institutions with the aim of improving public sector management efficiency (footnote 3). 16. The PCR noted that the project was the first of its kind for ADB in the ICT sector and in the Maldives. It recognized the complexity of the project components and the need to adequately consider the local situation, including sector development status, regulatory framework, and local capacity for implementing the project and absorbing the new technology. It also noted that the design and formulation of an ICT project should consider not only implementation, but also the operation phase. Overall, the design and formulation of the project was adequate. It took into consideration both hardware component and reform measures,

5 including tariff restructuring for internet services and setting up an appropriate regulatory framework. The validation rates the project highly relevant. B. Effectiveness in Achieving Project Outcome 17. The PCR rated the project highly effective in achieving its outcome. It noted that most of the objectives stated in the project framework were achieved. However, no data was provided to support this finding, as the NCC failed to collect and maintain data needed for evaluating project benefits. The MOFT and the MOTC did not provide the BME report as required (para. 12). The validation found this as a cause of the PCR s failure to measure project outcomes and impacts. Of the impact and four outcome performance indicators in the project framework, not even one was measured at project completion. On the outputs and outcomes, the PCR merely noted that these proved the project was important, timely, and effective in assisting telecommunication sector development in the Maldives. 18. At completion, the Government Network of Maldives, a fiber optic network interlinking government agencies in Malé and other islands, had been established. In Malé, 106 government offices were linked by this network, of which 79 were directly connected via fiber optic cable and 27 via asymmetric digital subscriber line. A network connecting 20 outer atolls was established with active connectivity to 4 via microwave technology provided by a local mobile operator. The NCC, renamed the NCIT, was established in 2003, as the main government agency for the development, promotion, and propagation of IT in the Maldives. The NCIT building was completed in December 2005 and was in full operation. A data center was established in the NCIT building, comprising the network, systems, database, and application infrastructure for interlinking all government agencies throughout the Maldives and for delivering public services. All aspects of the telecommunication sector reform were completed (footnote 5). 19. The validation concludes that since the expected outputs were actually produced, project outcomes could have been achieved. However, attainment of the target outcomes could not be fully ascertained in view of the insufficient discussion in the PCR, especially on the linkages between outputs and outcomes. Overall, the validation rates the project effective. C. Efficiency of Resource Use in Achieving Outcome and Outputs 20. The PCR rated the project efficient in achieving its outcomes and outputs. It used the same methodology as at appraisal and updated data to reevaluate the economic internal rate of return (EIRR). The economic reevaluation compared the economic costs and benefits of with and without project cases. Capital costs and operation and maintenance costs adjusted to economic prices were used in the reevaluation. 21. The validation notes that at appraisal, the benefits identified included those from the application for the national citizen identification card, hospital information system, vessel, vehicle and aircraft registration application, and use of the internet kiosks. A different set of benefits was measured in the PCR, comprising (i) time cost savings by avoiding the handdelivery of documents among government agencies, (ii) transport cost savings through delivery of documents via the internet, (iii) paper cost savings by exchanging documents through the internet, and (iv) other cost savings that are the unquantifiable benefits assumed to be 30% of the total savings generated through (i) to (iii) above. The validation finds the methodology used at appraisal and project completion appropriate. 22. Project costs were higher than at appraisal. To be implemented over 2 years, the project took 7 years (or 5 years delayed) compared to the schedule at appraisal. The PCR attributed

6 the delays to unrealistic planning at appraisal, the MOTC s lack of familiarity with ADB procedures, the complexity of the project, natural disasters such as the Asian tsunami of 2004, and geographic disparities. It noted that delays occurred at all stages of implementation. In spite of these, the physical and soft works were substantially completed. 23. Nonetheless, the validation also rates the project efficient in terms of resource use in achieving outcomes and outputs, based on the reevaluated EIRR by project completion, which was higher than the 12% benchmark rate. The sensitivity analysis showed that the project was viable for all scenarios, with the worst case resulting in an EIRR of 13.6%. D. Preliminary Assessment of Sustainability 24. The PCR rated the sustainability of the project likely. The PCR noted that (i) a general sector development foundation had been established, although detailed sector reform policies were to be implemented; (ii) the institutional framework had been established with clear responsibilities, although human resource still remained a key challenge; (iii) the physical infrastructure of the government network was in place though not fully used; (iv) some applications developed under the project were not functional and needed to be revised or redeveloped; and (v) system operations and management needed to be provided with an adequate budget (footnote 5). 25. The validation notes that the government had complied with the loan covenants requiring adequate budgetary resources to be provided to NCIT for project operations and maintenance. However, there was no certainty that the budgetary allocation would continue in the future. Since there seems to be no user charges for either the government or private sector users of the system, it is unlikely that the government would be able to sustain said facilities. In fact, the NCIT was already facing budgetary constraints in using the network and in strengthening applications developed under the project. For example, only 4 out of the 20 connected atolls have active connectivity because of a lack of operating budget. Yet, the NCIT was planning to develop new applications to further expand delivery of government services to citizens through ICT. The validation rates the project s sustainability less than likely. E. Impact 26. At appraisal, the initial environmental examination carried out determined that (i) the project did not have significant negative environmental impacts, (ii) the need for an environmental impact assessment did not arise, and (iii) monitoring and mitigation measures for the limited impacts identified were sufficient. During project implementation, no issues relating to environmental and sociocultural impacts were reported to ADB. The PCR did not rate the impact of the project. 27. The validation notes that at appraisal, the envisaged social impacts of the project were identified. First, it was envisaged that the introduction of the internet to island communities would augment and complement economic growth through cost-effective and efficient communication and information exchange. Second, the country s young population was ideally situated to take advantage of the new opportunities arising from such technological options, especially as computer literacy formed an important goal of the education policy framework. Third, the wider introduction of internet services to remote island communities would serve to break the barrier of distance, which was the single most important constraint to achieving balanced and sustainable national development. Fourth, greater access to public services such as licensing and registration would be beneficial to the community at large and likely to increase the overall efficiency of the island economies through reduction in the costs of business

7 development, and the ensuing multiplier effects through greater employment and incomeearning opportunities. Fifth, the availability of timely and efficient services would increase the people s confidence in the public service system (footnote 4). In view of the absence of monitoring reports, the links between attained outputs, expected outcomes, and envisaged impact were weak (paras. 17 and 19). 28. On gender, participation of the female labor force had declined from over 60% 30 years ago to less than half. The liberalization of the telecommunication market and the associated reduction in costs and the provision of public services through the internet would likely create new windows of opportunity for women in the outer islands, especially the younger generation of women, who were completing their schooling and looking for employment in the modern sectors. The overall improvement in communication services would open up the scope and arena of work for women and create greater opportunities for them to work from home, especially in the outer islands where employment opportunities were otherwise limited. Their traditional dependence and reliance on men to help them obtain public services would give way to independence and self-reliance as they can directly access public services without traveling to Malé to obtain licenses and other requirements to begin business ventures or obtain social services leading to greater empowerment, self-confidence, and the ability to take on leadership roles. Overall improvement in communications would provide greater incentives for women to participate in economic activities and enhance their social position (footnote 4). 29. While the results of a BME activity would have provided socioeconomic information on the impacts of the project and there being none, the appraisal s envisaged socioeconomic impacts and the completion and operation of the project outputs led the validation to conclude that the project had moderate impact. III. OTHER PERFORMANCE ASSESSMENTS A. Performance of the Borrower and Executing Agency 30. The PCR rated the performance of the borrower and the executing agency satisfactory. The borrower was the Government of the Maldives, the executing agency was the MOFT, the implementing agency was the MOTC, and the NCIT served as the project management unit. It noted that the government provided counterpart funds and all necessary support in a timely manner during implementation. The MOTC through the NCIT was responsible for day-to-day project management and facilitated the success of project implementation. It was adequately staffed, proactive, and speedy in resolving implementation issues. It had difficulty in meeting the initial project schedule because of the procedural complexities of the two-stage, two-envelope bidding process, and ADB s approval process. The borrower complied with most of the loan conditions and covenants. 31. Delays occurred at all stages of implementation, including planning, design, procurement, deployment, construction, installation, and commissioning. In several instances, the MOTC had difficulties with contractors who (i) failed to comply with their contract obligations; (ii) showed slow work progress; (iii) focused more on reducing their scope of work; or (iv) had improper work quality. The executing and implementing agencies also failed to provide a BME report (para. 12). Overall, the validation rates the performance of the borrower and executing agency satisfactory.

8 B. Performance of the Asian Development Bank 32. The PCR rated the performance of ADB satisfactory. It noted that the project was administered and supervised from. ADB provided substantial guidance to the government, the MOFT, and the MOTC, in the implementation of the project, in preparing documents and evaluation reports related to procurement, in engagement of consultants, and on technical issues. During implementation, ADB carried out 11 review missions, including the midterm review mission in 2008. ADB was closely involved in identifying potential problems and critical activities, and in resolving issues related to the implementation of the project. The MOFT and the MOTC recognized the role of the ADB missions in advising on contract administration and technical issues. 33. The validation notes that ADB missions had not followed up on the performance monitoring and evaluation of the project, a loan covenant. The ADB mission of 12 July 2007 noted that all major loan covenants were complied with, when in fact the MOFT and the MOTC had failed to submit the BME report to ADB. The validation also notes that the project had seven project officers during implementation, too many changes for one project. The validation also rates ADB performance as satisfactory. C. Others 34. The project did not require any resettlement or acquisition of right-of-way. At appraisal, other than temporary disturbances during construction, the project was not anticipated to have any medium- to long-term environmental impacts. IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS A. Overall Assessment and Ratings 35. Both the PCR and the validation ratings for the various major criteria resulted in an overall successful rating for the project. On relevance, the project was consistent with the government s development strategy and ADB s country partnership strategy. Project and design formulation were adequate. On effectiveness, expected outputs were actually produced. However, the project did not have a BME, which could have indicated whether outcome performance targets were achieved by the project. On sustainability, there seemed to be an absence of user fees to fund the project s operating and maintenance requirements, which relied on the government s budget allocation. This was an uncertain funding source as many sectors vie for the government s meager financial resources. On impact, the PCR did not give any rating and its discussion on the project s impact was focused on the environment. The validation gives a rating of moderate for impact based on the socioeconomic and gender impacts envisaged at appraisal. The comparative overall ratings are given below. Overall Ratings Criteria PCR IED Review Relevance Highly relevant Highly relevant Reason for Disagreement and/or Comments Effectiveness in Achieving outcome Highly effective Effective It could not be fully ascertained that the outcome targets were attained in view of insufficient discussion in the PCR, especially on the linkages between outputs and outcomes.

9 Criteria PCR IED Review Efficiency in achieving outcome and outputs Preliminary assessment of sustainability Efficient Likely Efficient Reason for Disagreement and/or Comments Nonetheless, envisaged outputs were achieved (para. 19). Less than likely The government had complied with the loan covenant to provide budgetary allocation for the project s maintenance. Since there seemed to be no user charges to provide a steady stream of funding, financing for the project s operations and maintenance does not seem to be reliable (para. 25). Overall assessment Successful Successful Borrower and executing Satisfactory Satisfactory agency Performance of ADB Satisfactory Satisfactory Impact Not rated Moderate Results of a BME activity would have provided socioeconomic information on the impacts of the project. In the absence of a BME, impact appears to be moderate, based on socioeconomic and gender impacts (para. 29). Quality of PCR Less than satisfactory The PCR did not provide sufficient discussion to justify the ratings on relevance, effectiveness, and sustainability (para. 40). ADB = Asian Development Bank, BME = benefit monitoring and evaluation, IED = Independent Evaluation Department, PCR = project completion report. Note: From May 2012, IED views the PCR's rating terminology of "partly" or "less" as equivalent to "less than" and uses this terminology for its own rating categories to improve clarity. Source: ADB Independent Evaluation Department. B. Lessons 36. The PCR drew out valuable lessons. Projects involving ICT tend to be complex and require more scrutiny. These projects could not be treated the same way as other infrastructure projects such as roads, ports, or airports where a large amount of expertise is already available. Project design would require knowledge and skills that are abundantly available only in the private sector. The regulatory environment for ICT projects is technical and requires knowledge on a mix of legal, technical, and economic fields. Project design should adequately consider the local situation, including sector development status, regulatory framework, and local capacity for implementing the project and absorbing the new technology. Project implementation could have involved external consultants, contractors, and local experts to avoid transitional issues. The validation finds these lessons appropriate. C. Recommendations for Follow-Up 37. ADB needed to follow up with the government on the conduct and preparation of the project s BME surveys and reports, which had not been fully complied with. This included conducting surveys on the use of the system as provided in the outcome performance

10 indicators and targets. The measurement of the project s socioeconomic and gender impacts needed to be included in the collection of data. 38. The government needed to provide adequate budget to operate and maintain the system. Already, lack of financial resources has been observed, with the NCIT facing budget constraints relating to the use of the network and strengthening of applications developed under the project. Only 4 out of the 20 connected atolls had active connectivity because of a lack of operating budget (para. 25). ADB needs to ensure that facilities provided are well maintained and operational and that required financial resources are provided regularly and on time to ensure sustainability. V. OTHER CONSIDERATIONS AND FOLLOW-UP A. Monitoring and Evaluation Design, Implementation, and Utilization 39. The validation notes that the importance of project framework was not appreciated and its use was not evident in the preparation of the PCR. The PCR had referred to objectives or outcomes, which were not in the appraisal and PCR project framework (paras. 6 and 13). The MOTC, the NCIT, and the consultants failed to submit the BME report, which could have been used in PCR preparation. While the ADB midterm review mission of 5 6 August 2008 evaluated the status of the project using the main evaluation criteria of relevance, effectiveness, efficiency, and sustainability, it had not cited any issue with regard to availability of BME reports. B. Comments on Project Completion Report Quality 40. The validation rates PCR quality as less than satisfactory. The reasons for PCR ratings were not discussed thoroughly. It does not explain the assessment fully. The PCR made many general observations not supported by data. There were several inconsistencies. For example, the claim that the methodology used in the reevaluation of the EIRR at project completion was the same as that during appraisal, yet a comparison of the two did not substantiate the assertion. C. Data Sources for Validation 41. The data sources used for the validation were mainly the project s report and recommendation of the President, the PCR, and review mission reports. Aside from project documents, ADB s country strategies for the Maldives, evaluation reports, government development plans, and United Nations publications were also used. D. Recommendation for Independent Evaluation Department Follow-Up 42. Given the poor quality of the PCR, the validation recommends the conduct of a project performance evaluation with emphasis on measuring the socioeconomic impact of the project. As a prelude to the impact evaluation, ADB needs to follow up on the BME and on the government s continued monitoring of project performance indicators as required under the loan covenant. This project could also be a case for a thematic evaluation study for ICT in the future.