Validation Report. Indonesia: Coral Reef Rehabilitation and Management Project Phase II. Independent Evaluation Department

Transcription

1 Validation Report Reference Number: PVR-324 Project Number: Loan Number: 1962 September 2014 Indonesia: Coral Reef Rehabilitation and Management Project Phase II Independent Evaluation Department

2 ABBREVIATIONS ADB Asian Development Bank BME benefit monitoring and evaluation CRITC coral reef information and training center CRM coral reef management CRMP coral reef management plan EIRR economic internal rate of return ha hectare HRD human resource development LIPI Lembaga Ilmu Pengetahuan Indonesia (Indonesian Institute of Sciences) MMAF Ministry of Marine Affairs and Fisheries NGO nongovernment organization PCR project completion report PIU project implementation unit PMO project management office NOTE In this report, $ refers to US dollars. Key Words asian development bank, artisan fisheries, associated ecosystems, benefit transfer method, catch per unit effort, coral reef, critical reef ecosystem, indonesia, marine management areas, monitoring, control and surveillance, reef management, social infrastructure facilities, sustainable natural resource management The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

3 PROJECT BASIC DATA Project Number: PCR Circulation Date: Nov 2012 Loan Number: 1962 PCR Validation Date: Sep 2014 Project Name: Coral Reef Rehabilitation and Management Project, Phase II Country: Indonesia Approved ($ million) Actual ($ million) Sector: Agriculture and Total Project Costs: natural resources ADB Financing: ($ million) ADF: Loan: (SDR equivalent, million) Borrower: OCR: 0.00 Beneficiaries: Others: Cofinancier: Total Cofinancing: Approval Date: 6 Dec 2002 Effectiveness Date: 11 May Nov 2003 Signing Date: 10 Feb 2003 Closing Date: 31 Dec Dec 2011 Project Officers: Location: From: To: W.D. Zhou M.N. Islam ADB headquarters ADB headquarters Dec 2002 Oct 2005 Sep 2005 Dec 2011 Validator: E. Lopez-Dee, Consultant Peer Reviewer: K. Hewitt, Evaluation Specialist, IED2 Quality Reviewers: E. Gozali, Principal Evaluation Specialist, IED1 M. Grace Agapito, Evaluation Officer, IED1 Director: W. Kolkma, Director, IED1 ADB = Asian Development Bank; ADF = Asian Development Fund; IED1 = Independent Evaluation Department, Division 1; IED2 = Independent Evaluation Department, Division 2; OCR = ordinary capital resources; PCR = project completion report; SDR = special drawing rights. A. Rationale I. PROJECT DESCRIPTION 1. Indonesia s large expanse of marine areas, much of which are fringed by coral reefs, support a wide variety of living aquatic resources. Considered as the coral biodiversity center of the world, its coral reef resources are said to contain 70 genera and 450 species of corals. These resources in turn support many coastal communities, artisans, and industrial fisheries. When this project was being contemplated, the total product of coastal and marine economic activities was estimated at 25% 30% of the country s gross domestic product. Of late, coral reefs have been depleted rapidly mainly due to extensive extractive activities, destructive fishing practices, wanton coastal development for industrial and tourism purposes, and pollution. Since coral reefs provide 90% of fish catch by coastal fishermen who are also increasing in number and who belong to the poorest segment of the society, their rapid deterioration proportionately reduces the fish catch per unit effort. These not only adversely affect the fishermen s incomes; but also further constrain their access to basic social infrastructure like water, sanitation, health care, transport, and even credit. Such is the importance of these reef resources to Indonesia s environment and economy that its planning system has now put stronger emphasis on using these resources with sustainability as its primary concern.

4 2 B. Expected Impacts 2. The project s desired impacts and corresponding performance indicators as defined in its design and monitoring framework were twofold: 1 (i) coral reefs and associated ecosystems rehabilitated, protected, and well managed as evidenced by an annual increase of 2% in live coral cover; and (ii) incidence of poverty in coastal communities lowered with average real per capita income in target coastal communities annually increased by 2%. C. Objectives or Expected Outcomes 3. The projected outcomes and their corresponding performance targets included the following: (i) enhanced national and local capacity to manage coral reef resources and effectively managed priority reef areas as measured by the main coral reef management (CRM) issues in six districts in three provinces being effectively addressed; and (ii) increased income and improved living standards of poor coastal communities of 10,000 households, which at completion should have a 20% growth in real income over the project s life and their living standards subsequently improved. D. Outputs 4. The expected impact and outcomes were to be achieved through the following components: (i) institutional strengthening and project management, and (ii) community-based resource management and development. The first component was to be accomplished by the development of policies, strategies, and guidelines for sustainable CRM at the national and at six district-government levels, leading to the (i) drafting and implementation of CRM strategies; (ii) preparation of their respective coral reef management plans (CRMPs); (iii) delineation of marine management areas in three to six districts; and (iv) formation of community-based monitoring, control, and surveillance units. Another initiative planned was the establishment of a centrally and regionally operated network of coral reef information and training centers (CRITCs) augmented by a functioning benefit monitoring and evaluation (BME) system, and a coral reef research program. Moreover, soft subcomponents on human resource development (HRD) and comprehensive information, education, and communication program were also prescribed at appraisal. The project was mandated to establish and operate a project management office (PMO), a project implementation unit (PIU) in the Indonesian Institute of Sciences (LIPI), and six regional PIUs; procure vehicles and equipment, minor civil works, and consulting services; award contracts for community organizing, training, and surveys; and conduct workshops, seminars, and study tours. 5. Initiatives under the second component were to further the efficacy of coastal community groups purposely organized for CRM and livelihood development in accordance with their respective CRMPs; achieve community-based compliance with and enforcement of sound CRM practices; and provide social infrastructure facilities and services to target beneficiaries. By the end of the project, it was expected that illegal fishing in the project sites would have been reduced by 40% 50%. E. Provision of Inputs 6. The Asian Development Bank (ADB) extended a loan of $33.0 million to finance both foreign and local currency cost components of the project. Disbursements reached $36.7 million after one loan reallocation and a 2-year loan closing date extension. The government s actual 1 ADB Report and Recommendation of the President to the Board of Directors: Proposed Loan to the Republic of Indonesia for the Coral Reef Rehabilitation and Management Project Phase II. Manila.

5 3 contribution amounted to $0.3 million compared to the $6.6 million estimated at appraisal while beneficiaries contribution amounted to $0.6 million compared to the $1.7 million estimated at appraisal. Project expenditures were $37.7 million, 8.8% lower than the appraisal estimate of $41.3 million as the rupiah depreciated against the dollar. Inputs provided were to fund institutional strengthening and project management at $19.8 million, up slightly from $17.53 million at appraisal; and community-based resource management and development at $17.1 million, down from $20.8 million largely because of curtailed contributions from the beneficiaries and lower government counterpart funds. Actual financing by ADB went up by 10.2%, which is equivalent to $36.7 million. The increment was allocated for tsunami assistance, which was not in the original financing plan. 7. Consultancy services were procured in accordance with ADB guidelines although the total amount of $3.9 million exceeded the appraisal estimate of $3.5 million. A total of person-months of consultancy services were used up for the PMO and LIPI project management functions and CRITC operations, comprising person-months of national and 54.0 for international consultants. These exceeded the appraisal requirement by personmonths, which were essentially utilized for the project s 2-year extension. The procurement of goods and services was also carried out consistent with established ADB procedures and expedited by a dedicated procurement committee at the PMO. Village social infrastructure facilities were built with beneficiaries providing labor, local materials, and other in-kind contributions. F. Implementation Arrangements 8. At appraisal, the Ministry of Marine Affairs and Fisheries (MMAF), through its Directorate General of Marine, Coastal and Small Islands, was proposed to be the executing agency having overall project jurisdiction. Within the directorate was the PMO, which was entrusted with the day-to-day project management. In each project province, there were the local PMOs called PIUs; and at the district level, the regional coordination units. Mandated to provide overall coordinative functions at the national level was the National Steering Committee chaired by the National Development Planning Agency (or the BAPPENAS), and at the local level was the Regional Advisory Council chaired by the Provincial Fisheries and Marine Affairs Service. The project s research information, monitoring, and training activities were solely assigned to the LIPI. These arrangements at appraisal were substantively adhered to throughout the project except for one modification. Less than 2 years after loan effectiveness, a number of earthquakeand tsunami-affected fishing villages were added to benefit from the project s multifaceted interventions. 9. Even with this change in scope and geographical coverage, which eventually required reprogramming and reallocation of project resources, the schemes proposed and followed were found to be sufficient in delivering all expected outputs. The project completion report (PCR) 2 further reported that the overall performance of consultants, civil works contractors, equipment suppliers, and nongovernment organizations (NGOs) was satisfactory despite start-up delays. A status quo in the conditions and covenants stipulated in the loan agreement was followed. Of the 58 covenants, 50 were fully and promptly complied with while compliance with four was delayed. Another four, which related to the required number of meetings of field implementation units, and community funds for the maintenance of social infrastructure, were only partially complied with. No project activities were required in relation to the ADB involuntary resettlement policy. 2 ADB Completion Report: Coral Rehabilitation and Management Project Phase II in the Republic of Indonesia. Manila.

6 4 II. EVALUATION OF PERFORMANCE AND RATINGS A. Relevance of Design and Formulation 10. The PCR rated the project highly relevant because it was designed to address the policy, legal, and institutional bottlenecks affecting resource management in the country s reef areas. It adopted a holistic approach to CRM and provided the poor coastal communities with basic social services, social infrastructure, and opportunities to augment their incomes from microenterprises to balance environmental and resource management with socioeconomic development. It placed strong emphasis on the active involvement of intended beneficiaries, local governments, and NGOs in planning and implementing CRM interventions. It continued to be relevant even after its completion as sustained poverty alleviation remained a primary goal in Indonesia s Medium Term Development Program for During and after completion, the project remained in line with the country strategy and program of ADB, which had an overarching poverty reduction objective including environmental management and protection. 11. Expanding its coverage to include tsunami-affected villages in other districts of the country further underscored its relevance. This validation supports the innovative value-adding features and ambition of the project. This validation, therefore, accords the project a highly relevant rating. B. Effectiveness in Achieving Project Outcome and Outputs 12. The outputs under the first component were generally met. As of loan closing, seven of eight districts had formulated their own regulations for CRM. Seven CRMPs instead of six were also prepared, approved at provincial and district levels, and finally integrated into their regional development plans to ensure sustainable implementation and financing. Complementing these outputs were the seven local CRITCs, fully established and equipped to be able to network with the national; and eight regional CRITCs. The project supported this initiative further by organizing a coral reef spatial information system and installing an expanded BME for CRM. Coral reef biomass, biodiversity, and fishing effort data have since been regularly uploaded into the national CRITC server. The project s national and local coral research agenda from 2005 to 2010 yielded 2 national and 21 local research activities. 13. Through its comprehensive HRD program, a total of 2,860 government and civil society personnel were trained on CRM, easily exceeding the targets by more than 50%. Graduate scholarships were also provided to 15 regional field and 10 central office personnel. Comprehensive extension programs on the sustainable use of coral reef resources and livelihood activities were developed and implemented in 57 coastal villages, 12 more than what was planned. The project managed to log a 91% awareness rating among public employees and coastal communities, exceeding the 70% pegged at appraisal. The PMO, national PIU-LIPI, regional coordination units, and PIUs were created, and later provided with trained staff and other necessary resources to function effectively. Also consistent with appraisal targets, consultants were deployed to provide technical assistance (TA). 14. The second targeted outcome of increasing income and improving the living standards of poor coastal communities was likewise achieved. At completion, it was reported that 769 community groups with 7,014 members in 57 coastal villages, compared with the target of groups comprising 10,000 members, were organized and participated in formal CRM training and livelihood development. Women accounted for 28% of total membership.

7 5 Community organizing also yielded 17 site-specific coral reef resources surveys as inputs to CRMP preparation, compared to the target of 10. District executives also designated seven marine management areas with an aggregate area of 130,000 hectares (ha) of coral reefs and associated ecosystems equivalent to 117% of the target. These areas were subsequently patrolled and protected by monitoring, control, and surveillance groups with about 2,000 members and complemented by 20 civilian employees deputized to monitor illegal and destructive fishing activities. As a result, surveyed project sites in 2011 showed a 71% decrease in the frequency of practices destructive to coral reefs, exceeding the target of a 40% 50% reduction in illegal fishing activities. Based on socioeconomic surveys conducted in 44 villages (in the eight project districts) out of the villages targeted, 298 social infrastructure facilities (e.g, village information centers, patrol boats, water supply, sanitation, and jetty wharves) were provided in all 57 project villages to improve the villagers living standards. The PCR reported that each type of facility had a minimum of 4,000 households utilizing it. 15. At completion, 879 village groups with 6,850 members were working in non-fisheriesrelated (e.g., embroidery, handicraft making, etc.) and fisheries-related (e.g., seaweed and grouper cage culture) microenterprises. About 50% of these microenterprises went financially well with the PCR attributing their success to solid cooperation among members, good technical skills, good management, and transparent bookkeeping. Socioeconomic data from 2005 to 2011 revealed that the overall average increase in household income throughout the project sites was 12.6%. While lower than the target of 20%, this validation believes that the project still made a significant difference in uplifting the income status of intended beneficiaries. 16. The PCR rated the project effective in achieving its intended outcomes. The targeted overarching outcome of rehabilitating, protecting, and managing coral reefs and associated ecosystems in the project sites was achieved. The various plans, policies, strategies, and regulatory measures that the project helped formulate enhanced national and local capacity to manage these resources and other priority reef areas. It effectively addressed the two key CRM issues on dynamite fishing and coral reef mining. Although unable to squarely meet the prescribed target, the project still significantly contributed in increasing fishing household income by an average of 12.6%, thus, also improving the living standards of these same groups of beneficiaries. Thus, this validation concurs with the PCR rating of effective for the project. C. Efficiency of Resource Use in Achieving Outcome and Outputs 17. The PCR rated the project efficient. It encountered start-up implementation delays as loan effectiveness, the signing of grant agreements between the MMAF and the district governments, and the procurement of consultant and NGO services took place beyond the prescribed timelines. Yet the project was able to catch up and eventually accomplished its targets with timely adjustments and improvements in management and operations. While this necessitated a reprogramming of project resources and extending the loan closing date, the project stayed on track to meet its targets. The extension was aptly utilized to allow the project s interventions to develop and mature, and to further build the capacity of district level institutions to ensure sustainability. The PCR recalculated the economic internal rate of return (EIRR) based on the actual area coverage of 132,950 hectare (ha) at 29.3% against the appraisal estimate of 19%. The two EIRR estimates were based on two different coverage areas. The appraisal estimate was based on 7,625 ha of sample project area while the PCR estimate was based on a total project area of 132,000 ha. Aggregate efficiency analysis used the benefit transfer method. This validation also notes the (i) proper recognition of economic costs, (ii) proper application of a manufactures unit value index for constant pricing, and (iii) appropriate use of shadow pricing parameters. The estimation of economic benefits both

8 6 at appraisal and completion relied mainly on the transfer benefits method. This validation notes that significant effort has been made to find accurate estimates of per ha benefits from coral reef and mangrove improvements. Nonetheless, the accuracy of these figures is difficult to ascertain. Given the relatively robust EIRR results and the fact that the project did not suffer from cost overrun, this validation concurs with the PCR rating of efficient for the project. D. Preliminary Assessment of Sustainability 18. The PCR rated the project likely sustainable. Fishing households group mobilization and management; participatory planning in the crafting, implementation, and monitoring of local regulatory decrees and social infrastructure facilities; and access to information and technology transfers through different media format were facilitated through the project. These could not have occurred without the comprehensive HRD activities at various levels, which the government and civil society partners have undertaken. These activities paved the way for a smooth collaboration between these entities and the villagers, which at completion remained robust. Socioeconomic surveys showed that (i) the public sector and coastal communities gained heightened awareness on the importance of coral resources, (ii) illegal practices were curtailed, (iii) village level infrastructures were built with each type being utilized by at least 4,000 households and some of them already being maintained through volunteerism, (iv) 50% of microenterprises put up were operating financially well with representative microenterprises posting financial internal rates of return ranging from 20% to 107% against the weighted average cost of capital of 16%, and (v) household incomes and living standards of villagers improved. Women s participation in various skills training and technology transfer activities, where they accounted for 24% of the total number of participants, enabled them to gain confidence and thereafter play important roles in home-based livelihood activities. They became recognized for their contributions in running the affairs of their community groups roles that were traditionally reserved for their male counterparts. Finally, a TA funded by ADB was underway to develop a subsequent initiative designed to sustain, strengthen, and institutionalize the interventions of the project. Hence, this validation subscribes to the likely sustainable rating in the PCR. E. Impact 19. The PCR did not provide a rating for the project s overall impact. Still, the project made a significant difference in coral reef protection and management, and in uplifting the income status and improving the living standards of intended beneficiaries. From various BME and coral reef biomass, biodiversity, and fishing effort reports, the overall average annual growth in live coral in all project sites was estimated at 9.4%, indicating that the appraisal target was achieved. Based on socioeconomic data obtained, the average growth of per capita income during reached 5.3% per year compared to the 2% targeted at appraisal. This positive increment in per capita income could be seen from the increasing trends in the catch per unit effort of fishermen, consistent with the observed improvements in live coral cover. Still, this validation opines that the positive changes that unfolded should not exclusively be attributed to the project. It should be noted that the project benefited from an earlier similar initiative that laid much of the required policy, institutional, and operational groundwork. Following all technical and socioeconomic data obtained, project impact appears to be significant. 20. The PCR indicated that project activities had minimal impact on the environment with no resettlement-related issue. Measures to safeguard the interests of affected indigenous peoples were implemented in accordance with the ADB Indigenous Peoples Policy.

9 7 III. OTHER PERFORMANCE ASSESSMENTS A. Performance of the Borrower and Executing Agency 21. The PCR rated the performance of the borrower and the executing agency satisfactory despite being constrained by start-up delays. The executing agency demonstrated good leadership and strong institutional capacity in implementing the project, thus, earning a satisfactory rating at closing date. The MMAF consistently met the disbursement targets so that the PMO and the PIUs could aptly perform their respective mandates in accordance with ADB procedures. The project performance monitoring system developed for the project turned in timely financial performance and physical progress monitoring reports and, hence, was able to keep track of how it was proceeding. External audit reports and third-party review reports of the contracted NGOs, researchers, and implementing entities at the village level helped ensure that assignments were carried out effectively. This validation supports a satisfactory rating for the performance of the borrower and the executing agency. B. Performance of the Asian Development Bank 22. The administration of the project by ADB was satisfactory. It carried out 1 inception mission, nine review missions, one midterm review mission, and one PCR mission. These missions recommended time-bound action plans that effectively helped the executing agency resolve implementation issues. ADB promptly attended to government requests for modifications in project coverage, cost sharing, consultancy services contract variations, and extension of the loan closing date. This validation agrees to the satisfactory rating accorded to ADB. IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS A. Overall Assessment and Ratings 23. The PCR rated the project successful. The project was highly relevant to the strategies and policies of both the government and ADB for enhanced capacity to sustainably manage and protect critical reef ecosystems and reducing poverty in the country. Its completion has already created a demand for its replication. While the project agonized over initial implementation problems, it still managed to achieve most of the intended targets. Economic benefits in terms of improved income generation evidently accrued to targeted fishing households. Women s status and participation in village organizations were enhanced. The institutions entrusted with natural resources management and regulation became more proficient through the capacity-building activities availed of by their technical staff, better networking facilities and resources, and the assistance provided by NGOs and extension workers. As a result, better and closer collaboration and coordination among implementing agencies, fishing communities, and other stakeholders emerged. At completion, favorable environmental consequences were also observed in terms of lower incidence of illegal fishing activities, improved fish catch, and increased live coral cover in some of the project sites. Overall, this validation rates the project successful.

10 8 Overall Ratings Criteria PCR IED Review Reason for Disagreement and/or Comments Relevance Highly relevant Highly relevant This validation supports innovative value-adding features and goals of the project (para. 11). Effectiveness in Effective Effective achieving project outcome and outputs Efficiency in achieving outcome and outputs Efficient Efficient Preliminary assessment of sustainability Likely sustainable Likely sustainable Overall assessment Successful Successful Impact Not rated Significant Refer to paras Borrower and executing Satisfactory Satisfactory agency Performance of ADB Satisfactory Satisfactory Quality of PCR Satisfactory Refer to para. 27. ADB = Asian Development Bank, IED = Independent Evaluation Department, PCR = project completion report. Source: ADB Independent Evaluation Department. B. Lessons 24. The PCR identified lessons from the implementation of the project that could be valuable to both ADB operations and to the borrower. This validation concurs with the PCR lessons, which essentially serve to reaffirm the benefits of a participatory approach to resource management. Key lessons are as follows: (i) mobilization and organization of coastal community groups and their subsequent participation in planning social infrastructure facilities did help foster a favorable climate for partnership among citizens, national and district governments, and the NGOs; (ii) coordination of implementation arrangements among enforcement and/or regulatory institutions and communities, and the comprehensive awareness campaigns ensured the acceptance of various CRM tasks entrusted to the communities; (iii) empowerment of women created numerous opportunities for them to be part of the decision-making processes and substantively contributed to improving their household income; and (iv) labor and local materials contributed by the fishing households for village level infrastructures or facilities instilled ownership and volunteerism that are necessary for the subprojects sustainability, and lowered investment costs. C. Recommendations for Follow-Up 25. This validation subscribes to the recommendations detailed in the PCR. The capacities of the national and district level institutions, that are charged with the protection and conservation of reef resources and associated ecosystems, and that of the target beneficiaries to organize themselves so they could participate in planning village level investments had been established. Government assistance is needed to further increase the capacity of fishing households to freely access credit, marketing support, and other TA to reduce business failures and optimize returns on community groups investments in microenterprises.

11 9 V. OTHER CONSIDERATIONS AND FOLLOW-UP A. Monitoring and Evaluation Design, Implementation, and Utilization 26. The project design required the establishment of the project performance monitoring system in the PMO. Although the system was established behind schedule, financial performance and physical progress monitoring were regularly undertaken. Baseline data were collected for reef health and socioeconomic surveys. The project carried out a midterm evaluation survey to gauge public awareness, reduction of destructive fishing activities, and extent of live coral cover. Participatory monitoring and evaluation at the community level to ensure that assignments were carried out effectively was also undertaken by the project, along with special studies to investigate the impacts of project implementation. Accordingly, BME indicators were developed and incorporated in the regular reporting to track down project progress. B. Comments on Project Completion Report Quality 27. This validation rates the quality of the PCR satisfactory. The PCR is consistent with Project Administration Instruction 6.07 where it made a systematic and clear assessment of the project s outcomes, impact, relevance, effectiveness, and efficiency. The lessons and recommendations given are well grounded since they were drawn from the specific performances of the different stakeholders and factual evidences gathered from the field. The report presented direct and detailed analysis of the issues, outputs, and outcomes. The design and monitoring framework is clear in presenting the appraisal targets compared with actual performance, enhanced with brief analysis of results. The financial and economic analysis of the microenterprises supported by the project in Appendix 19 was, however, confined only to the most predominant activities and/or components and, hence, could not provide an aggregate quantitative assessment of the project component s viability. However, the assumptions, project parameters, and methodology used in evaluating the project s overall efficiency were observed to be appropriate and up to standards. C. Data Sources for Validation 28. This validation made use of the report and recommendation of the President, PCR, and back-to-office reports. D. Recommendation for Independent Evaluation Department Follow-Up 29. The project was generally successful. It had already set the stage for the implementation of its Phase III. The Independent Evaluation Department may still consider conducting a project performance evaluation report, ideally to focus on documenting the best practices undertaken by the project, confirming evidences that the project indeed empowered fishing households and women, and heightened public awareness on the importance of CRM; while investigating the impacts of village level social infrastructure built in a participatory way. To encourage increased participation by fishing households and investments in microenterprises development, the project performance evaluation report could take a deeper assessment of the financial viability of these investments and recommend practical measures for minimizing business failures.