Indonesia: Sustainable Aquaculture Development for Food Security and Poverty Reduction Project

Validation Report Reference Number: PVR-430 Project Number: 35183-013 Loan Number: 2285 November 2015 Indonesia: Sustainable Aquaculture Development for Food Security and Poverty Reduction Project Independent Evaluation Department

ABBREVIATIONS ADB DFS DGA EIRR FIRR M&E MMAF NSC PCR PIU PMO Asian Development Bank District Fisheries Services Directorate General of Aquaculture economic internal rate of return financial internal rate of return monitoring and evaluation Ministry of Marine Affairs and Fisheries - national steering committee project completion report project implementation unit project management office NOTE In this report, $ refers to US dollars. Key Words adb, aquaculture, asian development bank, brackish water pond, fisher folks, food security, livelihood development, mariculture, pond, poverty alleviation, validation The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

PROJECT BASIC DATA Project Number: 35183-013 PCR Circulation Date: Jun 2015 Loan Number: 2285 PCR Validation Date: Nov 2015 Project Name: Sustainable Aquaculture Development for Food Security and Poverty Reduction Project Country: Indonesia Approved ($ million) Actual ($ million) Sector: Agriculture, Natural Resources, and Total Project Costs: 44.52 44.61 ADB Financing: ($ million) Rural Development ADF: 33.30 Loan: (SDR equivalent, million) 33.30 22.35 31.60 20.46 Borrower: 8.46 9.49 OCR: 0.00 Beneficiaries: 2.76 3.52 Others: 0.00 0.00 Cofinancier: Total Cofinancing: 0.00 0.00 Approval Date: 12 Dec 2006 Effectiveness Date: 13 Jun 2007 13 Jul 2007 Signing Date: 15 Mar 2007 Closing Date: 31 Dec 2013 31 Dec 2013 Project Officers: Location: ADB headquarters ADB headquarters Peer Reviewer: From Dec 2006 Jan 2009 To Dec 2008 Dec 2013 M. J. Rahman M. N. Islam Initial Reviewers: E. Brotoisworo, E. Gozali, Principal Consultant Evaluation Specialist, IED1 M. J. Dimayuga, Senior Evaluation Officer, IED1 Quality Reviewer: F. Ahmed, Lead Director: W. Kolkma, IED1 Results Management Specialist, IED1 ADB = Asian Development Bank; ADF = Asian Development Fund; IED1 = Independent Evaluation Department, Division 1; IED2 = Independent Evaluation Department, Division 2; OCR = ordinary capital resources; PCR = project completion report; SDR = special drawing right. A. Rationale I. PROJECT DESCRIPTION 1. The investment rationale for the project 1 was based on aquaculture s increasingly important role in assuring food security and improving household incomes, as well as in generating foreign exchange revenues from exports. Aquaculture which includes the development of sustainable and community-managed freshwater, brackish water, and marine aquaculture has become an alternative livelihood for fisher folk who are usually engaged in open sea fishing, and has helped reduce pressure on marine and coastal resources. Poor farming practices, environmental degradation and pollution, lack of access to credit by smallscale farmers, expensive inputs, problems in marketing, and conflicts in the use of coastal areas have lowered the productivity of the aquaculture subsector. 1 ADB. 2006. Report and Recommendation of the President to the Board of Directors: Proposed Loan to Indonesia for the Sustainable Aquaculture Development for Food Security and Poverty Reduction Project. Manila.

2 B. Expected Impact 2. The expected impact of the project was reduced poverty and increased food security. The performance targets were (i) food and nonfood household expenditures, and fish consumption to increase by 20% by year 6 from a baseline of about Rp155,000 in 2004; and (ii) incomes of 14,000 poor households to increase to above the poverty line by year 6. C. Objectives or Expected Outcome 3. The envisaged outcome of the project was to increase fish production and other aquatic products, and to improve the income, nutrition, and the employment status of poor fish farmers and coastal communities (report and recommendation of the President, para. 21). In the design and monitoring framework, the statement was simplified to increased fish production through sustainable aquaculture development. Outcome targets were (i) increased production, productivity, and range of products for all types of aquaculture operations from 2006 levels of at least 30% by year 6; (ii) incomes of fish farmer households and communities (about Rp320,000 in 2004) increased of 20% by year 6; and (iii) about 18,000 jobs created of the project by year 6. D. Outputs 4. The project was implemented in five districts in four provinces: (i) Langkat district in North Sumatera Province, (ii) Ogan Komering Ilir in South Sumatera Province, (iii) Karawang and Sumedang districts in West Java Province, and (iv) Buton in Southeast Sulawesi Province. It focused on developing small-scale and low-cost aquaculture that was environment friendly and could be easily replicated by fish farmers organizations and small- to medium-scale private entrepreneurs. The project had three components: (i) aquaculture production enhancement, (ii) aquaculture support services, and (iii) institutional strengthening and project management. 5. Component 1: Aquaculture Production Enhancement. This component aimed to improve the production of the major types of aquaculture systems in the project area. Specific activities under the component were outlined in five output targets: First, about 14,000 poor households would be organized and would participate in community-managed aquaculture enterprises. Second, brackish water aquaculture and mariculture facilities would include (i) 5,700 hectares (ha) of fish ponds rehabilitated and better managed (this target was revised to 4,100 ha due to smaller potential area for brackish water development in South Sumatra province); (ii) 300 units mariculture cages established and managed; (iii) 150 brackish water and mariculture demonstration units established; and (iv) 500 ha of seaweed culture facilities developed. The third target output group was for freshwater aquaculture, comprising (i) 45 ha of fish ponds rehabilitated or established, and (ii) 100 units demonstration facilities established (including for rice fish culture). Fourth, hatcheries would be established as follows: (i) 27 freshwater hatcheries (this target was revised to 20 freshwater hatcheries and 7 mariculture hatcheries); and (ii) 10 nurseries established. Fifth, other infrastructure to include (i) 150 units of water supply facilities provided, and (ii) 60 kilometers of access roads and pathways to be rehabilitated. 6. Component 2: Aquaculture Support Services. The expected outputs were production techniques and quality of aquaculture products improved by strengthening extension services and improving post-harvest handling, processing, and marketing.

3 7. The performance targets were as follows: (i) about 3,500 community groups served by extension workers (this target was reduced to 930 groups since the actual number of project beneficiaries in each group actually ranged from 10 20 members instead of 4); (ii) about 1,250 women s groups served by extension workers (this target was likewise reduced to 200 groups since members per group were more than the numbers in the appraisal plan; (iii) at least 300 fisheries extension activities conducted in all project sites by year 6; (iv) at least 50% of project beneficiaries assisted in gaining access to various financing schemes; (v) fish post-harvest and economic losses reduced by 30%; (vi) value of cultured fish and other aquatic products increased by 20%; (vii) sales of cultured fish increased by about 30% by year 6; (viii) market information, including prices for fish and other cultured commodities effectively disseminated to target beneficiaries; (ix) water quality in aquaculture production areas improved in all project districts; (x) water quality and fish disease laboratories established and functional at project sites; and (xi) applied aquaculture research (e.g., brood stock development, seed production, and feed development), ecological assessment studies, and other special studies conducted by suitable research and academic institutions 8. Component 3: Institutional Strengthening and Project Management. The expected outputs were as follows: strengthened capacity of national and district-level institutions, including private sector institutions, in aquaculture development and management; and support for establishing and operating the project management office and units. The performance argets were (i) 45 fisheries personnel of the Directorate General of Aquaculture (DGA) and participating district fisheries services trained in aquaculture, environment, and related fields; (ii) training and transport equipment provided to 70 extension officers (community-based and district fisheries service staff); (iii) five aquaculture technical centers upgraded, renovated, or newly established; (iv) computer-based aquaculture information system developed and installed at the DGA and linked to participating districts and concerned offices; (v) aquaculture-related information materials produced and distributed; (vi) up-to-date and consistent information presented in published materials; (vii) project management office (PMO) and project implementation units (PIUs) established in the DGA and five project districts; (viii) PMO and PIUs adequately staffed and equipped with necessary facilities; (ix) annual project implementation achievements consistent with predetermined or planned implementation targets for each year; and (x) funds budgeted for project disbursed and utilized in a timely manner. E. Provision of Inputs 9. The project costs at appraisal were estimated at $44.52 million equivalent. The Asian Development Bank (ADB) loan financed $33.30 million (74.8%). The remaining balance of $11.22 million equivalent (about 25% of the project cost) was funded by the national government and five district governments ($8.46 million equivalent or 19.0%) and project beneficiaries ($2.76 million equivalent or 6.2%). Project beneficiaries contributed labor and materials for construction and rehabilitation of hatcheries, fishponds, processing facilities, access roads, and small-scale social infrastructure. Actual cost at completion was $44.61 million. The imprest account helped expedite utilization of loan funds. Initially, implementation was delayed because implementation guidelines in the budget appropriation decree of the Ministry of Finance were not available.

4 F. Implementation Arrangements 10. The executing agency was the DGA of the Ministry of Marine Affairs and Fisheries (MMAF). It was responsible for overall project management. District governments were responsible for cross-sector coordination and implementation of field activities. Private sector and nongovernment organizations and community-based organizations were involved in selected project activities in communities. A PMO was established within the DGA, which oversaw day-to-day project implementation. A PIU was set up in each project district. A consultant team of international and national specialists provided technical assistance to the project at the national and district levels. A national steering committee (NSC), chaired by the Director General of Aquaculture, guided the overall direction of the project and provided interagency coordination. In districts, a regional advisory committee was established to oversee project coordination between the various local agencies and liaise with the NSC. II. EVALUATION OF PERFORMANCE AND RATINGS A. Relevance of Design and Formulation 11. The project completion report (PCR) rated the project highly relevant. The project was aligned with ADB s country strategy and program for 2006 2009, 2 which focused on three pillars the first one being broad-based economic growth (including investment in agriculture and/or aquaculture). 3 The project remained consistent with ADB s country partnership strategy for 2012 2014, 4 which emphasized poverty reduction, support for inclusive economic growth, and sustainable management of natural resources. The project supported the national strategy and policies for aquaculture development and poverty reduction as stipulated in the MMAF s Strategic Plan for 2005 2008, and it was consistent with the government s National Medium- Term Development Plan, 2010 2014, which prioritized poverty reduction. The project was also consistent with the MMAF s Medium-Term Strategic Plan for 2010 2014, which aimed to expand the country s aquaculture production. The project supported the ongoing decentralization during implementation. 12. Despite the strategic alignment, this validation notes two issues with the project design. First, the original concept to strengthen community and women s groups was changed (para. 7). The targeted number of community groups served was reduced from 3,500 to 930, which meant that these organizations had to be enlarged from 4 members per group to 10 20 members for community organizations. Likewise, the target for women s groups served by extension workers was also reduced from 1,250 to 200 (PCR, Appendix 1). The original intent to build community enterprises along small groups of household owners was no longer pursued. Second, the reduction in the area of fishponds rehabilitated from 5,700 ha to 4,100 ha also reflected a concern over the initial needs assessment at appraisal. This validation notes that despite the decrease in the rehabilitated area, there were no savings in the civil work cost at completion (PCR, Appendix 3). It may be inferred that the unit cost per hectare for civil works was higher than estimated at the design stage. This validation commends the project team for adjusting the project scope during implementation. Nonetheless, the abovementioned scope changes were due to factors that could have been foreseen at appraisal. Given the design-related issues, this validation considers the project design relevant. 2 ADB. 2006. Country Strategy and Program: Indonesia, 2006 2009. Manila. 3 The other two pillars were (i) inclusive social development (including education, support for vulnerable groups, and communal natural resource management); and (ii) good governance (including support for decentralization efforts to strengthen public service delivery and local participation in government). 4 ADB. 2012. Country Partnership Strategy: Indonesia, 2012 2014. Manila.

5 B. Effectiveness in Achieving Project Outcomes and Outputs 13. The PCR rated the project highly effective on the basis that most of the outcome targets were exceeded. This validation noted that at project completion, results of the household surveys showed that production and productivity of fish increased by an average of 60%, and other aquatic products increased by an average of 84%, which exceeded the project s outcome target by 30%. The household surveys also showed that the project beneficiaries net income improved by 70%, surpassing another outcome target by 20%. These were increases from 2004 to 2012. The project generated 306,382 person-days of employment consisting of 296,653 person-days in civil works activities for the poor fishing households and 9,729 person-days in aquaculture activities. The appraisal target of 18,000 jobs created was considered not applicable since aquaculture and mariculture create daily jobs rather than year-round job positions. 14. There were several key shortfalls on the project outputs: (i) fish ponds completed covered 3,277 ha compared with the 4,100 ha revised target (or 80% achievement); (i) four mariculture hatcheries were constructed instead of the seven targeted; (iii) about 38 kilometers of access road and pathways constructed and/or rehabilitated fell short of the original target of 60 kilometers; (iv) 139 women s groups participated in the project activities (or 70% of the target); and (v) water quality in aquaculture ponds improved only in the Langkat and Sumedang districts but deteriorated in Karawang, Ogan Komering Ilir, and Buton, while the target was improvement in all project districts. Considering that the project achieved its intended outcomes but could not complete its planned activities including several key outputs, this validation considers the project effective. C. Efficiency of Resource Use in Achieving Outcomes and Outputs 15. The PCR rated the project efficient in achieving outcome and outputs since it was costeffective. This validation notes the delay in project implementation because a Ministry of Finance decree that would guide the transfer of grant funds to project districts through ongranting agreements was lacking. This improved when the government endorsed the financial mechanism and a national budget for 2009 was approved. By December 2011, 59% of the project was implemented compared with 65% elapsed time. This validation notes that the 2- year delay in physical implementation was seriously addressed by higher staff time inputs available and closer monitoring, efficient processing of project procurement and disbursement, and frequent project reviews. This validation also notes that more frequent review missions helped to quickly put the project back on track. Problem analysis and problem solving during the review mission brought about faster project implementation to make up for the earlier delays. The project closed as scheduled on December 2013. 16. Economic returns for 19 representative aquaculture enterprises were reassessed at project completion. 5 Results showed that 18 aquaculture enterprises were economically viable with an economic internal rate of return (EIRR) of 40.75% for the project as a whole, surpassing the EIRR estimates at appraisal. 6 This validation notes that the EIRR recalculation incorporated optimistic early benefit realization for freshwater aquaculture enterprises. Further, labor income from project civil work (project expenses) should not have been included 5 The PCR indicated that the economic analysis conducted at project completion covered the project as a whole, using actual income and cost data from 2009 to 2012 for various freshwater, marine, and brackish water aquaculture enterprises that were gathered in the five project districts during the project completion survey. 6 At project appraisal, the EIRR values for individual schemes ranged from 15% to 21%, which were reported to provide an indicative range for the project s overall EIRR from five representative aquaculture enterprises.

6 as a net economic benefit. A recalculation by this validation, adjusting for these issues, nonetheless kept the EIRR robust at above 30%. The high economic returns of aquaculture enterprises benefit from fairly firm international prices, but there is a need to be cautious about price stability and other production risks. Considering the available data on resource and process efficiency, this validation considers the project highly efficient. D. Preliminary Assessment of Sustainability 17. The PCR rated the project most likely sustainable based on the strong commitment of the MMAF and district governments to provide continuous financial support, the expected growth of cooperatives in the project districts, and indications that investments under the project are relevant to the communities. The PCR underlined the project s effort to undertake an exit strategy (in December 2013). One of the measures agreed upon was the budgetary commitment of the district governments to enable the District Fisheries Services (DFS) to provide continued capacity building for beneficiary groups and extend technical support and production input assistance to these groups. The PCR reported that after project completion, (i) the DFS in all the project areas have continued supporting the project beneficiaries; (ii) most of the beneficiary groups organized under the project are still functioning and active; and (iii) groups have become successful and self-reliant in their business operations, expanding their aquaculture businesses using their own savings. 18. The PCR indicated that investments in 18 of the 19 aquaculture enterprises are financially viable with financial internal rates of return (FIRRs) ranging from 53% to more than 100%, which are above the prevailing financial cost of capital of 10% (PCR, Table A12.1). This validation could not review the FIRR estimates in detail but considers these high returns plausible because of the fairly robust overseas demands for seafood and aquaculture products. The aquaculture and mariculture enterprises, however, are prone to risks of production oversupply and the onset of diseases for which financial premium should have been incorporated in the FIRR calculation. Overall, based on the evidence on financial returns and continued government support, this validation concurs with the PCR that the project is most likely sustainable. E. Impact 19. The PCR rated the overall project s impact significant because expected impacts were substantially achieved such as reduced poverty and increased food security, with food and nonfood expenditures and fish consumption up by 20%, and incomes of 14,000 poor households increased to above the poverty line. This validation noted the increase of fish consumption in the project districts by 83% on average from 2006 to 2013, exceeding the 20% target. The project contributed significantly to reducing poverty incidence in the five project districts from 15% 25% in 2006 to 10% 16% in 2011. Moreover, the project is reported to have directly benefited 14,585 poor fish farmer households, exceeding the target of 14,000 households. 20. On the social aspects, the project improved food security and nutrition of poor fishing households, increased fish consumption, and improved health through the provision of water supply facilities. Fish farmers and women were also empowered, and gender equality enhanced. Women constituted up to 25% of total project beneficiaries and 92% of the total processing groups. Participation of women in training on aquaculture production and processing has significantly improved their opportunities to engage in livelihood enterprises and improve their family income. Women beneficiaries reported that the project addressed their immediate

7 economic needs, enhanced their status in the community, and empowered them to express their views on how the enterprises could be managed and operated. 21. The PCR indicated that the assessment of the project s environmental impacts showed that the construction and rehabilitation works have had minimal adverse environmental impact. The increased use of fish feeds in freshwater ponds and freshwater cages was found to cause water pollution, but its impact on the environment was minimal. 7 The project did not have any impact on resettlement and indigenous peoples. Environmental impact of aquaculture operation is well known, particularly due to pollution from intensive culture practices. This validation noted the adequate effort to address this issue through (i) the design of civil works that minimized the adverse impacts from the project as well as from the surrounding environment, (ii) the provision of environmental management training to PIU and extension staff, (iii) the establishment of laboratory facilities for monitoring water quality and fish diseases in each project district, and (iv) the encouragement of farmers to adopt low-density culture and well-adapted species. Besides mitigating adverse environmental impacts, the project is noted to have been designed to generate environmental improvement, particularly from project intervention under mangrove development. Mangroves were planted in 34.4 ha although droughts in the dry season and flooding in the rainy season caused seedling mortality, which is considered normal. 8 Considering the overall project effort and outputs, this validation concurs with the PCR that the overall project impact is significant. III. OTHER PERFORMANCE ASSESSMENTS A. Performance of the Borrower and Executing Agency 22. This validation noted that the lack of clarity in the financial transfer mechanism between the central and district government impeded project progress in the first 2 years of implementation. However, as soon as the financial mechanism was clarified and approved, the project was implemented smoothly, overcoming the initial delay, and was completed on time. Based on the satisfactory project implementation, which in many cases also exceeded outcome targets, this validation considers that the PMO and the PIUs had performed well. The PCR also indicated that the DFS in all project districts successfully set up the necessary local implementation arrangements and coordinated their activities with other agencies, avoiding major problems in project implementation. This validation agrees with the PCR that the DGA s performance as executing agency was satisfactory based on the project s successful and timely completion despite a 2-year start-up delay. This validation also notes that the DGA gave the district government opportunities to implement and manage the project components, which showed the government s positive attitude to decentralization. In almost all of the project components, the objectives were achieved, and in some even exceeded. The PCR rated the 7 As a result of training and extensions, fish farmers adopted the recommended fingerling or seed-stocking rate and appropriate feeding and fertilizer rates to avoid water pollution in their ponds or cages. 8 Survival rate of mangrove planting greatly varies depending on many factors, including natural conditions such as coastal soil and safety from disaster, and other human-induced factors, e.g., in India (Bhatt, J. R. and K. Kathiresan. 2012. Valuation, carbon sequestration potential and restoration of mangrove ecosystems in India reported a survival rate at 75% to 80% from planting 6.8 million mangrove saplings in 1,475 ha; Ranasinghe, T. S. 2012. Mangrove restoration efforts in Sri Lanka reported survival rates between 0.6% to 90% in six locations; Hoa, S. L. T., R. Suzuki, and M. F. Thomsen. 2012. Adapting to natural disasters and contributing to climate change mitigation: mangrove community forestry in Viet Nam reported 15% 20% survival rate in mangrove planting in Viet Nam; Mahardi, W. 2012. Lessons learned from the programme Let s Plant Mangroves: A case study from villages in Banten and Central Java provinces, Indonesia reported that planting in several locations showed survival rates between 0% and 95%. All papers published in Sharing Lessons on Mangrove Restoration Proceedings and a Call for Action from an MFF Regional Colloquium, 30 31 August 2012, in Mamallapuram, India.

8 performance of the borrower and the executing agency satisfactory. This validation supports this rating. B. Performance of the Asian Development Bank 23. The PCR rated ADB s performance satisfactory. ADB conducted a project inception mission, 12 review missions (2 missions per year on average), a midterm review mission, and a PCR mission during implementation to ensure proper project implementation. The missions included project administration to review the overall progress and issues encountered during implementation, and resolve issues affecting the project and other issues relating to project management. This validation noted that the review and midterm missions were useful in recommending time-bound actions to improve project implementation, which resolved implementation issues effectively. ADB acted promptly on issues encountered as well as the government s requests for minor changes in project scope, output targets, and loan reallocation. Considering ADB s effort during project implementation and the outputs of the project, this validation concurs that ADB performance was satisfactory. IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS A. Overall Assessment and Ratings 24. Overall, this validation finds the project goals consistent with ADB policies and priorities. The project was able to deliver the expected impact of reducing poverty and increasing food security in the project areas. Issues were noted in the initial scope assessment of civil work areas and the approach to beneficiary organization. The project, nonetheless, achieved its outcomes. Despite a 2-year implementation delay, the work was generally completed as planned in a cost-effective manner. The project was also most likely sustainable after project completion. Based on the above findings, this validation assesses the project successful (see table). Overall Ratings Criteria PCR IED Review Reason for Disagreement and/or Comments Relevance: Highly relevant Relevant Despite the revised scope, there were still some design flaws, which were reflected in the reduced number of community groups and brackish water fishponds (para. 13). Effectiveness in achieving outcome: Efficiency in achieving outcome and outputs: Highly effective Effective The project achieved its intended outcomes but could not complete its planned activities including several key outputs (paras. 14 15). Efficient Highly efficient Despite an initial implementation delay, the project was still completed on time and the recalculated EIRR remained high (paras. 16 17). Preliminary assessment Most likely Most likely of sustainability: sustainable sustainable Overall Assessment: Highly Successful successful Impact: Significant Significant

9 Borrower and executing Satisfactory Satisfactory agency: Performance of ADB: Satisfactory Satisfactory Quality of PCR: Satisfactory Comprehensive and sound discussion of the financial and economic viability of the project (para. 30). ADB = Asian Development Bank, EIRR = economic internal rate of return, IED = Independent Evaluation Department, PCR = project completion report. Source: ADB Independent Evaluation Department. B. Lessons 25. This validation noted that the PCR underlined several key features, which supported project success: (i) strong institutional capacity development, coordination, and teamwork among all parties involved in project implementation; (ii) effective strategy in empowering women through gender mainstreaming; and (iii) strengthened cooperatives and enterprises. This validation agrees with the lessons stated in the PCR. However, above all these, the most important lesson was to ensure that all requirements for project implementation should be in place early on and, hence, immediate steps can be taken to avoid delays in future projects, which was the key issue during the first 2 years of project implementation. C. Recommendations for Follow-Up 26. The PCR highlighted several aspects that needed follow up and briefly discussed these recommendations, including future monitoring, handover of project assets, budgetary support, capacity building, market assistance, replication, and formulating measurable output indicators. This validation considers these recommendations appropriate. This validation further recommends that the enterprises and agencies concerned should consider actions that will strengthen the financial resilience of enterprises against a market price downturn or harvest failures (e.g., due to diseases). These may involve the introduction of an insurance scheme and building cash reserves for the enterprises. 27. Considering the importance of the fishery sector, including aquaculture, in food security and poverty reduction over the medium term, it is advisable for ADB to remain engaged and support deeper strategic analyses of the fisheries sector to assess the needs for future policy or investment support. V. OTHER CONSIDERATIONS AND FOLLOW-UP A. Monitoring and Evaluation Design, Implementation, and Utilization 28. The basis for project performance monitoring and evaluation shall be the design and monitoring framework and the baseline surveys. This validation notes that the project established and implemented a project performance management system in compliance with the loan covenants, established a monitoring and evaluation (M&E) database, and developed internal M&E procedures. It notes that the project offices conducted most of the M&E internally; M&E was done only twice externally (i.e., M&E for resettlement and alternative livelihoods development related to farmland-to-wetland and farmland-to-forest conversions). This validation considers that project performance monitoring and evaluation was properly done, the result of M&E was adequately reflected in the project monitoring framework, and there were no major shortfalls in M&E activities.

10 B. Comments on Project Completion Report Quality 29. The PCR was well written and systematically presented. Project performance, including the review of effectiveness indicators in the design and monitoring framework, was systematically and adequately analyzed. The presentation of the main text with appendixes was consistent; and the presentation of environmental, procurement, covenants, and cross-cutting issues was adequate. However, both the economic and financial rate of return analyses to account for risks in output price fluctuation and diseases (e.g., sensitivity analyses) needed to be sharpened, and accounting of project benefits (para. 17) needed to be more accurate. Nevertheless, the quality of the PCR is rated satisfactory. C. Data Sources for Validation 30. The data sources used for this evaluation were the project s report and recommendation of the President, the PCR and its supplementary appendixes, loan review mission reports, and ADB's Country Strategy and Program for Indonesia, 2006 2009 and 2012 2014. D. Recommendation for Independent Evaluation Department Follow-Up 31. The project was successful in completing implementation on time and achieved substantial outcomes. High economic and financial returns are expected. However, the profitability of aquaculture and mariculture subsectors in the past had fluctuated. It would be useful to review the financial and economic returns of these investments over a longer period through a project performance evaluation.