ON A LOAN IN THE AMOUNT OF US$50 MILLION TO THE REPUBLIC OF INDONESIA FOR A DAM OPERATIONAL IMPROVEMENT AND SAFETY PROJECT (DOISP) P096532

Transcription

1 Public Disclosure Authorized Document of The World Bank Report No: ICR Public Disclosure Authorized IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-76690) ON A LOAN IN THE AMOUNT OF US$50 MILLION Public Disclosure Authorized TO THE REPUBLIC OF INDONESIA FOR A DAM OPERATIONAL IMPROVEMENT AND SAFETY PROJECT (DOISP) P Public Disclosure Authorized Water Global Practice East Asia and the Pacific Region January 29,

2 CURRENCY EQUIVALENTS (Exchange Rate Effective 06/08/2009) Currency Unit = Indonesian Rupiah 1.00 = US$ US$ 1.00 = 10, (Exchange Rate of January 10, 2017 used) Currency Unit = Indonesian Rupiah 1.00 = US$ US$ 1.00 = 13, FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS AF Additional Financing IRR Internal Rate of Return AKNOP Real demand for O&M ISR Implementation Status Report APL Adaptable Program Loan IWRM Integrated Water Resources Management Balai BBWS River Basin Agency (under MPW) MEAV Modern Equivalent Asset Value to estimate O&M budget BDSF Basic Dam Safety Facilities M&E Monitoring & Evaluation CDMU Central Dam Monitoring Unit MDG Millennium Development Goals CP Community Program MPW Ministry of Public Works and Housing CPF Country Partnership Framework NPV Net Present Value CPIU Central Project Implementation Unit NSCWR National Steering Committee for Water Resources CPMU Central Project Management Unit O&M Operation & Maintenance CPS Country Partnership Strategy PDO Project Development Objective DBOP Directorate for Operation and PIP Project Implementation Plan Maintenance in DGWR DGWR Directorate General for Water PIU Project Implementation Unit Resources of the Ministry of Public Works and Housing (MPW) DMU Dam Monitoring Unit PJT State-owned company for the operation of water resources 2 assets River Basin Organizations (BBWS) DOISP Dam Operational Improvement and Safety Project RBO DSP Dam Safety Project RKM Community Work Plan DSU Dam Safety Unit RPJM(N) (National) Mid-term Development Plan

3 EAP Emergency Action Plan RPJP Long-term Development Plan ESMF Environmental and Social Management Framework SEOP Spillway Emergency Operating Procedure GOI Government of Indonesia SID Surveys Investigations, Designs ICB International Competitive SOP Standard Operating Procedures Bidding ICOLD International Commission on TOR Terms of Reference Large Dams ICR Implementation Completion and Results Report UKL/UPL Environmental Management and Monitoring Plan IDR Indonesian Rupiah USD United States Dollars INACOLD Indonesian ICOLD Chapter WATSAL Water Resources Sector Adjustment Loan INT Institutional Integrity WISMP Water and Irrigation Sector Management Project Senior Global Practice Director: Guang Zhe Chen Country Director: Rodrigo Chaves Practice Manager: Sudipto Sakar Project Team Leader: Marcus Wishart / Ximing Zhang ICR Team Leader: Martin Albrecht 3

4 INDONESIA Dam Operational Improvement and Safety Project (DOISP) CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1. Project Costs and Financing Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Bank Lending and Implementation Support/Supervision Processes Annex 5. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 6. List of Supporting Documents MAP 4

5 Data Sheet A. Basic Information Country: Indonesia Project Name: Dam Operational Improvement and Safety Project ID: P L/C/TF Number(s): IBRD ICR Date: 01/29/2017 ICR Type: Core ICR Lending Instrument: SIL Borrower: Original Total Commitment: Revised Amount: USD 50.00M Environmental Category: Implementing Agencies: Ministry of Public Works and Housing Cofinanciers and Other External Partners: B. Key Dates REPUBLIC OF INDONESIA USD 50.00M Disbursed Amount: USD 47.02M Process Date Process Original Date Revised / Actual Date(s) Concept Review: 03/28/2006 Effectiveness: 06/08/2009 Appraisal: 10/15/2008 Restructuring(s): 09/19/ /29/ /23/2016 Approval: 03/19/2009 Mid-term Review: 12/15/ /21/2012 Closing: 12/31/ /30/2017 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Moderately Satisfactory Moderate Moderately Satisfactory Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Quality of Supervision: Moderately Satisfactory Implementing Moderately Satisfactory Agency/Agencies: 5

6 Overall Bank Performance: Overall Borrower Moderately Satisfactory Performance: Moderately Satisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: No Yes Satisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Major Sector/Sector Public Administration Public administration - Water, sanitation and flood protection Water, Sanitation and Waste Management Original Actual Other Water Supply, Sanitation and Waste Management Major Theme/Theme/Sub Theme Environment and Natural Resource Management Water Resource Management Water Institutions, Policies and Reform E. Bank Staff Positions At ICR At Approval Vice President: Laura Tuck James W. Adams Country Director: Rodrigo A. Chaves Joachim von Amsberg Practice Manager/Manager: Sudipto Sarkar Sonia Hammam Project Team Leader: Marcus J. Wishart Guy J. Alaerts ICR Team Leader: ICR Primary Author: Martin B. Albrecht Martin B. Albrecht 6

7 F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objectives of the Project are to (i) increase the safety and the functionality with respect to bulk water supply of large Ministry of Public Works-owned reservoirs; and (ii) strengthen the safety and operational management policies, regulations and administrative capacity of Ministry of Public Works. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Indicator Indicator 1 : Baseline Value Original Target Values (from approval documents) Dam portfolio risk assessment (Number, Custom) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Value quantitative or Qualitative) Date achieved 05/01/ /31/ /23/ /30/2016 Comments (incl. % Target 100% achieved. achievement) Indicator 2 : DGWR Dam Portfolio Management Program established (Number, Custom) Value quantitative or Qualitative) Date achieved 05/01/ /31/ /30/2016 Comments (incl. % achievement) Indicator 3 : Target 100% achieved. The dam portfolio management program comprises all 63 dams under the project, including an increase in O&M budget, training for staff, hydrological maps, as well as the establishment of a DOISP Referable Dam Works and Field Inspection and Certification per DSU Priority Plan (Percentage, Custom) Value quantitative or Qualitative) 0 % 75 % 94 % Date achieved 05/01/ /31/ /30/2016 Comments (incl. % Target over achieved (125%). achievement) 7

8 (b) Intermediate Outcome Indicator(s) Indicator Indicator 1 : Baseline Value Original Target Values (from approval documents) Number of dams rehabilitated (Number, Custom) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Value (quantitative or Qualitative) Date achieved 05/01/ /31/ /31/ /30/2016 Comments (incl. % Target overachieved (103%). achievement) Indicator 2 : Number of 34 priority dams with SEOP operational (Percentage, Custom) Value (quantitative or Qualitative) 0 >90% 165% Date achieved 05/01/ /31/ /30/2016 Comments (incl. % achievement) Indicator 3 : Target overachieved (183%). Originally, SEOPs for 34 dams were targeted. Under the project, SEOPs were prepared for 56 dams as part of the EAPs. Operating staff training took place for 63 dam. (Number, Custom) Value (quantitative or Qualitative) Date achieved 05/01/ /31/ /30/2016 Comments (incl. % Target overachieved (102%). achievement) Indicator 4 : Special studies for dams completed (Number, Custom) Value (quantitative or Qualitative) Date achieved 05/01/ /31/ /31/ /30/2016 Comments (incl. % achievement) Target achieved 100%. The dams for which the special studies were prepared will be included under the AF Indicator 5 : O&M Manuals (including related training) for Dams completed. (Number, Custom) Value (quantitative or Qualitative) Date achieved 05/01/ /31/ /30/2016 8

9 Comments (incl. % Target 91% achieved. achievement) Indicator 6 : Community-Dam Operator MOUs signed on Greenbelt Management (Number, Custom) Value (quantitative or Qualitative) Date achieved 05/01/ /31/ /30/2016 Comments (incl. % Target overachieved (113%). achievement) Indicator 7 : Household joining catchment management activities (Percentage, Custom) Value (quantitative or Qualitative) Date achieved 05/01/ /31/ /30/2016 Comments (incl. % Target overachieved (148%). achievement) Indicator 8 : DOISP Reservoir Survey completed (Number, Custom) Value (quantitative or Qualitative) Date achieved 05/01/ /31/ /30/2016 Comments (incl. % Target overachieved (103%). achievement) Indicator 9 : DSU staff recruited and trained (Percentage, Custom) Value (quantitative or Qualitative) Date achieved 05/01/ /31/ /30/2016 Comments (incl. % Target 100% achieved. achievement) INACOLD certification of Dam Professional (Percentage, Custom) Indicator 10 : Value (quantitative or Qualitative) Date achieved 05/01/ /31/ /30/2016 Comments (incl. % Target overachieved (125%). achievement) 9

10 G. Ratings of Project Performance in ISRs No. Actual Date ISR DO IP Disbursements Archived (USD millions) 1 12/24/2009 Satisfactory Moderately Satisfactory /17/2011 Moderately Satisfactory Moderately Unsatisfactory /14/2011 Moderately Satisfactory Moderately Unsatisfactory /29/2012 Moderately Unsatisfactory Unsatisfactory /03/2012 Moderately Satisfactory Moderately Satisfactory /20/2013 Moderately Satisfactory Moderately Satisfactory /06/2014 Moderately Satisfactory Moderately Satisfactory /29/2014 Moderately Satisfactory Moderately Satisfactory /26/2015 Moderately Satisfactory Moderately Satisfactory /15/2015 Moderately Satisfactory Moderately Satisfactory /13/2015 Moderately Satisfactory Moderately Satisfactory /24/2016 Moderately Satisfactory Moderately Satisfactory /22/2016 Satisfactory Satisfactory H. Restructuring (if any) Restructuring Date(s) Board Approved PDO Change ISR Ratings at Restructuring DO Amount Disbursed at Restructuring in USD millions 09/19/2013 MS MS IP Reason for Restructuring & Key Changes Made Extend closing date, improve project organization, reallocate loan proceeds, slight change in intermediate outcome indicators. 06/29/2015 MS MS /23/2016 MS MS Extend closing date to allow for completion of contracts. Extend closing date to allow for preparation of Additional Financing (DOISP2, P161514) 10

11 I. Disbursement Profile 11

12 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country Context. Indonesia s geography as a tropical archipelago with more than 17,000 islands and high seasonal rainfall variability was reflected in the complexity of and requirement for sound water resources management. Despite more than 280 dams across the country, around 170 of which classified as large, Indonesia s storage capacity remained low compared with other countries 1, which was constraining long term development goals of food security, energy security and water security. A large number of the existing assets such as dams, barrages or canals had been built before 1990, bringing them to the end of their economic lifetime and thus making them a safety hazard unless rehabilitated. While there had been rapid infrastructure development over the past decades, operation and maintenance (O&M) of these dams have not been adequately addressed. This increases the safety risk, the loss of effectiveness due to sedimentation, and high expense if they need to be replaced or rehabilitated. 2. Government Strategy. The Government of Indonesia recognized and outlined in development plans 2 the importance of providing water for its growing population. At the time of appraisal, urban water demand was growing together with economic development, population growth and urbanization while much of the country s staple food rice was grown on irrigated areas that consumed about 85% of the bulk water supply. With the high seasonal variability of rainfall, Indonesia relied heavily on reservoirs and barrages on rivers for water supply 3. In the dry season, reservoirs ensured a second and third annual harvest through irrigation of around 4.7 million hectares, raising rural incomes and stabilizing food prices. About 18% of the total installed power generating capacity in Indonesia was generated from hydropower. Large dams also functioned as key infrastructure for flood prevention, which otherwise were a common cause for natural disasters. 3. Considering future impacts of climate change 4, reservoirs would play a pivotal role in the management of Indonesia s water resources. The Government was responding with the intention of building 15 new reservoirs in the coming decade 5 as well as with strengthening the capacity and resources for operation and maintenance and timely remedial repairs. In the 1990s, with World Bank support, Indonesia had introduced a dam safety framework 6. Linked with the political move towards decentralization, the Government had initiated sector-wide legal, regulatory and administrative reform for water resources, which had been concluded by a water resources law 7. 1 Storage capacity in Indonesia was about 100 m3 per capita in 2005, compared with around 416 m3 per capita in China, 3,600 m3 per capita in Brazil, or 190 m3 per capita in India at that time. 2 Long-term Development Plan (RPJP) , and in several mid-term development plans (RPJM) with the RPJMN at the time of appraisal. 3 Most large cities such as Jakarta, Surabaya, Makassar and Semarang depend largely on reservoirs and barrages for their water supply - greater Jakarta depends on one large reservoir for about 80% of its tap water; 4 Most studies and observations predict seasons to be more extreme in either sense: the wet season will become shorter with rainfall coming in fewer, more intense events, while the dry season will be longer with more pronounced periods of drought. 5 The plan of 15 new dams was the status at appraisal. In 2014, the current Government published their plan to build 65 new dams until Through the Dam Safety Project DSP ( ). 7 Law No. 7/2004, introducing Integrated Water Resources Management (IWRM), better governance as well as improved financial sustainability of hydraulic infrastructure. This law was cancelled by the Constitutional 12

13 4. Rationale for Bank Involvement. Given Indonesia s demonstrated political will including the establishment of dam safety institutions, the passing of a water resources law, and the recognition for the importance of reservoirs with regards to the future development of the country Bank support could contribute to further development and implementation of comprehensive regulatory and administrative framework for dam management, ultimately increasing dam safety and functionality. Following a long-standing relationship between the Bank and Indonesia in the water resources sector and a focus on dam safety 8, the project intended to consolidate the first institutional and policy achievements of the Dam Safety Project (DSP) and sector reform achievements. 5. The Bank built on broad experience in technically demanding tasks and institutional development in the sector with a track record in dam safety in India, China, Sri Lanka and other countries. Further, the Bank had broad experience in several regions on the linkage between loss of reservoir functionality and sediment import from the upper watershed a predominant issue in reservoir management in Indonesia. 6. Higher Level Objectives to Which the Project Contributed. The project intended to contribute towards meeting the Millennium Development Goals (MDGs) to increase access to water supply 9 both for human consumption and for productive use a core goal for the Government of Indonesia as well as the Bank. Further, the project intended to contribute to the Government s strategy to mitigate climate change and adapt to the changes by reducing its vulnerability of water related disasters. Finally, the project was key to maintain the momentum for sector-wide engagement and coherent policy and institutional support, which aimed at improving the performance and the sustainability of this sector, specifically on river basin management, irrigation and flood protection. The Country Partnership Strategy (CPS) at the beginning of the project in 2008 focused on strengthening Indonesia s institutions and investments in (water) infrastructure, climate change and disaster mitigation, amongst others. The Project Development Objective (PDO) was consistent with these priorities. 1.2 Original Project Development Objectives (PDO) and Key Indicators 7. The objectives at the time of approval were to (i) increase the safety and the functionality with respect to bulk water supply of large Ministry of Public Works-owned reservoirs; and (ii) strengthen the safety and operational management policies, regulations and administrative capacity of Ministry of Public Works. Implicitly, the aim of the PDO is the extension of the lifetime of reservoirs and dams, delaying the urgency to build new costly reservoirs. 8. Key indicators at approval were linked to the two parts of the PDO (i and ii) were: 9. PDO part (i): Increase the safety and the functionality with respect to bulk water supply of large MPW-owned reservoirs; Court in Impacts on the institutional set-up have been avoided through the issuance of a Ministerial Decree while the Law is being revised and updated. 8 Notably in form of the Dam Safety Project DSP ( ), sector-wide policy reforms (such as the reforms associated with the Water Resources Sector Adjustment Loan WATSAL), and investments such as irrigation loans in the 1990s as well as the Water Resources and Irrigation Sector Management Project WISMP ). 9 Millennium Development Goal 7c stated: Halve, by 2015, the proportion of the population without sustainable access to safe drinking water and basic sanitation. 13

14 Key indicators: a) Reduction in dam failure risk of about 34 rehabilitated dams using an appropriate Risk Assessment Method & achieving operational Spillway Emergency Operation Plans for these dams. Extreme Hazard dams reduced from 3% to 0%; High Hazard dams reduce from 81 to <5%; Moderate Hazard dams increased from 0% to >95%. Overall Hazard of the 34 dams reduced by > 30% on Risk Score. b) Directorate General for Water Resources (DGWR) Dam Portfolio management improved by established: Annual Dam O&M Funding Program, Dam Human Resources Development Program, Dam Hydrology Program, Dam Quality Assurance system and a functional Central Dam Monitoring Unit (CDMU). c) DSU has completed > 75% of its priority dam inspection & certification load (including 2-3 large mine tailings dams). 10. PDO part (ii): Strengthen the safety and operational management policies, regulations and administrative capacity of MPW; Key indicators: a) Government Regulation on Dams and Ministerial Decrees issued. b) Administrative procedures and capacity enhanced. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 11. The PDO was not revised during the project. 12. The targets of the intermediate outcome indicators were modified to be achievable by the initial closing date of 31 December The modification included the reduction of the targeted number of rehabilitated dams from 34 to Main Beneficiaries 13. The direct beneficiaries were to be the urban communities and over 2.5 million farm households who depended on the 63 dams and reservoirs that were altogether covered under the project for their water supply and livelihood (for 34 of these 63 dams physical works were included). Beneficiaries also included downstream communities who could be placed at physical and/or environmental risk if the safety of these dams was compromised. Communities in the direct vicinity of the dams and reservoirs were directly involved in project activities through the community participation program under Component 2, collaborating on watershed management and being familiarized with appropriate safety measures under the Emergency Action Plans. 1.5 Original Components 14. The project was composed of five components: 15. Component 1: Dam Operational Improvement and Safety Works and Studies (Project Cost: US$ 31.48m). The objective of the Dam Operational Improvement and Safety Works and Studies Component was to restore dam performance and safety. This was to be achieved through minor rehabilitation and remedial works 10, developing surveys, investigations and designs (SID) 10 Minor rehabilitation and remedial works refers to works on existing dam structures that are acceptable under the project s designated safeguards framework. Examples include physical works (repair dam crest, access road, drains, stabilizing slopes, etc.) and cleaning the dam of weeds, scrubs and trees. If more complex 14

15 for medium to major works, providing Basic Dam Safety Facility (BDSF) repair and/or upgrading for improved safety monitoring and preparedness systems, establishing a river inflow and sediment monitoring system, and an assessment of spillway capacity and downstream flooding risks. A total 34 prioritized large dam/ reservoir sites were to be targeted for physical works, out of which four were ready for implementation in year one, the others were to be undertaken in phases during the following years. SIDs for medium and major works to restore and/or improve operational performance and safety of approximately other 22 dams/reservoirs were to be prepared for implementation in a successor project. Both the BDSF repair and/or upgrading as well as the spillway capacity and downstream flooding risk assessment were to be prepared for 34 dams. The river inflow and sediment monitoring system was to be set up for about 63 dams. 16. Component 2: Operations & Maintenance Improvement and Capacity Building (Project Cost: US$ 13.24m). The objective of this component was to support improved operations and maintenance and to strengthen capacity building of the dam agency. This was to be achieved through preparing O&M plans, Standard Operating Procedures (SOP) and manuals, as well as undertaking needs-based budgeting and O&M activities. Further through preparing dam and reservoir management plans and emergency spillway operation plans, staff training participatory programs with local communities on reservoir and dam management, and providing incremental operating costs for O&M, to be borne by GOI. Most of these activities were to be conducted in the aforementioned 34 dams, only the community participation program was to be conducted in 20 reservoirs. 17. Component 3: Reservoir Sedimentation Mitigation (Project Cost: US$ 13.12m). The objective of this component was to provide for measures to mitigate the risk of sedimentation of selected reservoirs and for sustained performance and safety. This was to be achieved through bathymetric 11 surveys to determine the total available water storage in reservoirs affected by sedimentation; through feasibility studies, designs and the Environmental and Social Management Framework (ESMF) for medium-term interventions within the reservoir, such as dredging, hydrosuction, flushing, etc.; preparations for a sample study for decommissioning of severely silted reservoirs, to be financed under the successor project; and through piloting of institutional models and plans to construct sediment retention and river bank protection structures of upstream rivers and (sub-)catchments. 18. Component 4: Dam Safety Institutional Improvement (Project Cost: US$ 2.14m). The objective of the Dam Safety Institutional Improvement component was to further strengthen and consolidate the regulatory framework and national dam safety institution and strengthen MPW s capacity for portfolio management and regulation in order to sustain rehabilitation works and reservoir life. This was to be achieved through preparing needed Government and Ministerial regulatory documents and concept/academic papers, a public awareness campaign about dams and reservoirs, through strengthening and developing the Dam Safety Unit (DSU), providing it with a fully furnished and equipped office and updated guidelines, through establishing a dam engineer and technician training and certification system, and through supporting incremental costs of the Central Dam Management Unit within the ministry. 19. Component 5: Project Management (Project Cost: US$ 4.36m). The objective of this component was to provide overall project management and to support the Central Project safety issues existed, detailed SID were conducted, which included studies on cracks, seepage, intakes, stability, etc. BDSF includes repair and installation of instrumentation such as piezometers, gauges, etc. 11 Bathymetric survey describes a measurement to determine the depth profile of a water body. 15

16 Management Unit (CPMU), the Central Project Implementation Unit (CPIU) and each Project Implementation Unit (PIU). 1.6 Revised Components 20. The project components were not revised during the project. 1.7 Other significant changes 21. Reorganization of the implementing agency. The project s implementing agency, the Directorate General for Water Resources (DGWR), Ministry of Public Works, was reorganized during implementation. This led to new implementation units and procedures with subsequent delays in disbursement. The reorganization also led to the establishment of a new Directorate for Operation and Maintenance (DBOP) within DGWR, in which the CPIU was re-established after the first restructuring. 22. A change in Government in October 2014 introduced new policies that, amongst other, focused on water related projects to achieve food, water and energy security 12. This involved the construction of 65 new dams and the rehabilitation of existing dams to increase the share of reservoir-dependent irrigation schemes from 11% to 20%, or 1.5 million hectares. While the project s relevance increased with the Government s new policy, the new focus also rapidly increased the local demand for dam related expertise. As a consequence, quality national dam experts were limited on the market, with implications on the quality of work produced, such as designs and special studies for selected dams. 23. Project restructuring. The closing date of the project was extended three times, first from 31 December 2013 to 31 December 2015, and then to 31 December 2016, and again to 30 June These extensions were needed to complete ongoing contracts that were delayed due to changes in the original design, a slow procurement process, delays in implementation of civil works 13, as well as complete preparations for the Additional Financing for DOISP 2 (P161514). Further, the target of the Project Outcome Indicator (i)(a), as well as Intermediate Outcome Indicator (a)(i) on dams to be rehabilitated was changed from 34 to 33 given one of the originally included dams experienced damage during implementation and required emergency repairs financed by Government budget. Finally, Outcome Indicator target (a)(ii) describing the number of Spillway Emergency Operation Plans publically accepted was changed from 34 to Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 24. Establishing a long-term, programmatic approach to dam safety. The Government had embarked on a programmatic approach to develop a portfolio management approach across its dams and reservoirs. Building on DSP, DOISP was envisaged as the first project to support this agenda. Unlike DSP, which focused solely on dam safety, DOISP was designed to also include 12 The vision and mission of the new Government Administration is described in the Nawa Cita, which translates into nine priorities. 13 See second and third restructuring paper, June

17 effective operation and management of dams. More extensive preliminary surveys, investigations and designs (SID) were started under DOISP to allow implementation in a successor project Lessons of former projects incorporated into preparation. Several lessons from dam safety projects in Indonesia and other countries, as well as from other preceding projects in the sector 15 have been incorporated into the design of this project. The experience showed the importance of three aspects: (i) proper documentation of pre-project status of dam safety; this had already been undertaken by the national Research Center for Water Resources for all large dams on Java; (ii) attention to both structural and non-structural remedial measures to ensure dam and reservoir sustainability; this is part of the project s design to also focus on institutional development; and (iii) putting in place effective mechanisms for budgeting and maintaining the level of O&M allocation and dam safety achieved at the end of the project. 26. Adjusting absorption capacity. The project was originally conceived as six-year Specific Investment Loan (SIL) with total loan financing of about USD 130 million. At appraisal, doubts emerged about the absorption capacity of the Ministry of Public Works for such an amount within the timeframe of one project. Hence the decision was taken to apply a programmatic approach with a more modest financial commitment of the first project of around USD 50 million, bringing the total project cost to around USD 70 million. The programmatic approach would then allow more flexibility to prepare the second phase through a successor project. A two-tranche Adaptable Program Loan (APL) was also considered at this stage but not pursued further given mixed experience in project performance due to political or logistical constraints that often prevented meeting the triggers necessary to advance the project to the next phase. 27. Risk score rating and projections of score reductions. The PDO indicator includes a reduction in the risk score for 34 dams (33 after restructuring), most notably aiming to reduce the number of high risk dams from 81% to <5%, and to achieve a reduction of the Overall Risk Index of >30% 16. The ICR team found several uncertainties and inconsistencies with this indicator. 28. First, based on discussions and documents reviewed for this ICR, this targeted level of reduction was found to be high, making it difficult for the project to achieve this target. For some dams, the necessary reduction in the risk score to achieve targets outlined in project preparation documents such as the Project Implementation Plan (PIP) or the Project Appraisal Document (PAD) was almost impossible to achieve. The modified ICOLD method used to calculate the score assesses certain types of hazards and weighs them accordingly 17. A certain risk exists due to the structure of the dam (volume of the reservoir, height of the dam), which cannot be changed through measures proposed under the project. Other risks originate from the size of downstream population to be evacuated (which in many locations is increasing due to population growth) or are associated with new or future downstream developments. Moreover, the risk cannot become zero, and the possible reduction is not proportional 18. Considering this, it can be concluded that the target of the 14 Discussion reported in several Aide Memoires of preparation missions. The preparations for possible future implementation were undertaken as Special Studies under Component E.g. the First Water Resources and Irrigation Sector Management Program (WISMP) 16 The ICOLD risk categories are: low (<15), moderate (16 45), high (46 75) and extreme (>75). 17 The 12 risk types considered in the modified ICOLD method are described in Annex There is a limit of how much each dam can reduce its risk score. E.g. a dam rated as extreme hazard (i.e. a score of >75) can only reduce the risk score by e.g. maximal 20 points, reducing its overall risk rating by only 20-30%. A dam rated as moderate hazard (i.e. a score of <46) may also be able to reduce the risk score by maximal 20 points, reducing its overall risk rating by 40-50%. 17

18 project outcome indicator 1 (a), which aimed at reducing the percentage of dams rated as high hazard from 81% to less than 5%, was overly ambitious and could not realistically be achieved under the project. 29. Second, the Project Outcome Indicator in the PAD did not clearly define the list of dams to which the percentages applied. The indicator suggests amongst others that the proportion of high risk dams should be reduced from 81% to <5%; but the method for establishing the baseline figure of 81% or the target figure of <5% were not clearly defined. Based on the information provided in the PAD and PIP 19, the 34 dams mentioned in the indicator description comprised of two extreme risk dams (6%), 31 high risk dams (91%) and one moderate risk dam (3%). The modelled reduction in risk score to be achieved under the project further projects that out of the 34 dams, 7 (20%) would be classified as high risk after the project intervention significantly more than the targeted 5%, which translates into 1.7 dams (i.e. 1 dam in absolute numbers), and not considering the point raised above that some of the projected reduction in risk score was unrealistic to be achieved. 30. Considering this uncertainty about the intended target percentages, the outcomes are assessed by disregarding the absolute numbers and only considering the difference in risk score intended to be achieved. For example, 81% minus 5% is a net reduction of 76 percentage points, or 26 dams. Comparing this target with the actual number of dams for which the risk score was reduced to moderate (18 dams) the project outcome can be rated as reasonably successful. 31. Third, another factor influencing the achievement of the Outcome Indicator (i) (a) targets were changes in baseline values for the PDO targets that were established during appraisal. Risk scores for dams are generally prone to fluctuations due to changing circumstances and conditions of the dam and its surroundings, such as a change in downstream population, damages to the structure of the dam, etc. This applies even more for the number of old dams in the portfolio, constructed several decades ago. Given that the physical remedial works at dams under the project were carried out in phases, works for some dams were carried out several years after the baseline scores were calculated. By then, the condition of several dams had changed and scores were found to be different than the baseline outlined in the PAD. This impacted the possibility of the project to achieve its PDO indicator targets of reducing hazard categories. 32. Commitment from client. The experience of the Dam Safety Project (DSP) confirmed that works on existing dam structures for safety and operational improvement are technically complex, new to the Indonesian Government and national contractors and thus require dedicated support for capacity building. This lesson has been incorporated into the project design, with better alignment of institutional development and execution of works to attune better to the absorptive capacity of government agencies. 33. Further, recognizing the linkage between the loss reservoir functionality and sedimentation import from upper watersheds, a catchment-based activity based on the principles of Integrated Water Resources Management (IWRM) are needed. The experience of projects like the Water Resources and Irrigation Sector Management Projects 1 and 2 (WISMP), as well as a variety of Trust Funds on applying IWRM principles in Indonesia have been incorporated into the design of this project. However, implementation of these programs are often complicated by institutional arrangements involving a range of different stakeholders, strict Government regulations on budget allocations and execution and the delineation of roles and responsibilities. 19 See Figure 1 in Annex 2, Risk Score for Indonesian Dams to be rehabilitated under DOISP 1. 18

19 34. Assessment of risks. The PAD identified a series of risks to a successful implementation of the project, the most critical of which were (i) Government commitment to reform and budget allocation for O&M requirements; (ii) limited capacity to assure quality of surveys, designs and implementation, as well as limited capacity of financial management staff; (iii) a lack of community participation; (iv) improperly done environmental and social assessments; (v) unfamiliarity with Bank guidelines and procedures. The overall project risk was rated moderate after mitigation. The risks assessment and mitigation measures were generally on target, and while some were more successfully mitigated than others, there were no unanticipated risks or missed opportunities for mitigation that seriously limited achievement of project outcomes. The quality of some achievements, however, was adversely affected by delays in contract management. 2.2 Implementation 35. Decentralized implementation. Based on the lessons from the previous Dam Safety Project (DSP), the design for rehabilitation and special studies was to be decentralized to the river basins instead of a centralized CPIU. However, this decision did not take into account the number of dam related experts on the market in Indonesia. So for approximately 8 10 design or study contracts each year, the number of highly-qualified national experts available either as individuals or through the firms turned out to be limited, leading to the selection of less qualified experts and ultimately impacting the quality of the work produced. Decentralized implementation also impacted the community related activities, which were supposed to be implemented by the provincial basin organizations, as per design. Given that only one or two dams per province were being included into the community program, it did not make sense from an administrative and financial efficiency point of view, to have each provincial basin organization conduct these activities themselves. Hence in all provinces except for Central Java, the community related activities were conducted by the PIU of the central level basin organizations. 36. Other issues that were not taken into consideration when decentralizing implementation to the basin-level were the complex implementation structure and the division of roles and responsibilities between the different levels of Government operating in one basin. O&M is the responsibility of either central or provincial level government, depending on the location, although maintenance works are often carried out by provincial or district agencies, through various forms of delegated arrangements. In the case of Indonesia s largest and most important dam, the Juanda dam in Jatiluhur, the division of roles and responsibilities is even more complex. As part of a cascade of three dams, different line ministries are involved. The National Electricity Agency PLN is in charge of two of the dams (Cirata and Saguling, not part of DOISP), the Ministry of Public Works (MPW) of Juanda / Jatiluhur. A state-owned enterprise (PJT) was created under the Ministry of State-owned Enterprises to be in charge of the profitable operations, while the maintenance works are the responsibility of the basin agency responsible (BBWS) under MPW. At the same time, the BBWS and the PJT arranged cross-support between each other to help with O&M activities, against an agreed fee. Since many of these arrangements are a mix between formal responsibilities of the agencies and non-formalized arrangements, it is complicated to initiate changes to these arrangements. Only recently, a joint coordination committee was established to coordinate the upstream hydropower operations and downstream flood control and bulk water supply operations. 37. A risk index for assessing dam safety. For the Risk Assessment, the project was required to use an appropriate Risk Assessment Method, as stated in the Results Framework. The scoring was done using the modified ICOLD Condition Rating Method. The modified ICOLD Method provided the project with an internationally recognized methodology, but it also tied the risk assessment into the constraints and shortcomings of that methodology (further described in Chapter 2.1 above). Especially the fluctuations of the risk scores between baseline in 2008 and a 19

20 reassessment of the score before the project works began (which has happened between 2011 and 2014) caused the score of about half of the dams under the project to start out with a higher risk score than envisaged in the baseline. This had an impact on the project s ability to achieve the targets. This issue may have been resolved early on if it had been addressed during the mid-term review. 38. Procurement and contract management. During implementation, delays occurred in the procurement process and contract management of large procurement packages for Basic Dam Safety Facilities 20 (BDSF) and for several dredgers. BDSF have a significant influence on dam safety and as such on the dam risk rating. However, a delay in bulk procurement of the facilities directly resulted in a delay in achieving the risk index targets. Similarly, the contract management for the supply of several dredgers experienced issues causing a delay in provision and installation. 39. The initial procurement arrangement for the BDSF was to divide it into five packages with an estimated value per package between USD 608,000 and USD 642,000. Based on this package value, National Competitive Bidding was agreed as preferred procurement method. Two packages were awarded in 2013 and installed at the sites while other packages were delayed. At the time of finalizing this report, the procurement of the remaining 23 BDSF was still in process. Following the delay, the Government changed the procurement method to International Competitive Bidding (ICB) and combined dams into regional packages in order to attract more foreign firms to the tender. It is expected that proper installation of the facilities can be completed in time, ensuring their functionality and training of Government staff for proper operation, as well as including standard guidance to ensure the compatibility among the different systems. 40. The contract management for the procurement of the dredgers encountered financial difficulties that caused a delay in the process. Despite provisions allowing for demonization of the contract in multiple currencies, the contract between the CPMU and the Indonesian supplier for delivery of the dredgers was signed in IDR. This imposed a heavy financial burden on the supplier after the Indonesian Rupiah (IDR) suffered from an estimated 30% depreciation against the US Dollar after contract signing. Rather than cancelling the contract and rebidding the contract was extended and the dredgers finally delivered to site in December The implications of these delays on the outcome indicator Reduction of Failure Risk are such that several of the dams will only actually drop to the lower risk category as targeted once the BDSF are installed and operationalized. 42. Mid-term review restructuring. The project experienced delays in the early stages of implementation and disbursement in , which caused physical works to fall behind schedule. The main reasons for the delays were the slow adjustment of implementation units to Bank procurement rules and a reorganization of the main implementing agency DGWR into new Directorates, and in connection with this, the transfer of the CPIU from the Directorate of River and Coastal to the new Directorate of O&M. Further, an ongoing internal investigation by the Institutional Integrity team (INT) for another project in Indonesia involved a company that was also the lead firm in an existing joint venture contract under DOISP. As a result of the investigation, that company was barred from new tenders, which impacted the representation of the international 20 Basic Dam Safety Facilities include measuring tools and gauges that measure structural changes of the dam, such as seepage, water height, embankment settlement, seismic intensity or reservoir water level. 20

21 partner in the joint venture and had implications on the performance of this contract due to the absence of key consultants, including the team leader, for an extended period of time. 43. From 2011, overall project performance improved significantly and it was decided during the mid-term review that the project would undergo a restructuring that included a revision of the intermediate outcome targets, the reallocation of loan proceeds and the extension of the closing date from December 2013 to November The restructuring was deemed a Level 2 given that the PDO did not change, and there were no changes to the safeguards or other arrangements. 44. Dam Portfolio Management. The newly established Directorate for Operation and Maintenance under the DGWR attributed more attention towards a dam portfolio management approach, underscoring the Government s commitment to the management of infrastructure and the importance of O&M for the sustainability of their assets. This commitment was reflected in an increase in O&M budget for dams over the following years: While in 2009, the government expenditures on O&M amounted to IDR 21 billion, by 2014 it had doubled to almost IDR 48 billion 22 an improvement compared to before, but still below actual O&M requirements. 45. Aside from the new Directorate, Dam Monitoring Units (DMU) were established in each river basin, tasked with systematically monitoring and reporting on the situation of each dam to the also newly established Central Dam Monitoring Unit (CDMU) at national government level. Following this overall increase in commitment to dam safety, support to the project CPIU also improved and project progress improved significantly after the restructuring. Disbursement increased, and overall Implementation Progress improved from Unsatisfactory (May 2012) to Moderately Satisfactory (February 2014), and to Satisfactory in December Dam Safety Panel. A Dam Safety Panel was constituted at the beginning of the project, but mobilization turned out difficult due to the large number of dams to be covered over a geographically dispersed area. Experts of various specializations 23 have visited a number of dams during the project, including an international expert specifically focusing on the installation of the Hollow Cone Valve at the Juanda dam in Jatiluhur. The panels tasks included checking the quality of remedial works, monitoring seepage and providing recommendations for guidelines and procedures that were in draft. Recommendations focusing on Standard Operating Procedures (SOP) for cascade dams under emergency operations procedures included institutional mechanisms for coordinated operations procedures, hydro-met monitoring and flood forecasting & early warning procedures as well as downstream protection and emergency preparedness plans. 47. Emergency Action Plans. With the passage of Government Regulation No. 37 on Dams in 2010, the development of emergency action plans (EAPs) to reduce potential impacts caused by dam failure became mandatory for large dams. With support of the project, DGWR has developed comprehensive guidelines which provide a basic structure for the development of EAPs. To date, DGWR and the river basin organizations have developed EAPs for about 80 dams, of which 56 have been financed by the project. A review of a selected number of plans indicates that the level of detail and the quality of EAPs varies widely, particularly in terms of the downstream inundation mapping and modeling, as well as in sensitization of downstream communities. In terms of operationalization of the EAPs, Indonesia has experienced one event (Way Ela dam in 2013) that 21 Project Restructuring Paper, August See also Chapter 1.7 Other Significant Changes. 22 Project Progress Report, CPMU Specialization of experts included hydro-mechanics, dam safety, hydrology, instrumentation, construction and management. 21

22 provides insights into the effectiveness of the EAPs. With lessons from that event, it will be beneficial to further strengthen coordination and implementation capacity of the involved stakeholders. This would require continuous coordination between the RBOs, local governments, and emergency management authorities to ensure that the EAPs are updated regularly, as well conducting simulation exercises to reinforce and test the EAPs to benefit the execution of the EAPs in the event of dam failure. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 48. M&E design: The objective and indicators were well designed to measure the project activities. Both PDO and intermediate indicators are reflecting the main objective stated in the PDO, i.e. increasing the safety of reservoirs and the strengthening the safety and O&M policies, regulation and capacity of the Government. The objective and indicators that were set-up at the design stage remained valid until ICR reporting, with the exception of the overly ambitious risk score targets. The means and method to measure the project achievements were well set-up. As part of the work of the preparation team, the baseline data of the risk score for 34 dams/ reservoirs was collected during appraisal and become a benchmark to measure the project s achievements. This baseline score, however, proved to be inaccurate for some dams some years later when physical works started and the score was re-assessed due to damages that had occurred in the meantime. These possible changes in the baseline were not taken into account for M&E design. 49. M&E implementation. M&E system was implemented as designed. The M&E system was put in place at the startup of the project. CPMU supported by TA consultants undertook internal monitoring quarterly. External monitoring was done independently by consultants at the mid-term and completion stages. At mid-term, a team of individual experts conducted independent monitoring on water resources, economic, and environment including cost and benefit analysis of the dams. Interim completion evaluation has been also prepared by individual consultants hired by CPMU. There is a minor discrepancy in the internal reporting (ISR) of the project achievements. The ISR result section only reports the number of dam portfolio risk assessments without highlighting the number of dam with failure risk reduction. Although it was not fully tracked in the ISR, the project s M&E continuously monitor the number of dam with failure risks reduction in line with the indicators set up in the loan agreement. 50. M&E Utilization. The data collected and the monitoring of progress and quality of project rehabilitation have assisted the project to: (i) Identify and remedy construction quality problems and to incorporate better controls. Up to 2011, the O&M budget was calculated using the Modern Equivalent Asset Value (MEAV) method, which resulted in insufficient O&M funds being allocated to each dam. After an independent monitoring study provided recommendations to MPW on how to calculate O&M demands, real needs of the planning figures were used to calculate the provision of O&M funds (AKNOP); and (ii) Identify the bottleneck in project management. 2.4 Safeguard and Fiduciary Compliance 51. Safeguards. Overall, the project did not create any significant environmental and social issues during its implementation. The project is a Category B project that triggered OP 4.01 on Environmental Assessment and OP 4.37 on Safety of Dams. In the Environmental and Social Management Framework (ESMF), the works under DOISP have been scoped so as not to include any works with major environmental or social impacts. While the project is in compliance, reporting on the safeguard implementation including reporting of Environmental Management and Monitoring Plan (UKL/UPL) for a number of subprojects was lacking. This is largely due to the absence of a safeguard focal point assigned in the CPMU. For the successor project, capacity building and more training on safeguard is required for implementing units in any levels. 22

23 52. Fiduciary. The project has complied with the World Bank s Procurement Guidelines and remained moderately satisfactory throughout project implementation. This is mainly due to replacement of key experts after contract signing and delay in the procurement process of major contracts: particularly the procurement of the Hollow Jet Valve replacement (CT-CW-2), as well as significant delay in the contract implementation for the supply of dredgers (BOP-GD-02). These resulted in delays in the project implementation schedule. 53. Overall, the project had an adequate financial management system with a moderately satisfactory performance rating. The project encountered some problems at the beginning of the implementation; disclaimer opinion on FY2010 project s financial statement, delay on IFR submission, delay of budget effectiveness, and weaknesses on payment verification. But the CPMU took follow-up actions as recommended by the auditors and the World Bank missions to improve the project s financial management performance. After 2010, the project consistently received unqualified audit opinions on its financial statements. At the time of closing, the project was able to submit its financial report timely and complete the follow-up of all audit findings, including completing the refund of ineligible expenditures to the designated account. One issue that remained until the end of the project is the continuous delay of budget effectiveness, delaying project implementation. Close coordination between CPMU, PIUs, and the planning bureau of the ministry, and having a financial management consultant within the CPMU who fully understands the budgeting processes, are pre-requisites for a timely availability of the budget for this type of project. 2.5 Post-completion Operation/Next Phase 54. Additional Financing under preparation. To ensure the safety of all existing dams and all newly constructed dams, the Government is continuing its programmatic approach to dam safety. Given the success of DOISP, the Government as next step requested to scale up the project to include support to the completion of the original dams and to further extend support in preparation for the expanded portfolio, resulting from the Government s ambitious development program of 65 new dams until Hence Additional Financing (AF) is currently under preparation to support this second phase (DOISP 2) scale up. The original components of DOISP will be retained under the AF. The DOISP2 AF will also scale up the portfolio approach developed under this project to further enhance the regulatory and administrative arrangements for dam management and safety. It will build on a rapid, simplified screening to be carried out under this project before the closing date of June This screening aims at defining the risk profile and identifying prioritized interventions under DOISP2. A range of analytical work is being carried out in parallel to the preparation of the AF, including a Public Expenditure Review on Public Spending for O&M, a study on benchmarking of dam safety within river basin organizations, amongst others. 55. Special Studies. During the preparation of DOISP, the 34 dams to be included for remedial works were part of a list of 63 priority dams as curated by the Government. A number of other dams on that list required works that could not be carried out without more in-depth special studies to determine the root causes for their reduced safety and inform detailed design of risk-reducing remedial works as well as necessary environmental and social safeguards requirements. 22 of these remaining special studies were included in this project to be carried out as preparation for the successor project (DOISP2). These 22 dams would then be the first number of dams to be addressed under the portfolio approach, while the remaining 115 major dams owned by the Ministry would undergo more in-depth analysis and prioritization before they are selected for remedial works. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 23

24 Relevance of Objectives Rating: High 56. The PDO of increasing the safety and the functionality with respect to bulk water supply of large Ministry of Public Works-owned reservoirs; and strengthen the safety and operational management policies, regulations and administrative capacity of Ministry of Public Works remained relevant throughout implementation. It was rooted both in the Government s Long-term Development Plan, of which several programs are related to water resources, including infrastructure development, food security, energy, climate change, environmental protection, health, and employment creation, among others. The Long-term Development Plan is segmented into fiveyear medium-term development plans (RPJMN), which translate the long-term goals into specific development priorities during that period. The first RPJMN placed emphasis on infrastructure development, sustainable development of natural resources and food security, among others; all of which are reliant on the effective and sustainable use of water resources. The Government has subsequently launched an ambitious program to develop 65 new dams, with around 13 already under construction. This program will improve storage capacity and overall resilience but will also increase the burden on Government capacity, making DOISP highly relevant. 57. The PDO is also aligned with the World Bank s Country Partnership Frameworks, both at the time of appraisal as well as at the time of the ICR. During appraisal in 2008, the PDO aligned with the CPS 24 focus areas of strengthening Indonesia s institutions and investments in (water) infrastructure, climate change and disaster mitigation. At the time of the ICR, the PDO aligned with the current CPF s focus areas Infrastructure Platforms at National Level and Collecting More and Spending Better 25. The project objective is especially relevant as the current Government further developed its vision of water and food security and translated it into an ambitious program of constructing 65 new dams within its legislative period In this context, support is becoming more relevant now than at time of appraisal given the rapidly increasing portfolio, competition over water resources, need for multi-purpose management and challenges of national capacity. Climate change scenarios are predicting more intensity of extreme weather, putting additional constraints on safe, secure supply of water. Relevance of Design Rating: Substantial 58. Following the first Dam Safety Project (DSP) in the 1990s, Indonesia embarked on a sector wide, programmatic reform of its water resources and irrigation sector. This project is thereby following a series of projects, laws and regulations in the broad water resources sector, with the aim of addressing its strategic dams and reservoirs at a portfolio level. This project was designed as the first of a series of projects to increase dam safety, functionality and prolong the life span of the asset. The design of DOISP aimed to be an evolution of the DSP, establishing a basic framework for dam safety at central and provincial levels and carry out remedial works. DOISP as the follow up project intended to consolidate the achievements of DSP and develop a portfolio approach for dam safety, as well as carry out minor remedial works as well as prepare more extensive surveys for more complex works to be done under a successor project. The DOISP design thereby aimed at advancing the Government s reform agenda and strengthen the performance of institutions responsible for maintaining dam safety. 24 Country Partnership Strategy FY09 FY 12, The World Bank Group, Country Partnership Framework FY16 FY20, The World Bank Group,

25 59. As described in chapter 2.1, the project s components were well designed to advance the PDO, and implementation arrangements were built on the lessons learned from DSP, emphasizing the need for a decentralized model of implementation in the course of Indonesia s decentralization laws that were passed in About 60% of the total project cost consisted of investments in infrastructure, studies, investigations and designs (SID) or dam safety instrumentation. The results indicators adequately measured the progress towards achieving the PDO with the exception of one of the indicators as already described in section 2.1 where the target was set unrealistically high. 3.2 Achievement of Project Development Objectives (Efficacy) 60. The Project Development Objective consists of two parts: (i) increase the safety and the functionality with respect to bulk water supply of large MPW-owned reservoirs; and (ii) strengthen the safety and operational management policies, regulations and administrative capacity of MPW. Overall, the Project Development Objective was largely achieved as described below. PDO part (i) Increase the safety and the functionality with respect to bulk water supply of large MPW-owned reservoirs. Rating: Substantial 61. The first part of the PDO was to be achieved by reducing the risk of dam failure for 33 dams 26 through physical remedial works, by preparing Spillway Emergency Operation Plans (SEOP) for 56 dams and by improving the Government s dam portfolio management program. Overall, the failure risks of all dams in the project have been successfully reduced (see Table 1 below as well as the information provided in Annex 2). The Outcome Indicator results show that the project has successfully reduced the failure risk of all extreme hazard dams to high ; the failure risk of 20 out of 31 dams from high to moderate ; and the overall hazard of dams under the project by 25%.The number of extreme hazard dams in the portfolio has been successfully reduced to 0%, as set out in the indicator target. The number of high hazard dams in the portfolio has been reduced from 31 to 13 (38%), thereby falling short of achieving the target of reducing 27 high hazard dams to moderate hazard set out in the PDO indicator 27. The overall hazard of the 33 dams has been reduced by 25%, falling slightly short of the target of >30% set out in the PDO indicator. The other targets of part (i) of the PDO indicator, namely improved DGWR Dam Portfolio management and the Dam Safety Unit (DSU) completing >75% of its priority dam inspection and certification load has been achieved. 62. The reasons for the shortcoming of the reduction of high hazard dams from 81% to <5% has been described in Chapter 2.1 above: The original indicator targets were unrealistic. Considering the type of works done in the time and with the resources available, the overall hazard reduction of 25% can be considered a significant achievement of the project. At the time of this report, implementation was still ongoing, and with finalization of several outstanding works 26 The target number of dams in the outcome indicators was changed from 34 to 33 with the first restructuring due to damage that occurred to one dam (slope sliding) of the initial list of 34, causing it to be rehabilitated urgently using Government budget. Later, another dam was rehabilitated instead under the project, leading to a total number of 34 rehabilitated dams despite the revised target of As shown in Table 1 below, the baseline of 34 dams at appraisal included two extreme risk dams, 31 high risk dams and one moderate risk dam. Out of the 31 high risk dams, 20 were reduced to moderate risk leaving 11 dams in the high risk category; two dams, however, were added to the high risk category after their extreme risk was reduced to high risk, reaching a total of 13 high risk dams. 25

26 (specifically the installation of the Basic Dam Safety Facilities BDSF), the risk score is expected to be reduced further for several dams by a few points. 63. The Spillway Emergency Operation Plans were prepared as part of Emergency Action Plans for 56 dams 28 under the project. Dam portfolio management improved as can be seen by an increase in O&M budget between 60% - 120% compared to 2009, ongoing training of more than 650 staff and dam operators, hydrological maps on probable maximum floods for different regions and quality assurance systems as well as by the establishment of a functional Central Dam Management Unit and Dam Monitoring Units in the river basin organizations. 29 Table 1: Detailed results of Project Outcome Indicator number (i) and its sub-indicators a-c. No. as Project Outcome Indicator and At Re-calculated At ICR Target per RF sub-indicators. appraisal baseline (i)(a) Number of dams for which dam failure risk should be reduced (i)(a)a. Number of extreme hazard 2 (6%) 2 (6%) 0 (0%) 0% dams (i)(a)b. Number of dams at high hazard = 13 (38%) From 81% to <5% Number of moderate hazard dams = 21 (62%) From 0% to >95% (i)(a)c. Overall hazard reduction 25% >30% Note: The number of high hazard dams at ICR consists of 11 originally high hazard dams plus 2 originally extreme hazard dams. The overall number of dams reduced from high hazard to moderate hazard is 20. This Project Outcome Indicator corresponds to the Project Outcome Indicators described in the Results Framework for PDO Part (i) (See Table A2-1 in Annex 2) and is also listed as Intermediate Outcome Indicator No. 1 in Chapter F of the Data Sheet. PDO part (ii) Strengthen safety and operational management policies, regulations and administrative capacity. Rating: Substantial 64. The second part of the PDO was to be achieved through issuing Government Regulations, Ministerial Decrees and enhancing administrative procedures and capacity. Overall, this has been achieved. A Government regulation on dams was issued during the implementation of the project, regulating overall management of dams and its safety procedures. Further, 33 guidelines and manuals were prepared and staff were trained accordingly. The number of guidelines and manuals and its status prepared under the project is listed below: Table 2: Overview of Guidelines and Manuals drafted and issued. Guidelines and Manual Number Remarks Ministerial level 4 4 approved Directorate General Level approved 16 in deliberation Others (Risk Assessment} 5 5 approved Note: Deliberation refers to ongoing internal Government consultations among different departments. 28 Originally, it was envisaged to prepare 33 Spillway Emergency Operation Plans. This number was increased to 56 during restructuring realizing SEOPs could be prepared under the Emergency Action Plans. 29 Annex 2 provides more detailed information on the results achieved. 26

27 65. Administrative procedures and capacity was enhanced by training key staff as well as establishing a Dam Monitoring Unit (DMU) in each of the Balais as well as a Central Dam Monitoring Unit (CDMU) in DGWR. The DMUs monitor dam safety and regularly report on it to the CDMU. Their reports are the basis for priority maintenance activities as well as maintenance budget planning. Table 3: Overview over achievements disaggregated by PDO and PO Indicators. PDO Project Outcome Indicator Achievem ent Remarks Part (i): increase the safety and the functionality with respect to bulk water supply of large MPW-owned reservoirs; Part (ii) strengthen the safety and operational management policies, regulations and administrative capacity of MPW. Indicator (i) (a): Reduction in dam failure risk of 33 rehabilitated dams using an appropriate Risk Assessment Method and Achieving operational Spillway Emergency Operation Plans for these dams. Extreme hazard dams reduced from 3% to 0%; High hazard dams reduced from 81% to 5%; Overall hazard of 33 dams reduced by >30% of Risk Score Indicator (i) (b): DGWR dam portfolio management improved by established: Annual dam O&M funding program; Dam human resources development program; Dam hydrology program; Dam quality assurance program; Functional CDMU Indicator (i) (c): DSU has completed >75% of its priority dam inspections and certification load (including 2-3 large mine tailing dams) Indicator (ii) (a): Government Regulation on Dams and Ministerial Decrees issued Indicator (ii) (b): Administrative procedures and capacity enhanced Largely achieved. Achieved. Overachieved. Achieved. Achieved. 34 out of 33 dams rehabilitated; Extreme hazard dams reduced to 0%, high hazard dams reduced to 38% (target: 5%), overall hazard of 34 dams reduced by 25.3% (target >30%) SEOP achieved for 56 dams (target: 34) See also Table 1 94% of dams inspected out of a target of >75% 3.3 Efficiency Rating: Moderate 66. The economic rationale of dam rehabilitation and safety improvement is to ensure safety and reduce risks of dam failure and consequently prevent or minimize damages to downstream property and environment, loss of life, loss of reservoir supply and replacement costs of the dam or construction cost of alternative water supply. An economic and financial analysis was prepared at the time of appraisal, taking into account certain assumptions on dam failure probabilities, financial data on public expenditures and currency exchange rates at that time. For this report, an economic analysis was also prepared, following the methodological approach of the appraisal stage. More detailed information including assumptions used for the analysis can be found in Annex 3. 27

28 67. The results of the economic analysis indicate a positive Net-Present Value of US$ 56.3 million as well as Internal Rate of Return of 28.7% at 6% discount rate. This is less than assumed at appraisal (NPV US$108.2 million, IRR 32.8%), due to a smaller overall reduced risk score achieved than originally targeted. 68. It is noteworthy that dam safety is unique in that it represent an extreme situation characteristic of a low probability event, but with high consequences or hazard. Thus, conventional rules of economic analysis are difficult to extrapolate and extend to such events. For instance, human life is generally accepted as the most important consequence and often dominates dam safety decisions. Unfortunately, the confidence with which life loss can be estimated is low, which is a hindrance to credibility and value of dam safety risk assessment results 30. However, application of cost benefit analysis is still relevant as there are insufficient disposable financial resources to improve the safety of every dam without limit. The analysis is particularly useful for setting priority or determining the order in which dam safety rehabilitation projects should be approached within a portfolio of dams. The extremely low probabilities of dam failure masks the risks. However, when the failure actually happens due to neglect or lack of necessary measures, the consequences are usually very costly, as evidenced by the recent two dam failure events in Indonesia itself The project was extended three times to a total of 3.5 years beyond initial closing date. The reason for these extensions was due in part to: (i) initial delays in implementation, (ii) reorganization of the implementing agency, (iii) issues around procurement and contract management, and (iv) safety concerns with regards to weather conditions Given that the economic indicators remain below the value calculated at appraisal, and given the time extension of altogether 3.5 years, efficiency has been rated as moderate. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 71. Given the high relevance of the PDO, the substantial relevance of the design and implementation, the substantial achievement of the PDO to date and the planned activities throughout the final year of implementation 33, and given the moderate efficiency, the project has been rated as moderately satisfactory. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 72. The implementation of Community Participation (CP) piloted in 20 schemes for reservoir greenbelt and 4 schemes for watershed management program was running during 2011 to Loss of life was not used in the quantitative calculations for the economic analysis. 31 In 2009, the Situ Gintung dam collapsed due to a failure to implement appropriate structural and nonstructural dam safety measures (98 casualties). In 2013, the Way Ela dam collapsed due to a hill landslide; but after the experience of Situ Gintung four years prior, government agencies were better prepared with early warning systems and awareness among communities. As a consequence, no human life was lost, only three people went missing and structural damage could be reduced although the storage volume of the Way Ela dam is more than 40 times bigger than that of Situ Gintung. 32 The third extension of 6 months was mainly caused by weather related delays in installation of two Hollow- Cone Valves in Indonesia s largest dam. Due to the El-Niño phenomenon the dry season of 2016 experienced unusually high rainfall, thereby causing higher water levels in the reservoirs than planned. This impacted the installation of the valves, which had to be delayed due to safety concerns. 33 At the time of submission of the ICR, negotiations for the proposed Additional Financing of DOISP2 had been successfully concluded. 28

29 Though still limited in scale, there are positive impacts achieved by the program in terms of community awareness to maintain dam safety and conserve the watershed area, as well as impacts on social and economic life of the community both directly and indirectly. The economic impact of the program on communities has been less visible, mainly because income generating activities proposed through the community work plans (RKM) were limited in scope, and have not yet been implemented in some areas. With some improvements, in particular strengthening community institutions, recruiting capable facilitators and support from relevant stakeholders, the CP program initiated by DOISP is worth continuing in the successor phase of the project. 73. The project did not have a specific poverty or gender focus. (b) Institutional Change/Strengthening 74. The long-term program of support to dam safety, from the Dam Safety Project, over several water resources projects to the current programmatic approach to dam safety, has helped the Government establish a comprehensive institutional framework that positions well in advance of large increase in the dam portfolio through the planned and ongoing construction of 65 new dams until The project had a clear focus on strengthening institutions with regards to the overall dam safety portfolio in Indonesia. Achievements included issuing a Regulation on Dams 34. It further included establishing Dam Management Units in each river basin organization, in charge of systematically monitoring on the condition of dams in the respective Balais. For the most part, these DMUs have become important and well-functioning entities in the Balais. Further indicators of successfully strengthening institutions are the increase in O&M budget, as mentioned above, the capacity building of Government officials, and the establishment of a Central Dam Management Unit, overseeing the overall dam safety situation in the country Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops No beneficiary survey or stakeholder workshop has been conducted for this report. 4. Assessment of Risk to Development Outcome Rating: Moderate 75. A number of risks to the development outcome exist, none of which rank as Significant or High. The main risks include: 76. The biggest long term risk remains the lack of a dedicated revenue stream to support O&M due to insufficient water user fees. Although farmers are the biggest user of bulk water from reservoirs, they do not pay user fees for water provided through irrigation systems, as per the Indonesian constitution. Revenue streams for dam operators are therefore limited to fees from other water users, such as water utilities or hydropower operators. The revenues generated through these streams are limited so that the budget for O&M of dams relies heavily on Government transfers. This increases the risk of continuous provision of sufficient funds to ensure dam safety and functionality. The project provided assistance to the overall management of O&M in a programmatic way, and while the O&M budget for dams has increased during the course of the project it continues to remain vulnerable to shifting priorities. Further mitigation measures to this risk provided by the project included preparing O&M plans, new regulations, establishing an own Directorate for O&M within the Ministry of Public Works and Housing, establishing a Dam 34 Government Regulation No. 37/2014 on Dams was issued as an umbrella for national dam safety policy and management 35 This was still in progress at the closing date for this report. 29

30 Monitoring Units at the river basin and central level, and providing training on O&M to dam operators. The Additional Financing will further ensure the continuation of Government commitment to improving the adequacy and appropriateness of O&M in the sector. 77. Coupled with the financial risk of O&M is the risk associated with incentives for new dam construction rather than for sustainable rehabilitation or O&M of dams. Especially the Government program of constructing 65 new dams until 2019 has implications on the availability of funds, experts on the market and priorities within the Government agencies to deliver this ambitious agenda and at the same time provide sufficient support to dam safety measures and regular O&M activities. Along with the construction of new dams comes the commitment for continuous recurring expenditure requirements for O&M, which have to be taken into account for future planning and budgeting. The institutional results achieved under the project will be key to ensuring dam safety and functionality. The expected results under the AF will further support this aim. 78. Aside from the financial constraints, a risk to development outcome is also caused by a shortage of skilled human resources available for dam safety overall. Following a zero-growth policy for the civil service over the past few decades, limitations in qualified personnel continue. More frequent rotations throughout the civil servants career paths translate into longer time to gain relevant experience for the small number of new staff designated to dam safety in the ministry. The decentralization reform of 2001 has furthermore shifted responsibility for dam related works to central government, creating a gap between mostly older, experienced local-government staff in the field, most close to retirement, and junior to mid-career professionals with only limited experience to fill the existing positions. With the large portfolio of new dams, the growing need for skilled personnel to carry out O&M, and with additional positions created by establishing new institutions under this project (CDMU, DMU or Directorate for O&M), the risk pertains that sufficiently qualified persons can not be found to ensure dam safety for all dams in the future. 79. Uncertainties further exist with regards to the continuous support of communities in the vicinity of the dams and reservoirs to the safety of the dam. The support of the communities depends on many factors, including the economic hardship forcing them into income-generating activities with negative impact on the dam/reservoir, or a lack of understanding of direct and indirect impacts of their actions on dam safety. This risk could be exacerbated by an inadequate proactivity from the Government agencies tasked with community engagement. Within the project, the Community Participation program was undertaken with good results, achieving positive impact for both the dams as well as well as the communities. It is proposed to be continued in the AF project DOISP2. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 80. The Bank team was well experienced and knowledgeable drawing on a long standing relationship with GOI on water resources in general, and on the development of dam safety policies and institutional capacity in particular. The Government agencies responsible for dam safety were very motivated in improving the functionality and safety of its dam portfolio, and the Bank provided sound advice and assistance in this process. Good preparation using lessons from national and international experience was fundamental for the successful implementation of the project and the acquired knowledge proved to be a worthy asset during its execution. The complexity of the implementation arrangements were, however, slightly underestimated, leading to delays in implementation after project effectiveness. 30

31 81. The Results Framework was well designed in capturing the PDO and well aligned with the components. Issues appeared with regards to the high targets to be achieved under the PDO indicator (i)(a). Further, the extension of 3.5 years indicates an underestimation of the initially designed project duration of 4.5 years. (b) Quality of Supervision Rating: Moderately Satisfactory 82. Bank supervision took place on a regular basis, providing appropriate and well-targeted advice and observations. The aide memoires provided evidence of regular supervision and professional advice given by the Bank s experts throughout the project. Timely response to client request was regularly provided, and adjustments were successfully made to reorganization of the client s implementation agency DGWR. Despite a frequent turnover of Task Team Leadership, expert advice was provided regularly to support the client in a range of upcoming issues. Nonetheless, substantial delays in the procurement and installation of some works packages occurred. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 83. The quality at entry and the quality of supervision are rated Moderately Satisfactory, thus, the overall Bank Performance is rated Moderately Satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 84. The Government visibly supported the Project given that it is in line with a long-term commitment in the sector, and the first of several projects in a programmatic approach to dam safety. The project was implemented as a national government project, involving only nationalgovernment budget. The overall policy guidance was provided by the National Steering Committee for Water Resources (NSCWR) in Bappenas, which has been providing the same useful oversight to other World Bank financed projects in the sector. Implementing agency was the Directorate General for Water Resources (DGWR) in the Ministry of Public Works. 85. Reorganization of the DGWR led to the establishment of a special Directorate for Operation and Maintenance, to which the Implementation Unit for this project was moved. This reflects an increased commitment from the Government towards allocating resources to maintaining water resources infrastructure assets the basis for a dam safety program. With the achievements of the majority of the project s targets, the increased commitment to O&M and the good progress in the Community Program, the Government has shown its commitment to the project. Cooperation between DGWR and the Bank was satisfactory throughout the Project s execution. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 86. As mentioned, the implementing agency DGWR was well committed to the programmatic dam safety approach. The advice given and the actions agreed upon in the aide memoires were mostly followed up in due time. Along with creating a Directorate for O&M, the Government also increased allocated funding to O&M in the sector, issued Dam Regulations and associated manuals 31

32 and guidelines, and followed through with numerous capacity building and training opportunities under the project. 87. Shortcomings include insufficient safeguards management due to the absence of a safeguards focal point, lack of sufficient attention to management of contracts as well as fiduciary performance. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 88. Given the satisfactory performance of the Government, and the Moderately Satisfactory Performance of the Implementing Agency, the overall Borrower Performance is rated Moderately Satisfactory. 6. Lessons Learned 89. Mechanisms of measuring achievements should include only factors/variables that can be influenced by the project. The modified ICOLD method was used to measure risks to dam failure. While this method is an internationally accepted methodology, it includes several items that are not imposing an increased hazard, still add to the overall risk score, but are difficult or impossible to influence under the scope of a project. Examples are the business risk to the dam owner in case of a failure, the potential damage to downstream population in case of a failure, downstream asset development or structural specifications such as dam height or reservoir volume. This can be problematic if the results indicator is requiring the score to pass a certain threshold into a lower overall risk score as success measure. It is recommended to carefully consider PDO indicator targets that take into account which factors the project can influence and utilize more disaggregated scoring targets that are not aiming at a lower risk category but a certain reduction in the total score for each dam. 90. The identification of and participation by the local government agencies in the Community Participation Program (CP) is essential to the effectiveness and sustainability of the initiative. CP activities were to be implemented by the river basin organizations (Balai) through local facilitators in selected pilot locations to prevent and mitigate the impact of sedimentation on reservoirs. While the facilitators effectively fulfilled their contract requirements, they received little tangible guidance from the Balais, who did not have community engagement as part of their basic tasks and functions. The limitation to certain selected sites further limited the scale up of the program for those Balais eager to do so. While the CP program under the project was successful in engaging communities with maintenance, dam safety and watershed management, leadership of Balais remains crucial for effectiveness and sustainability of the initiatives. To improve the CP program in the successor project, it should be ensured that the local government agency responsible for CP has the resources and skills required, and also receives appropriate training to provide effective guidance to the facilitators carrying out the activities; that it includes dam maintenance through the sedimentation management aspects, involving communities in land care education and using market-based mechanisms for ecosystem services by local people. 91. The benefits of decentralizing management responsibilities to the subnational level can be eroded if there is insufficient capacity to meet the increased demand. The decision to decentralize several of the activities under the project to sub-national Balais was aimed at avoiding some of the lessons from the predecessor project DSP. However, it did encounter several constraints during implementation, such as a lack of resources and experts on the market to carry out a number of designs and studies within a short period of time; or to provide resources towards 32

33 small scale pilot activities in geographically dispersed locations. In order to attribute activities to implementing agencies at the right levels of government, their capacity to carry out the tasks should be well examined, and support should be provided from central level in case there are constraints or shortages. 92. The weakness in the design of contracts can cause significant delays, with negative implications on meeting overall project targets. Significant delays occurred in the procurement of some packages and contracts under the project, especially the procurement of the dredgers, the hollow cone valve as well as the BDSF. The reasons include financial difficulties due to the depreciation of the Indonesian Rupiah, exacting management of ongoing contracts or the design of procurement packages. A limitation of specialized expertise on the local market exacerbated these constraints. Overall, it can be concluded that weaknesses in the design of the contracts as well as overall contract management can cause significant delays with implications on meeting overall project targets. It is recommended that especially factors such as currency exchange rate developments, or the size of procurement packages for similar goods and services be examined carefully and the task team provide thorough assistance on the process to the PIUs. Going forward into scaling up the overall project approach to the larger dam portfolio in Indonesia, it is further recommended to carry out a market analysis of available experts to ensure the procurement plan can be carried out in a realistic time frame. This is especially important as the Government is currently constructing 65 new dams throughout Indonesia, which does require a large number of specialized experts and imposes even further constraints on the availability of these. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 93. Comments were received by the borrower during several discussions. In principle, the borrowers agreed with main observations conducted and lessons learned suggested by the Bank team. Further, initial lessons of the ICR have been used to inform the preparation of the Additional Financing of the successor project DOISP A summary of the borrower s comments can be found in Annex 5. 33

34 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Components Appraisal Estimate (Bank Finance + Counterpart fund) (USD millions) Actual/Latest Estimate (Bank Finance + Counterpart fund) (USD millions) Percentage of Appraisal Component % Component % Component % Component % Component % Total % (b) Financing Source of Funds Type of Cofinancing Appraisal Estimate (USD millions) Actual/Latest Estimate Percentage of (USD Appraisal millions) Borrower % International Bank for Reconstruction * 85% and Development *Note: This amount reflects the actual expenditures made under the project as of January The amount differs from the disbursed amount indicated in the Data Sheet (Section A: Basic Information: USD 47.02M), which reflects the amount disbursed by the IBRD into the GOI s Special Account. 34

35 Annex 2. Outputs by Component Results Framework Table A2-1 below provides the Results Framework as it was outlined in the Loan Agreement. Table A2-1: Results per Outcome Indicator Project Development Project Outcome Indicators Objectives (as per loan agreement, including amendments) Part (i): To increase the safety and functionality with respect to bulk water supply of large MPW-owned reservoirs; Part (ii): Strengthen the safety and operational management policies, regulation and administrative capacity of MPW Indicator (i) (a): Reduction in dam failure risk of 33 rehabilitated dams using an appropriate Risk Assessment Method and Achieving operational Spillway Emergency Operation Plans for these dams. a. Extreme hazard dams reduced from 3% to 0%; b. High hazard dams reduced from 81% to 5%; c. Overall hazard of 33 dams reduced by >30% of Risk Score Indicator (i) (b): DGWR dam portfolio management improved by established: a. Annual dam O&M funding program b. Dam human resources development program c. Dam hydrology program d. Dam quality assurance program e. Functional CDMU Indicator (i) (c): DSU has completed >75% of its priority dam inspections and certification load (including 2-3 large mine tailing dams) Indicator (ii) (a): Government Regulation on Dams and Ministerial Decrees issued Indicator (ii) (b): Administrative procedures and capacity enhanced Progress as of December 2016 a. 2 dams (6%) reduced from extreme hazard to high hazard. b. 20 dams reduced from high hazard to moderate hazard, reducing the high hazard risk from 81% to 38% c % a. Ongoing for 58 dams b. Trained staff by DSU in 28 dams, by TA CPIU in 64 dams, international training in 2 batches c. Ongoing for 33 dams d. Available and ongoing e. Established dams inspected by TA CPIU; - Priority dams (special study) not yet certified - 25 dams non-doisp have been certified - 47 dams under safety review PP 37/20103 on Dams Issued: 10 ministerial regulations to be issued On-going in 4 PIU Directorates of DGWR and 10 PIU-Balais / B(B)WS 35

36 Table A2-2: Results by Intermediate Outcome Indicators Intermediate Outcomes Intermediate Outcome Indicators Outcome (a) Outcome (a) The functionality, safety and (i) Dam Portfolio Risk economic life of restored large Assessment indicates that MPW-owned dams is improved functionality of about 33 MPWowned dams is restored, safety and sustainable, while basic operational safety of un-repaired risks of at least 22 dams greatly dams has been kept at a reduced and that the risks of 23 manageable level during the un-restored dams (under Project. successor project) are still manageable; (ii) Spillway Emergency Operation Plans (SEOPs) publicly accepted and warning system communications functional for 34 dams. (iii) Economic Lifetime expectation increased by 5%. (iv) Bulk water supply capability restored for >25 dams. Outcome (b) (i) DGWR aware of the level of serious sedimentation of important large dams and can formulate a mitigation program to prolong economic life & ensure dam safety (inclusive of decommissioning planning if necessary). (ii) (Sub-)catchment management and improvement carried out reducing erosion on community participation Outcome (c) DSU adequately staffed as per its DOISP Priority Implementation Plan and, national consulting firms and contractors certified by INACOLD as eligible for dam design and works. Outcome (b) (i) Reservoir Survey of 30 or more severely sedimentimpacted large dams completed and structural and non-structural sedimentation mitigation program for these dams is ready for execution. (ii) At least 25% of the households in the pilot areas join and participate. Fifty percent of erosion-prone sites in and along rivers in the (sub-) catchment identified and improved. Outcome (c) Fifteen structural and/or functional certified DSU staff, and a total of 15 private engineering firms and contractors certified. Progress as of October 2016 (i) 34 dams rehabilitated under the project. 23 Special Studies prepared. (ii) 56 SEOPs prepared (iii) Economic lifetime has been extended through repairs and improved functionality, but the exact percentage was not available in time for this report. Based on discussions with the teams, it is estimated that expected economic lifetime has been increased by more than 5%. (iv) Bulk water supply capability was restored through restoration of spillway functionality, remedial works and improved emergency operating plans, altogether for more than 25 dams (i) Done for 31 dams. (ii) 37% of households participated. 15 staff recruited and trained, but none certified as certification regulations changed during implementation to no longer be under Government responsibility. Preparations have been conducted to facilitate certification. 36

37 The tables and figures in Annex 2 below provide additional information about the outputs, achievements and results of the project. The information provided in the tables and figures may partially be overlapping with indicators described in the Results Framework above as well as the Data Sheet, but are not intended to be of identical format. The information below is rather intended to provide an additional, more detailed understanding of the project s overall achievements and results in form of. Information is taken from project documents, such as the PAD, the PIP or project progress reports, as well as depict the author s own design. Results by Component Table A2-3: Results by Component Progress of Outcome Indicators (as of October 2016) Project Component and Outcome Indicators Component 1 Units Target Target (after restructuring) Achieved Dams Rehabilitated Dam Achieved. Remarks Basic Dam Safety Facilities (BDSF) installed Dams Delays in procurement process. Remaining BDSF currently retendered in different package format. Special Studies Dam Prepared and submitted for review and clearance. Improve operational hydrology, review flood frequency, assess spillway capacity and downstream risk and preparation Component 2 Preparation of O&M plans (O&M manuals, instrument monitoring & data processing, SOP, NBB) Emergency Action Plan (EAP) Staff Training for O&M and Safety Monitoring Community Participation (CP) Component 3 DOISP Reservoir Survey Study for decommissioning of Dam Included in the scope of work of the Emergency Action Plans (EAP) and Special Studies. Dam Submitted for review. Dam Finalized and included in special studies where applicable. Dam Training has been conducted. Dam Community MOUs signed, 2 dams under PJT I not pursued further Dam Included in scope of work of EAP and Special Studies Pack Cancelled as no dam was to be decommissioned 37

38 severely sedimented reservoir MOU between community group and dam management in pilot area Percentage of HH Joined the CP Component 4 Preparation of regulatory documents (guidelines and manuals) on dam safety Awareness campaign and training on regulation and guidelines Strengthening and equipping DSU and CDMU Establish and support Dam Safety Panel DSU staff recruited and training INACOLD Certification of Consultant Firms and Contractors INACOLD Certification of Dam professionals Catchment MOUs signed, action plans partly completed. % % of households have joined in watershed management. % Ministerial Regulations related with dam safety and management were finalized, another 29 related guidelines and manuals were also prepared, 24 of which are already completed. % trainings on dam safety management conducted and attended by 651 staff from River Basin Organizations (Balais). Various seminars on better dam management were carried out. % Completed Pax international and 5 national panel members Pax Completed, although rotation caused some of the staff to move. Nos During implementation, regulations changed that certification is now awarded by the Association of Consultants; INACOLD helped preparing TOR Pax Dams under DOISP Number of Dams under DOISP: 63 Emergency Action Plans (EAP) finalized: 56 Dams with Physical Works: 34 Basic Dam Safety Facilities (BDSF) installed: 10 (23 ongoing) Dams with risk reduced from extreme to high: 2 Dam with risk reduced from high to moderate: 20 Special studies: 23 Reservoir Survey: 31 Figure A2-1: Overview of dams included in DOISP and selected results achieved. 38

39 Table A2-4: Risk score per dam. For calculation of scores see Table A2-5. Score classes: Low risk 0 15; moderate risk 16 45; high risk 46 75; extreme risk No. Name of Dam Risk Score (modified ICOLD method) Targeted Achieved reduction in risk score (%) Achieved risk category reduction Baseline corrected Target Latest of risk score baseline (expected available score in score in PAD) score PAD (%) before (Oct works 2016) 1 Way Jepara Moderate 2 Batutegi High 3 Merancang High 4 Tiu Kulit Moderate 5 Pengga Moderate 6 Batujai Moderate 7 Kalola Moderate 8 Salomeko Moderate 9 Nglangon Moderate 10 Simo Moderate 11 Sanggeh Moderate 12 Cacaban Moderate 13 Penjalin High 14 Gembong Moderate 15 Tempuran High 16 Greneng Moderate 17 Kedung Ulin High 18 Pacal High 19 Ketro High 20 Cengklik Moderate 21 Krisak Moderate 22 Plumbon Moderate 23 Gondang Moderate 24 Wonorejo Moderate 25 Wlingi Moderate 26 Karang Kates High 27 Sengguruh Moderate 28 Selorejo Moderate 29 Sampean Baru Moderate 30 Jatiluhur High 31 Darma High 32 Malahayu High 33 Wadaslintang High 34 Sempor High TOTAL 35.4 % 25.3 % 39

40 Table A2-5: Guidance to determine risk scores of Indonesian dams based on modified ICOLD method. (Taken from Project Implementation Plan, 2008). Risk Class Extreme High Moderate Low Example Contribution to Risk (Appropriate weighting is given in brackets) Reservoir Capacity (ML x 10 3 Extreme High Moderate Low 30 x 10 ) 3 ML Dam Height (m) Evacuation Requirements (No. of Persons) (Amended as per Reference 3) Potential Downstream Damage (to existing structures) Business Risk to Owner arising from Dam Failure (Additional Consequence to ICOLD) Addition Availability of al Factors for Existing Dams Addition al Factors for Addressi ng Structure Deficien cies construction and Maintenance records. Availability of processed instrumentation And surveillance records. Level of effort extended in previous safety > 120 (6) Extreme > 45 (6) Extreme >250,000 (12) High (12) High (6) 120 to 1 (4) High 45 to 30 (4) High 250,000 to 10,000 (8) Moderate (8) Moderate (4) 1 to 0.1 (2) Moderate 30 to 15 (2) Moderate 10,000 to 1 (4) Low (4) Low (2) < 0.1 (0) Low < 15 (0) Low None (0) None (0) None (0) (4) 20 m (2) Moderate (4) Low (4) High (6) None Low Moderate High Low (3) (2) (1) (0) (2) None Low Moderate High None (3) (2) (1) (0) (3) None Low Moderate High None Evaluations. (3) (2) (1) (0) (3) New or future downstream development. High Moderate Low None Moderate Impact of Deficiency on Failure of the Dam Flood capacity related deficiencies Static stability related deficiencies Earthquake resistance related deficiencies (3) (2) (1) (0) (2) Extreme High Moderate Low (6) (4) (2) (0) (18) (12) (6) (0) (12) (8) (4) (0) Moderate (2) High (12) Moderate (4) Risk Score for the Dam Under Consideration. 48 Risk Class (I, II, III, IV) - Ref. pp.41 of ICOLD Bulletin No. 72. III (High) Risk Class I (Low) II (Moderate) III (High) IV (Extreme) Risk Score

41 Figure A2-2: Risk Score for 63 dams under DOISP1 as of preparation. The 34 rehabilitated dams are on the left side, between Krisak and Tiu Kulit. Source: Project Implementation Plan (2008). 41

42 Billion Indonesian Rupiah Development of O&M budget Annual budget for Operation and Maintenance Figure A2-3: Budget disbursed for Operation and Maintenance (O&M) under the Directorate for O&M in DGWR, Ministry of Public Works and Housing. Figures for 2016 reflect allocated budget. 42

43 Annex 3. Economic and Financial Analysis Background In Indonesia, many dams suffer from operational deficiencies, while some dams have structural and mechanical problems that could become a safety hazard. Some dams are prematurely nearing the end of their economic life; others have lost a large proportion of their reservoir capacity. As a result, the volume of water storage in many reservoirs is decreasing as well as the irrigated areas. The project: supported operational and physical improvements of 33 dams; promoted structured O&M program by ensuring adequate dam O&M budget funding; initiated a program to address watershed erosion and sedimentation established sustainable institutional arrangements and program for community participation in reservoir management; and strengthened national dam safety assurance institutions The economic rationale of dam rehabilitation and safety project is to ensure safety and reduce risks of dam failure and consequently prevent or minimize damages to downstream property and environment, loss of life, loss of reservoir supply and replacement costs of the dam or construction cost of alternative water supply. The primary beneficiaries of DOISP are both urban and rural communities of 8 million people (about 2 Million households dependent on reservoirs for their water supply and livelihood as well as all downstream communities, who could be placed at physical and/or environmental risk if dam safety is compromised. Methodology The ICR stage economic analysis followed the appraisal stage methodological approach. Dam failure probability weighted benefit losses were estimated for without project and with project scenarios to generate incremental benefits. The assumptions used in the economic analysis are summarized in table A3.1. The sensitivity of the results to key uncertain or relevant variables (such as dam failure probability reduction assumption) was analyzed. It was difficult to estimate the probability of failure of any given dam unless extensive and detailed historical data exists on a dam portfolio, which is commonly not the case in developing countries. At both appraisal and ICR stage, the dam failure probability is assumed to be 13% over a period of 20 years. In the base case, this failure probability was assumed to decline by 90% at appraisal stage and by 35% at ICR stage (targeted) following project interventions. The ICR stage dam failure probability reduction assumption is set at 25.3%. This is the mean reduction in Risk Score of 34 dams assessed by SMEC during project preparation in 2008 and by the CPIU TA in October 2016 for with and without DOISP scenarios. Table A3.1: Assumptions for economic analysis No Items At appraisal At ICR 1 Dam failure probability without the project 13% 13% 2 Dam failure probability reduction with the project 90% 25.3% 3 Total Irrigation service area (ha) 601, ,113 4 Cropping Intensity Annual cropped area (ha) 901,668 1,021,

44 6 Gross margin (US$/ha) Total Annual Net Production Value (US$) 712,317, ,294,759 8 Proportion of Total Annual Net production 30% 30% protected 9 Averted farm income loss (US$) 213,695, ,188, Number of houses 1,997,500 1,997, Value of house per unit (US$) Total Value of Houses (US$) 3,595,500,000 3,594,600, Proportion of the value of houses protected 10% 10% 14 Averted loss of value of houses (US$) 359,550, ,460, Total potential losses avoided (US$) 573,245, ,648, Discount rate 12% 6% Costs The actual and estimated cost build-up of the project is displayed in table A3.2. It shows that the project encountered significant delays in implementation. The project has disbursed so far about 93.8% of the total eligible financial resources. Delays in implementation and inability to utilize the full funds allocated lowers the returns to the project. For the cost calculations, Government contributions to the project have been considered, as well as O&M spending amounting to approximately US$ 1.1 million per year. Table A3.2 Estimated and actual disbursement schedule Year Disbursement at Appraisal (US$ Million) Disbursement at ICR (US$ Million) Total Benefits The primary benefits included: (i) reduced risks of dam failure; (ii) prolonged life of dams and reservoirs and restored performance, and strengthened institutional capacity for asset management; and (iii) mitigated watershed erosion and sedimentation. The improvements both of institutional/operational and physical nature that reduce the risk of dam failure will consequently prevent or minimize damages to downstream property and environment, loss of life, loss of bulk water supply, and eventual (expensive) replacement of a dam or for construction cost of alternative water supply. The analysis quantified only benefits from avoided farm income and household asset losses. Other real benefits such as avoidable loss of hydropower, expenditures for replacing or rehabilitating the dams, and emergency response costs in case the dams fail were not included, mainly because these were not considered at the appraisal stage. For instance, the 32 dams produce 516,216 MWh of hydropower energy. 44

45 Results of the economic analysis The results of the economic analysis for the ICR and appraisal stage is summarized in table A3.3. The analysis confirms that the project is economically viable as predicted at the appraisal stage. However, the absolute magnitude of the returns at ICR stage is lower than that assessed at appraisal stage. The major reasons for the discrepancy are: The substantial difference in assumption regarding reductions in the dam failure probability with the project interventions, and Delays in implementation that affected disbursement schedule and the resulting cost and benefit build-up. Table A3.3: Summary Results of Economic Analysis for DOISP Indicator Value at Appraisal Value at ICR 12% Discount rate 6% discount rate NPV (US$ Million) IRR (%) Sensitivity Analysis Analysis of the sensitivity of the economic viability of the project indicate that the results are robust. The project would not be viable, if only potential losses of farm income is considered as benefit. Table A3.4 Summary Results of the Sensitivity Analysis for DOISP No Variables At Appraisal At ICR NPV IRR (%) NPV IRR (%) (US$ Million) (US$ Million) 1 Counting potential losses of farm income and household assets (base case) 2 Counting only potential losses of farm income 3 Counting only potential losses NA NA of household assets 4 50% reduction in dam failure probability 5 25% reduction in dam failure probability 6 20% reduction in dam failure probability

46 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility Lending Guy J. Alaerts Lead Water Resources Specialist EASRE Task Teak Leader Ilham Abla Senior Operations Officer EASIS Alessandro Palmieri Lead Dam Adviser GWADR Bisma Husen Procurement Specialist Vivianti Rambe Environmental Specialist Trithep Lohavisavapanich Economist Theodore Herman Consultant (Dam, Institutional) Amien Sunaryadi Senior Governance Specialist Lis Nainggolan Consultant (Social Safeguards) Unggul Suprayitno Senior Financial Management GGODR Specialist Farida Zaituni Consultant EASIS Supervision/ICR Marcus Wishart Senior Water Resources Specialist GWADR Task Team Leader Paulus van Hofwegen Senior Water Resources Specialist GWADR Task Team Leader Ilham Abla Senior Operations Officer EASIS Task Team Leader Guy J. Alaerts Lead Water Resources Specialist EASRE Task Team Leader Xiaokai Li Senior Water Resources Specialist Task Team Leader Cuong Hung Pham Senior Water Resources Specialist GWADR Martin Albrecht Water and Sanitation Specialist GWADR Satoru Ueda Lead Dam Specialist Bisma Husen Senior Procurement Specialist EASRP Enggar Prasetyaningsih Procurement Specialist Andrew Daniel Sembel Environmental Specialist EASIS I Gusti Ngurah Wijaya Financial Management Specialist Kusuma Unggul Suprayitno Senior Financial Management GGODR Specialist Budi Permana Procurement Specialist Angelia Budi Nurwihapsari Procurement Specialist Flavio Chaves Consultant (Environmental Safeguards) Ninin K. Dewi Consultant (Social Safeguards) GSURR Tatong Permana Anggrimulja Consultant (Financial Management) Krisnan P. Isomartana Environmental Specialist Theodore Herman Consultant (Dam Institutional) Irama Aboesoemono Consultant (Community Participation) Achmad Affandi Nasution Consultant (Good Governance) Regassa Ensermu Namara Senior Water Resources Economist GWADR 46

47 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of staff weeks USD Thousands (including travel and consultant costs) Lending FY FY FY FY Total: Supervision/ICR FY FY FY FY FY FY FY FY FY Total:

48 Annex 5. Summary of Borrower's ICR and/or Comments on Draft ICR The borrower prepared a completion report in April 2015 and submitted a draft to the Bank team for information. It included an overview of the activities and achievements of each component as well as the results of a survey among the project s PIUs on issues and challenges they encountered during implementation. The recommendations provided in the borrower s ICR can be summarized as follows: The procurement process of the project partly inhibited timely implementation. Given the sometimes lengthy processing of loan proceedings and the procurement process, as well as a lack of familiarity and capacity of some PIUs with the Bank s procedures and regulations, some PIUs decided to utilize their own budget for remedial and rehabilitation works instead of the loan money available. This was the especially the case for urgent rehabilitation works that needed to be carried out due to damage that had occurred or an imminent risk the structure was posing. It is recommended to provide assistance to participating sub-national government PIUs to familiarize themselves better with Bank procedures and guidelines, and enable them to access loan proceedings in line with their planning and budgeting cycle. Procurement plans should take into account limitations and constraints imposed by simultaneous activities carried out under the project. Specifically with regards to limitations in the number of experts available in the market in one area at a time puts constraints on the possibility to follow a diligent procurement process and implement works in time. If a series of works have to be undertaken in a short period of time within a geographic location, these constraints should be taken into account and implementation periods should be adjusted accordingly. This is especially valid considering the amount of specialized dam-related works carried out outside of the project s scope, within the Government s program of constructing 65 new dams until Implementation arrangements should be designed to take into account skills, capacity and resources as well as the designated roles and responsibilities within the decentralized environment in Indonesia. Especially activities community participation program in individual locations, turned out to have high transaction costs for local agencies to conduct, and were therefore done by central government agencies, who in turn lacked experience and access to the community level. Subnational agencies such as provincial river basin organizations (Balais) do have more experience and more direct access to local communities, but have limited resources for engagement in dispersed locations that involve high transaction costs to access. Further, they would benefit from more thorough technical guidance from central level to conduct their activities. Community Participation Activities should take into account boundaries within which implementing agencies can operate. This involves specifically those activities, which have the potential to cause controversies when carried out by agencies that have no clear mandate for it. Despite memoranda of understanding or similar forms of agreements, government agencies with a technical mandate voiced their hesitation to spend time and resources on activities or purchase goods provided under the community participation program, which go beyond their description of responsibilities and duties. Examples for this is the provision of roads or assets for local businesses in return for the communities role in monitoring dam safety. The agencies involved mainly report a fear of consequences from auditors. This should be taken into account when designing the future community participation program. 48

49 Annex 6. List of Supporting Documents Project Appraisal Document Indonesia, Dam Operational Improvement and Safety Project (DOISP) (2009) Loan Agreement and amendments ( ) Borrower s Project Implementation Plan and annexes (2008) Annual Work Plans ( ) Monitoring and Evaluation Reports Project Annual Progress Reports Borrower s Project Completion Report (2016) Aides Memoire (various, ) Implementation Status Reports (various, ) Mid-term Review (2012) World Bank Indonesia Country Partnership Strategy / Framework, various, Indonesia National Long-term Development Plan and Mid-term Development Plans , ,

50 MAP 50