Actual Project Name : Second Eastern. Project Costs (US$M US$M): Sector Board : Transport Cofinancing (US$M US$M): US$M): ICR Review

Transcription

1 Public Disclosure Authorized IEG ICR Review Independent Evaluation Group Report Number : ICRR Project Data: Date Posted : 08/03/2012 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Country: Indonesia Project ID : P Appraisal Actual Project Name : Second Eastern Project Costs (US$M US$M): Indonesia Region Transport Project L/C Number: L4744; L7765 Loan/Credit (US$M US$M): Sector Board : Transport Cofinancing (US$M US$M): Cofinanciers : Board Approval Date : 06/22/2004 Closing Date : 06/30/ /30/2011 Sector(s): Roads and highways (92%); Sub-national government administration (4%); Aviation (2%); Ports waterways and shipping (2%) Theme(s): Rural services and infrastructure (29% - P); Infrastructure services for private sector development (29% - P); Urban services and housing for the poor (14% - S); Other accountability/anti-corruption (14% - S); Decentralization (14% - S) Prepared by : Reviewed by : ICR Review Group: Coordinator : Peter Nigel Freeman Robert Mark Lacey Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: The Project Development Objectives (PDOs) of the Second Eastern Indonesia Region Transport Project (EIRTP-2), which were identical in the Project Appraisal Document (PAD) and Loan Agreement, were to: 1. Improve the condition of the strategic road network in order to reduce transport costs and enhance regional accessibility; 2. Support decentralization of planning and management responsibilities for civil works on provincial and kabupaten (i.e.district) roads to their respective governments; and 3. Increase the efficiency, quality and transparency of civil works procurement and implementation. b.were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project consisted of the following three components : Component 1. Civil Works for Preservation of Roads and Transport Facilities (appraisal estimate US$ million; actual US$ million). This comprised five sub-components to cover rehabilitation (of pavement surface - not full-depth reconstruction), betterment (upgrading) and bridge replacement of (a) national inter-urban, (b) national urban, (c) provincial, and (d) kabupaten roads, and a fifth sub-component for (e) improvements to facilities serving interfaces between road and non -road transport services, such as small sea ports, ferry terminals and local airports. Component 2. Implementation Support (appraisal estimate US$ million; actual US$ million). This

2 component covered consulting services and a small amount of equipment required to support implementation of the project to enhance quality and technical efficiency of the works, as well as provision of funds for incremental operating costs for the Project Management Unit and the Directorate of Technical Affairs in the Directorate General of Highways, and for the conduct of technical audits by the Inspectorate General of the Ministry of Public Works (MPW). Component 3. Road Sector Capacity Building (appraisal estimate US$ 8.70 million; actual US$ 2.62 million). This component consisted of technical assistance and training to strengthen the planning, management, and financial capacity of responsible agencies and authorities in the road sector in the EIR, with an emphasis on support for the ongoing decentralization process. The technical assistance in this component consisted of (i) extension of an initiative started under the preceding project (EIRTP-1) to develop and strengthen planning, programming and budgeting procedures, with particular emphasis on providing support for kabupaten staff for maintenance planning and management; (ii) further support for establishment of the road fund and the road board started under EIRTP -1; (iii) expansion of the traffic and road asset monitoring systems started under EIRTP-1 to cover provincial and local roads; (iv) improving management of environmental and socio-cultural impacts of road development in environmentally sensitive areas; and (v) development of a medium-term (10 to 15 year) Kalimantan Road Network Development plan, taking into account the likely future roles of waterways and coastal shipping, the need for institutional capacity building in local institutions, and the policy initiatives needed to assist regional development and transport provision in an environmentally fragile region. Training and training-related consultant services were provided under this component to (i) investigate the feasibility of institutionalizing the training needs analysis function in local government; (ii) review and revise five training courses, deliver seven existing courses which were not included in EIRTP -1, and develop two additional interactive computer training courses; (iii) update the Training Management Information System; and (iv) provide support to enable about 40 roads officials to complete relevant Masters degree programs offered by leading Indonesian universities, and for 26 road officials to complete a special Masters degree program administered by the Ministry of Public Works in collaboration with the Asian Institute of Technology (AIT). The above amounts exclude contingencies and taxes (US$ million). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost The appraisal estimate for the cost was US$ million including a front end fee of US$2 million and provision for contingencies and taxes of US$ million. The final cost was US$ million including a front-end fee of US$1 million; the loan was fully disbursed. Financing IBRD provided a loan of US$200 million of which US$ million was disbursed and the balance cancelled. Borrower Contribution The Borrower's estimated contribution was US$89.89 million of which US$84.32 million was disbursed. Dates The project was originally expected to close on June 30, 2009, but was extended twice, first by 21 months due to project restructuring and by a further six months to complete outstanding works. The main elements of the restructuring were (i) cancellation of several technical assistance (TA) activities in the Road Sector Capacity component, (ii) revision of the measurement methods of a few performance indicators, and (iii) reallocation between disbursement categories. The principal purpose of the reallocation, which included nearly $ 28 million in previously unallocated funds, was to shift project resources in response to the government s updated strategic plan for maintenance of national roads. As a result, project resources for sub -national roads were reduced by about 23 percent (to $81 million) while resources for national roads more than tripled (to about $77 million). 3. Relevance of Objectives & Design: a. Relevance of Objectives: Relevance of Objectives High. All three of the PDOs - enhancement of regional accessibility, decentralization support and efficiency, and quality and transparency improvements - were and still are highly relevant to the country environment and national strategies. The Country Partnership Strategy (CPS) , current at closure, aims to improve existing Government programs, strengthen the institutions involved (both state and non-state), and encourages replication where possible. EIRTP-2 was consistent with the current CPS priorities throughout design and implementation; it fostered a competitive private sector and supported Indonesia s infrastructure investment. b. Relevance of Design:

3 Relevance of Design Substantial. The results framework provided a clear and logical causal chain to ensure that the project s components and specific activities would be linked to the achievement of the PDOs in support of the Government s development priorities and sector strategy. However, the quality of the framework would have been improved had it better accommodated the varying capabilities of the kabupaten. 4. Achievement of Objectives (Efficacy): Improve condition of the strategic road network in order to reduce transport costs and enhance regional accessibility Modest. In terms of output, the original target was to improve about 3,120 km of strategic road rehabilitation and betterment, the actual output was 1,682 km (54 percent), while out of the original target of 2,342 m of bridges and bridge work, the actual output was 1,644 m (76 percent). The shortfall was mainly due to the steep increase in construction costs during the implementation period, but also to increased unit costs, because of worsening road conditions during the longer than expected implementation period. The intermediate performance indicators were also only partly achieved. Sub-component 1.5, transport facilities serving to interface between road and non -road transport services, was not implemented (see 3b above). Sub-component 1.5, transport facilities serving to interface between road and non-road transport services, was not implemented because the scope was outside the field of expertise of the implementing agency. In terms of outcome, the reduction of transport costs and enhancement of regional accessibility on the improved roads in all categories was achieved and in some cases exceeded : e.g. 19 percent Vehicle operating cost (VOC) reduction on provincial roads against a target of 11 percent; 39 percent VOC reduction on kabupaten (district) roads against a target of 25 percent, and 52 percent travel time improvement on selected project roads against a target 20 percent. However, less traffic benefitted because of the reduction in scope. Support decentralization of planning and management responsibilities for works on provincial and kabupaten roads to the respective provincial and kabupaten governments. Modest In terms of outputs, one of the intermediate performance indicators was the number of agreed technical assistance (TA) sub projects completed. The revision of the original target from seven to four was made in the Project Restructuring of June 2009 because of changing circumstances. For example, neither the Ministry of Finance nor the Ministry of Home Affairs supported the need for a commercially managed Road Fund, while the Directorate General of Highways decided that the proposed traffic and road asset monitoring systems were premature in the lower technological environment of provincial and kabupaten roads and that the existing manual technology should be retained. The TAs covering improving the management of environmental and socio -cultural impacts of road development in sensitive areas and the Kalimantan Road Network Development Study were completed. However, the two remaining TAs which were to directly support the participating sub -national governments, i.e. management support for decentralized institutions, and consolidation of the planning, programming and budgeting procedures for road sector support, were not implemented. In the former case no budgetary support was forthcoming from the Ministry of Home Affairs and in the latter case a lack of time and appropriate staff led to cancellation. In terms of achieving outcomes related to decentralization support, the project originally intended to measure the performance of participating provincial and kabupaten governments with outcome indicators such as number of provinces using the planning, programming and budgeting priority setting procedures (originally developed under EIRTP-1) and the number of kabupaten with adequate funding for routine maintenance of the network. However, by 2009 it was clear that these indicators could not be measured on an ongoing basis or at closing due to ambiguity as to how the measurement of the indicators would be defined, as well as technical and capacity issues. Despite this, many of the participating sub-national governments were able to implement the intended civil works program in each district, supported by the guidance and periodic training provided by the Directorate General of Highways and the consultants. To supplement the missing data for indicators, supplemental performance evaluation data were collected in terms of timeliness, documentation and successful completion. An index was created on which a score of 70 percent was considered the threshold for acceptable performance. The data reflected wide variation in the management performance of the various sub -national governments, with an average score of about 62 percent for provinces and 66 percent for kabupaten. Only about half of the participating provinces and kabupaten attained the 70 percent threshold. Increase the efficiency, quality and transparency of works procurement and implementation. Modest It was a challenging task to manage more than 150 civil works procurement cases, most of the procurement experienced substantial delays. The transparency of works procurement was improved by the Anti Corruption Action Plan, but its implementation was moderate. The outcome indicators show that 98 percent of bid openings were

4 attended by one or more Community Representative Observers (CROs), but the CROs only submitted final reports documenting their findings in about half of the cases. Effective empowerment of civil society in an Anti Corruption Action Plan requires sufficient budgeting to cover reasonable costs, but the funds provided were insufficient. The Inspectorate General carried out technical audits for 43 percent of the sub-projects, against a target of 52 percent. The project experienced significant delays in implementation, and the loan closing date was extended twice. This was mainly due to delays in the sub-national contract packages because of the complications caused by the expiration of the existing on-grant agreement (described in 3b above) in March This problem was eventually solved through alternative arrangements that allowed project implementation to continue, but the situation reduced procurement efficiency. 5. Efficiency: The estimation of ERRs for this project was challenging because of the large number of links for different road types in different states with varying road conditions and traffic usage. At both appraisal and completion, a "representative" package was used. Although the packages were similar, they were not identical because of changes in the intervening years, affecting inter alia, lengths of road treated and the type of improvements actually made. At appraisal the Indonesian Integrated Road Management System and the Kabupaten Road Planning Procedures were used, but this methodology was rejected at completion in favor of the HDM -4 model. Comparisons between appraisal and completion show apparent improvements at completion in the ERRs for main and provincial roads, but a decrease for kabupaten roads for percent of the network sampled. Since there are too many variables it is not possible to say with certainty what the ERRs actually were, although based on the ICR review evaluator's judgement it is likely that there was an overall improvement. However, taking into account the substantial delays in this project, the much higher than expected costs, the nonperformance of most TAs and the difficulties with procurement efficiency is, at best, modest. a. If available, enter the Economic Rate of Return (ERR ERR)/Financial Rate of Return (FRR FRR) at appraisal and the re-estimated estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal ICR estimate No No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: With high relevance of objectives, and substantial relevance of design, but only modest efficacy of all three objectives and of efficiency, overall outcome is assessed as moderately unsatisfactory. The project clearly had significant shortcomings in the achievement of its objectives. The scope had to be reduced and two important TAs were not implemented. The Ministry of Home Affairs should have been a key player and was not fully involved or committed. a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The reduction of transport costs and enhancement of regional accessibility achieved by the improved road conditions may quickly be lost if local governments lack the knowledge, capacity, commitment, or funding to carry out routine maintenance, particularly for sub-national roads. Although the project contributed to improving the project management capacity of these sub -national governments, they still have relatively weak capacity for road network planning and management due to the failure to implement the technical assistance projects relevant to decentralized institutions. Similar risks for the higher order roads are less because the institutional capacity is stronger. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry:

5 The Bank designed the project with relevant and timely objectives and assigned a qualified team with an appropriate skills mix. Based on the previous Bank experience with EIRTP -1, the economic, environmental and social aspects were reasonably well appraised, although the method of estimating the ERR was questionable. While the task team ensured that the PDOs were consistent with Government and Bank strategies, the objectives were over ambitious given the complexity of dealing with such a large number of local authorities. The "on-grant" financing agreement between the national and sub -national governments -- which required 100 percent up-front financing by the sub-national governments to be refunded later by the national government -- proved to be problematic since many of the local governments lacked sufficient funds to provide the full portion of the contract amount. This was more significant in respect of poorer sub -national governments. The team did not anticipate the lack of commitment by the Ministry of Home Affairs that diminished the achievement of the second PDO related to decentralization support, particularly the failure the carry out the crucial planned TA activities. Costs were poorly estimated, and insufficient account was taken of the problems associated with measuring performance in a plethora of different local governments. Components outside of the scope of expertise or responsibility of the implementing agency have very little chance of successful implementation unless agreed up front with the Ministry concerned and supported with appropriate training. The Ministry of Home Affairs was responsible for management support for decentralized regions, but the Ministry was not accountable for any part of the project, which led to delays and finally cancellation. Quality-at at-entry Rating : Moderately Unsatisfactory b. Quality of supervision: The supervision team was well-balanced and experienced. Formal supervision missions were carried out, on average, twice a year, except during 2007 when the TTL was based in the Country Office and carried out supervision on an ongoing basis without formally designated missions. The mission aide-memoires were comprehensive, and actions were appropriately discussed, which helped the Implementing Agency (the Directorate General of Highways) to formally address key issues with sub -national governments and other agencies. The environmental and social safeguards specialists actively participated in site visits, and played an important role in ensuring safeguard compliance. An exceptionally large number of geographically dispersed procurement packages were executed, albeit with some delays, and the physical investments were mostly implemented despite weaknesses in project preparation. The transparency of civil works procurement was also enhanced. However, the ICR points out (page 17) that the frequency of site visits was insufficient considering the size of the civil works component (153 contract packages) especially since missions were often combined with reviewing another project, and civil works quality issues were not sufficiently highlighted or monitored. The task team could have been more proactive in anticipating the possible failure of the decentralization support TAs, and in discussing and suggesting alternative ways to support decentralization capacity building at an earlier stage of the project. The team should also have insisted on the early engagement of the required implementation support services, prior to the physical works implementation, to prepare for the design and bidding processes, and to properly supervise the implementation progress and quality. Independent monitoring by a third party could have been encouraged. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government released the funds and provided counterpart funds in accordance with the loan agreement. The National Development Planning Agency (Bappenas) stayed engaged throughout implementation by participating in the mission discussions, and took a lead in addressing inter -ministerial issues. Inter-ministry coordination, however, was ineffective in the implementation of Management Support for Decentralized Institutions, which was a critical TA for decentralization support to achieve the project objectives. The Ministry of Home Affairs, which was responsible for implementing MSDI, failed in the procurement of the TA after several attempts, and failed to budget the final consultant selection process, which disabled the implementation of the TA. The failure of both the planning, programming and budgeting procedures and the Management Support for Decentralized Institutions

6 was largely caused by a lack of ownership; the national government did not exercise appropriate responsibility in taking the lead in the decentralization support aspects of the project. The on-grant agreement, which necessitated the sub-national governments to pre-finance 100 percent of the works was unpopular and in many cases unaffordable. This was later amended by the national government after the expiration of the on -grant agreement in March 2009 and replaced with a more acceptable arrangement. This suggests there should have been more discussion with local governments up front. Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: Despite serious challenges posed by the need to consolidate a number of subnational governments and the technical complexity of the project, the overall performance of the implementing agency, the Directorate General of Highways and its Project Management Unit, resulted in the satisfactory physical implementation of most of the 153 road and bridge sub-projects completed before closure. The Directorate also took the initiative in a timely manner to propose, negotiate, and help process the project restructuring and closing date extensions. The Directorate suggested solutions to the issue of the expiration of the on -grant agreement, which ultimately allowed for the continuation of the sub-national civil works packages and was a major factor in ensuring their completion. The Project Management Unit and the consultants made an effort to meet the requirements of the Anti -Corruption Action Plan, by providing guidance to the participating sub -national governments on matters of governance, fiduciary, and safeguards compliance. However, this was insufficient to attain final reports from the community representatives from half of the governments concerned. On the other hand, some local governments took a very positive and proactive approach to the safeguard issues, which in many cases were fairly new to them (for example, in West Kalimantan, where the local government satisfactorily addressed a complaint from a local NGO regarding a quarry location near a protected national park ). Several overdue complaint cases related to sub-national packages were concluded before the closing date with close coordination and guidance from the Directorate general of Highways. However, while the Directoratedid a good job of implementing the sub -national civil works component, its leadership in ensuring the implementation of the decentralization support technical assistance fell short of expectations. Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: Design: Achievement of the PDOs was measured by three main indicators : (i) vehicle operating costs (VOC) and travel time savings to measure the improvement of the condition of the strategic road network; (ii) provincial governments adoption of economic-based priority-setting procedures and kabupaten governments allocation of maintenance funding to measure decentralization support; (iii) compliance of designated bidding procedures to measure the efficiency, quality and transparency of works procurement, supported by the physical output measurements as intermediate indicators. These indicators were logically linked with the achievement of the PDOs, but the M&E system was inadequately designed since it was too complex and impractical for the local authorities involved and they had limited capacity for effective data collection. The vehicle operating cost and travel time savings data needed a large scale survey, and could be collected only at the completion of the packages. There were thus no baseline data for these indicators. b. M&E Implementation: Implementation: Because of the unusually large number of packages, it was considered more practical to conduct a comprehensive survey of vehicle operating cost and travel time savings at the end of the project. The designated data for decentralization support indicators were difficult to collect. The original intention of the provincial indicator was to measure the extent of application of economic-based priority-setting procedures which were introduced to the provinces under the planning, programming and budgeting procedures technical assistance component under EIRTP-1. This procedure was proposed to utilize the Integrated Indonesian Road Management System adapted to provincial government requirements or a new system which was expected to fulfill similar objectives. However, the planning, programming and budgeting procedures services under EIRTP -1 were involved in adjusting the Road

7 Management System for use at the central level, but this was not concluded, and eventually the measurement of this indicator became too imprecise and could not effectively represent the progress of the decentralization support. The collection of data for kabupaten governments allocation of maintenance funding was also probtlematical due to the large number of participating kabupaten governments with varying conditions and capabilities. These indicators could have measured the outcome of decentralization support better by directly assessing the project management capacity of each regional government. c. M&E Utilization: Utilization: The assessment of decentralization support needed supplemental data derived from measuring management performance of the provincial and kabupaten governments. Because of the challenges in collecting such data, and the fact that some aspects were designed only to be measured at the end of the project, opportunities to utilize M&E for project management purposes during implementation, and also for policy decisions and other uses, were limited. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: EIRTP-2 was classified as Category B. According to the PAD, safeguards policies on Environmental Assessment, Natural Habitats, Cultural Property, Indigenous Peoples, and Involuntary Resettlement were triggered. The ICR reports that compliance was considered by the Bank's safeguards specialists to be satisfactory throughout implementation. An Environmental Management Plan (EMP) and a Land Acquisition and Resettlement Policy Framework were agreed upon for the project. The ICR (page 7) reports that there were no significant deviations or waivers from the Bank s safeguards policies and procedures. Environment : The Environment Assessment Policy (OP/BP/GP 4.01) was applied satisfactorily. There was an Environmental Assessment at appraisal and the EMP included environmental screening procedures for all sub -projects. Those with high enough potential impacts that they would have required a full Environmental Impact Assessment (EIA) were not eligible for financing under the project. Sub-projects determined to require only the less intensive instruments of a site-specific Environmental Management Plan and Environmental Monitoring Plan were considered consistent with the project s Category B rating and were therefore eligible. The majority of the screened sub-projects fell under the category of standard operating procedures, whereby it was determined that they would only have impacts as a result of the construction process, and these were dealt with through the technical specifications in the works contracts. Natural Habitats (OP 4.04, BP 4.04, GP 4.04) and Cultural Property (OPN 11.03, OP/BP 4.11): The Bank s safeguard policies on Natural Habitats and Cultural Property were not mentioned specifically in the EMP, but both were covered under Government Regulation No. 27 of 1999 and the EIA. On investigation, neither policy was found to be applicable to any of the sub-projects. Social: Indigenous Peoples (OD 4.20): Two sub-projects required social impact assessments (SIAs) because they involved issues related to isolated and vulnerable people. One was located in East Nusa Tenggara Province and one in Papua Province. There were no significant impacts of the project on indigenous communities at either project location. The SIAs and Indigenous Peoples' Development Plans (IPDPs) were developed by the Government through appropriate consultation with the relevant parties, including the affected communities, and cleared by the Bank. The most critical aspects of the IPDP were implemented prior to the signing of the contract with the contractor for the sub-projects concerned. Documents and plans were disclosed on both the Borrower and Bank websites. Involuntary Resettlement (OD 4.30): Twelve sub-projects involved land acquisition and resettlement. Only one required a full Land Acquisition and Resettlement Action Plan and for the rest only a simple plan was produced. Land acquisition and resettlement were satisfactorily implemented. There was one sub-project that needed further remedial actions, for which the local government made an extra effort to prevent one project -affected person from becoming worse off as a result of the land acquisition activity. This was well implemented, and documented in the aide-memoires. All relevant plans were documented and disclosed on the borrower and Bank websites. b. Fiduciary Compliance: Fiduciary : The fiduciary aspects covered an Anti -Corruption Action Plan and its implementation, as well as procurement and financial management activities. There were some moderate shortcomings in these areas, but, according to the ICR, they did not prevent the timely and reliable provision of information or negatively affect project

8 implementation or outcomes. Anti-Corruption : Implementation of the Anti-Corruption Action Plan was marginally satisfactory since it was insufficiently budgeted. For EIRTP-2 it was structured on the basis of six key elements; (i) empowerment of recipients, (ii) empowerment of civil society for oversight and feedback, (iii) establishment of procurement policies to mitigate collusion, (iv) building strong task teams with effective tools, (v) a complaints handling system, and (vi) clear definition of remedies. The implementation of the plan has contributed to the intended transparency by the effective disclosure of public information, and through the community representative observers and complaints handling mechanism. The third party monitoring faced difficulties in selecting the right organizations, but it produced results in the end. A Complaints Handling Secretariat was established in January 2007, and functioned under the Project Management Unit (PMU). Initially only substantial complaints were forwarded to the Inspectorate General (IG) of MPW, but later the IG requested all the complaints to be discussed with them. A total of 21 complaints were received, out of which 20 had been resolved at the time of preparation of the ICR. The one remaining complaint was about the payment delay by a contractor,which was settled by the end of Procurement : The main procurement challenge under EIRTP-2 was the exceptionally large number of civil works contract packages (sub-projects) across the many participating provinces and kabupatens. Delayed approval of budgets caused some suspensions of construction activities on site and extensions of contracts at the end of 2009/early A number of packages required rebidding due to unsuccessful bidding or malpractice in the bidding process. Bids were announced through local and national newspapers. The national road contracts were announced through the Directorate General of Highway's website, and all international competitive bidding contracts were announced through DG Market and UNDB Online. Major delays were experienced, however, during the supervision consultants' selection process, which took more than two years to conclude, and affected the implementation of the civil works component. There were a number of variations (technical design, quantity change, etc.) to signed contracts. Financial Management : The ICR reports that the project s financial management system, including accounting, budgeting, internal control, funds flow, financial reporting and auditing, was acceptable to the Bank. Overall, Financial Management Reports were submitted on time. The audit findings reported several weaknesses in internal controls particularly in the areas of payment verification and project management, in which the auditor noted issues on payments verification in construction works, consultant services and monitoring. The follow up on these findings was relatively slow, and some of the issues were unresolved at closing. Some ineligible expenses were identified, and these were refunded to the Designated Account. c. Unintended Impacts (positive or negative): None d. Other: None 12. Ratings: Outcome: Moderately Satisfactory Risk to Development Significant Outcome: ICR IEG Review Moderately Unsatisfactory Significant Reason for Disagreement /Comments Relevance of objectives was high, and that of design substantial, but efficacy of all three objectives was modest, as was efficiency. The project had significant shortcomings in the achievement of its goals. Bank Performance : Moderately Satisfactory Moderately Unsatisfactory The team did not anticipate the lack of commitment in the implementation environment that diminished the achievement of the second PDO related to decentralization support, particularly the failure the carry out the crucial planned TA activities. Costs were poorly estimated. Problems arising from the financing agreement between central and local authorities (especially for the poorest local

9 Borrower Performance : Moderately Satisfactory Quality of ICR : Moderately Unsatisfactory Satisfactory entities) were not foreseen. The task team could have been more proactive in anticipating the possible failure of the decentralization support TAs, and in discussing and suggesting alternative ways to support decentralization capacity building at an earlier stage of the project. The failure of both the Planning Programming and Budgeting Procedures and the management support for decentralized regions was largely caused by a lack of ownership; the national government did not exercise appropriate responsibility to take the lead in the decentralization support aspects of the project. NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Adapted from the ICR: The best fitting approach for supporting sub -national public service provision during the decentralization transition depends on the stage that transition has reached, and may vary over time. The experience of this project shows that, given the point reached by Indonesia, a focus on the country's own inter -governmental transfer system would be more effective than trying to improve the benefits achieved from relatively modest amounts of Bank lending. It is important to take proactive action to enable the early engagement of the required implementation support services prior to the physical works implementation to prepare for the design and bidding processes, and to properly supervise the implementation progress and quality. The late engagement of implementation support consulting services has occurred repea tedly in Bank projects in Indonesia, which hampers the smooth start of implementation. Financing agreements between central and local authorities should be thoroughly discussed with the latter at the design stage. In this case, a grant agreement, which necessitates that sub -national governments pre-finance 100 percent of the works and then claim back the amount against work done from the national government was unpopular and in some cases unaffordable. Components outside of the scope or responsibility of the implementing agency have very little chance of successful implementation unless agreed up front with the Ministry concerned. The Ministry of Home Affairs was responsible for management support for decentralized regions, but the Ministry was outside of the accountability for the whole project which led to delays and finally cancellation. Effective empowerment of civil society in an Anti Corruption Action Plan requires sufficient budgeting to cover reasonable costs. Community Representative Observers attended all the bid openings in the project, but many did not submit the required reports because they were not provided sufficient allowances and expense payments. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: Overall the descriptions of the issues that emerged during implementation is insightful and useful. Moreover, the

10 quality of evidence provided is good, but sometimes presented in such a way as to take the focus off the main issues which were the failure of the TA components and the scaling back of the physical works due to the large increases in cost. Incorrect descriptors are given to the achievement of the individual objectives rated under efficacy. There is no discussion of the implications for the project's decentralization objective of the reallocation of resources away from sub-national roads and towards national roads. Although there is an attempt to be concise by referring to ICR annexes the constant need to search for information to understand the text make it difficult to read. The over use of acronyms (some of which were not defined) also made the document difficult to follow. a.quality of ICR Rating : Satisfactory