Document of The World Bank

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Transport ector Unit East Asia and Pacific Region Document of The World Bank FOR OFFICIAL UE ONLY IMPLEMENTATION COMPLETION REPORT (CL-43070) ON A LOAN IN THE AMOUNT OF U$234 MILLION TO THE REPUBLIC OF INDONEIA FOR A UMATRA REGION ROAD PROJECT June 29, 2006 Report No: ID This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

2 CURRENCY EQUIVALENT (Exchange Rate Effective December 31, 2005) Currency Unit = Indonesia Rupiah (IDR) IDR million = U$ U$ 1.00 = IDR 9,843 FICAL YEAR January 1 to December 31 ABBREVIATION AND ACRONYM ABIU AWPs BANGDA BAPPEDA BAPPEDALDA BAPPENA BPKP CA DGH DGLT DGRI DPUP/K EA EIRR EITRP ENPV GoI GP ICR IRM Kabupaten Kota MoF MPW NBF NRRP PAD PMU PMUC QAG REPELITA K77 RRP TA ToR TPG Accident Blackspot Investigation Unit Annual Work Programs Directorate General of Regional Development (MHA) Provincial Development Planning Board Provincial Environmental Impact and Management Agency National Development Planning Agency Financial and Development upervisory Board Country Assistance trategy Directorate General of Highways (MPW) Directorate General of Land Transportation Directorate General of Regional Infrastructure Kabupaten/Provincial Public Works Office Environmental Assessment Economic Internal Rate of Return Eastern Indonesia Region Transport Project Economic Net Present Value Government of Indonesia Global Positioning ystem Implementation Completion Report Inter-urban Road Management ystem Autonomous Level II Local Government (Rural District) Autonomous Level II Local Government (City District) Ministry of Finance Ministry of Public Works Non-Bank Financed Northern umatra Region Road Project Project Appraisal Document Project Management Unit Project Management Unit Consultant Quality Assurance Group Indonesia National five Year Development Plan Technical Guidelines for Planning and Programming of Kabupaten Roads umatra Region Road Project Technical Assistance Terms of Reference Transport Planning Commission Groups Vice President: Country Director: ector Director: Task Team Leader: Mr. Jeffrey. Gutman, Acting, EAPVP Mr. Andrew D. teer, EACIF Mr. Jitendra N. Bajpai, EATR Mr. Moustafa Baher El-Hefnawy, EATR

3 INDONEIA UMATRA REGION ROAD PROJECT CONTENT Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 6 5. Major Factors Affecting Implementation and Outcome ustainability Bank and Borrower Performance Lessons Learned Partner Comments Additional Information 26 Annex 1. Key Performance Indicators/Log Frame Matrix 27 Annex 2. Project Costs and Financing 29 Annex 3. Economic Costs and Benefits 31 Annex 4. Bank Inputs 33 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 36 Annex 6. Ratings of Bank and Borrower Performance 37 Annex 7. List of upporting Documents 38 Map IBRD 29149

4 Project ID: P Team Leader: Baher El-Hifnawi Project Name: UMATRA REGION ROAD PROJECT TL Unit: EATR ICR Type: Core ICR Report Date: June 29, Project Data Name: UMATRA REGION ROAD PROJECT L/C/TF Number: CL Country/Department: INDONEIA Region: East Asia and Pacific Region ector/subsector: Roads and highways (91%); ub-national government administration (7%); Central government administration (2%) Theme: Rural services and infrastructure (P); Pollution management and environmental health (); Municipal governance and institution building () KEY DATE Original Revised/Actual PCD: 05/28/1996 Effective: 07/31/ /23/1998 Appraisal: 07/07/1997 MTR: 05/16/2002 Approval: 03/31/1998 Closing: 12/31/ /31/2005 Borrower/Implementing Agency: Other Partners: REPUBLIC OF INDONEIA/MINITRY OF PUBLIC WORK TAFF Current At Appraisal Vice President: Jeffrey. Gutman Jean-Michel everino Country Director: Andrew D. teer Dennis De Tray ector Manager: Jitendra N. Bajpai Jeffrey. Gutman Team Leader at ICR: Moustafa Baher El-Hefnawy Jacques Tollie ICR Primary Author: Moustafa Baher El-Hefnawy 2. Principal Performance Ratings (H=Highly atisfactory, =atisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, U=ubstantial, M=Modest, N=Negligible) Outcome: ustainability: Institutional Development Impact: Bank Performance: Borrower Performance: UN M QAG (if available) Quality at Entry: Project at Risk at Any Time: Yes ICR

5 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The Project s original development objective was to improve transport efficiency and accessibility within the four provinces of the Northern umatra region and adjacent regions. The more specific objectives (stated as outputs in the Project Appraisal Document -PAD) were to: (a) strengthen integrated planning and management of transport infrastructure in the region within the Government of Indonesia's (GoI) decentralized framework; (b) develop and preserve the road networks in the Project region to environmentally sustainable and economically efficient levels; and (c) improve resource use and service delivery through gains in policy implementation and technical efficiency. Assessment of Objectives The Project's development objective and its location reflected GoI s goal of achieving more equitable and balanced regional development. The specific goals were in line with Repelita VI (1994/5 1998/9) which identified programs for network preservation, capacity improvements and road construction to support regional development. In addition, the Project s objectives addressed the need to strengthen managerial capacity and improve the efficiency of resource use at the local level following the decentralization process embarked on in the mid-1990s. The Project's objectives were clear and consistent with the 1997 Country Assistance trategy (CA) which called for: (a) increasing efficiency and productivity in the public and private sectors; and (b) enhancing equitable and sustainable regional development. The Project reflected the change in focus from sectoral-level investments in national or kabupaten roads, for example, to a more balanced regional perspective. The CA envisioned lending for infrastructure to remain an important part of Indonesia s program particularly for strengthening capacities at all government levels and supporting the decentralization of responsibilities. The Project's specific objectives responded directly to the findings of the Bank's 1997 Road ector tudy that identified (i) meeting the rapid growth in demand for road transport; and (ii) supporting sustainable development in less developed regions as the sector s two main challenges. This was the first project in Indonesia to pilot a regional approach to the road sector by aiming to implement planning and management systems at a regional level. It was innovative seeking to integrate road network development and preservation needs at the different government levels in each province and for the region as a whole. Clearly, the Project s objectives were ambitious but, nevertheless, realistic in terms of GoI s strong commitment and its implementation capacity (see Assessment of Design below). The objectives were also attainable within the Project duration. 3.2 Revised Objective: Due to the Asian financial crisis, U$50 million from the loan were cancelled in January Although the overall Project objective was not changed, in May 1999, and as a result of the significant depreciation of the Indonesian Rupiah against the U dollar, the network development sub-objective was dropped, the Project's geographical coverage was expanded to include all of umatra and the Project s name was changed from the Northern umatra Region Roads Project (NRRP) to the umatra Region Roads Project (RRP). 3.3 Original Components: In line with its three specific objectives, the Project had three main components, each consisting of several subcomponents. (Refer to the PAD for specifics.) - 2 -

6 A Al) trengthening Planning and Management of Transport Infrastructure within GoI's Decentralized Framework upport BAPPENA transport planning and management guidance; Cost: U$0.6 million. This sub-component consisted of technical assistance (TA) to support the development of decision criteria and guidelines from the National Development Planning Agency (BAPPENA) for provincial input. The proposed TA was to enable BAPPENA to create a positive working relationship with the Provincial Transport Planning Commission Groups (TPG). A2) Develop provincial strategic transport planning capacity; Cost: U$9.5 million. This sub-component consisted of technical assistance for support to (i) the Project Management Unit (PMU) (U$6.0 million); and (ii) the Provincial Development Planning Board (BAPPEDA) in each province (U$2.3 million) and for studies to assess transport priorities in other regions and prepare future regional transport projects (U$1.2 million). pecific activities included conducting surveys, definition of a regional transport strategy, the preparation of long term plans and feasibility studies and the improvement of administrative audit and data processing procedures in the provinces. A3) Improve annual planning/programming; Cost: U$0.5 million. This sub-component consisted of technical assistance to the Directorate General of Regional Development (BANGDA) and the PMU (the latter being funded under the general PMU Consultancy--component A2) to improve procedures in the following six areas: (i) planning surveys and analysis; (ii) environmental screening and studies; (iii) audit, monitoring and eligibility review; (iv) programming and budgeting; (v) financial and physical monitoring; and (vi) database establishment and maintenance. These activities were to be accompanied by training in each province. B) Improvement of the National, Provincial, and Kabupaten (district) Road Networks B1) Regional road network preservation; Cost: U$202.8 million. This sub-component comprised road maintenance, minor widening and betterment. Maintenance was for kabupaten roads and entailed a (i) Core maintenance program U$ 53.2 million, comprising routine maintenance and overlays, wholly GoI-funded; and (ii) a periodic maintenance program financed under the loan U$ 78.8 million. Minor widening (up to two meters and within the existing ROW) was limited to national/provincial roads only U$14.6 million. Betterment was for both national/provincial roads (U$12.0 million) and kabupaten roads (U$44.3 million). For kabupaten roads, the selection was to be based on the five-year plans and defined strategic road networks. B2) Regional road network development; Cost: U$143.9 million. This sub-component comprised capacity expansion and network extension. Capacity expansion 130 Kms of national and provincial roads U$55.7 million. Network extension 130 Kms for the national and the provincial road networks U$27.8 million; and 825 Kms for the kabupaten road networks (25 Km per kabupaten) U$49.8 million. Land acquisition for both subcomponents constituted another U$10.5 million

7 C) Improvement of Resource Use and ervice Delivery Cl) Enhance quality and technical efficiency of design and construction processes; Cost: U$2.4 million. This sub-component comprised the establishment of an integrated implementation and quality assurance (QA) management team established under the PMU (financed under component A2) to design sustainable solutions to improve the standards of design and construction, focusing on: (i) construction contract monitoring and control processes, (ii) the design process, (iii) the construction audit process, (iv) quality of construction, and (v) advice on technical and contractual matters. In addition, the component included services for improved strategic mapping (U$1.4 million), and specialized survey equipment to allow a complete survey of the Project area and a global positioning system (GP) to improve the mapping in two provinces (U$1.0 million). C2) trengthen road traffic and transport policies; Cost: U$9.6 million. This sub-component consisted of: (i) road traffic safety--a three year program comprising institutional strengthening of provincial road traffic and transport agencies and selective investments in civil works and equipment to address critical road safety issues (U$6.4 million); and (ii) vehicle overloading--a three-year program comprising institutional strengthening, awareness campaigns, enforcement actions and provision of equipment to reduce vehicle overloading (U$3.2 million). Assessment of Design There was a strong, clear link between the Project s components and its specific objectives. The first component correctly acknowledged that strengthening integrated management of the road sector in a decentralized framework required institutional support at the national, provincial and kabupaten levels. The second component excluded the financing of periodic maintenance of national and provincial roads due to a relatively strong maintenance culture for higher level roads and focused instead on strengthening maintenance capacity for kabupaten roads. This built on the Bank s experience and the success of the Kabupaten Roads Projects (3490-IND, 3579-IND and 3732-IND) that laid a foundation for systematic maintenance procedures and funding. The eligibility of kabupatens to participate in the Project was conditional on the definition of kabupaten strategic networks and the preparation of five-year work plans that included annual routine maintenance of 150 Kms and overlay of 20 Kms for each kabupaten. This was to help strengthen a planning culture at the kabupaten level as well as to demonstrate government commitment to the Project. The design utilized several of the lessons learned from earlier projects, particularly with respect to having realistic and specific goals achievable within the Project s time frame. Another lesson was the need for the central government to leverage its budgetary resources to ensure more efficient resource use by local government. This was reflected in the eligibility criteria for kabupaten participation mentioned above. It was not clear, though, how lessons from the Bank s experience in regional projects in the health and education sectors in Indonesia were considered in this Project s design. The implementation of such a Project with a large number of subcomponents involving different levels of governments and different agencies within each level was demanding. The lead implementing agency, the Directorate General of Highways (DGH), had executed several projects involving the Bank and other donors. It had competent staff and had played a lead role within the government team during preparation. However, as there were five more implementing agencies involved (BANGDA, BAPPEDA, Provincial and Kabupaten Directorates of Public Works (DPUP, DPUK) and the Directorate General of Land Transportation (DGLT)), a teering Committee and PMU were established - 4 -

8 to facilitate coordination and to coordinate activities respectively. Heeding one of the lessons learned from an earlier project, the Bank provided a large TA program to support the PMU both at the central and provincial levels. A Project Implementation Memorandum documenting the respective roles, responsibilities and procedures for project implementation for each agency was also prepared. Despite the overall sound design of the Project, the TA programs to support planning (Component A) were not clearly defined and resulted in confusion requiring changes later on although some of the changes were required to adapt to the evolving decentralization process. In addition, the need to establish TPGs in every province (under the BAPPEDA TA) might not have been totally appropriate as some provinces such as Western umatra had a strong planning tradition and did not find much use for this group. There did not seem to be sufficient analysis of the institutional capacity of participating provinces. 3.4 Revised Components: Due to the impact of the Asian financial crisis, the road network development was abandoned and the focus shifted to network preservation as explained below. A1-A3) Transport planning and management subcomponents: No significant change B) Improvement of the National, Provincial, and Kabupaten Road Networks. Improvement of the road networks was to be accomplished through both network preservation and development. As a result of the Asian financial crisis and the fiscal pressures on GoI, the focus of this component became almost entirely network preservation as explained below. B1) Regional road network preservation. The Kabupaten periodic overlay sub-component was cancelled as a direct result of the Asian financial crisis. Moreover, to capitalize on the large depreciation of the Rupiah, the geographical scope of the Project was expanded to include the four provinces of southern umatra; therefore the beneficiaries of this sub-component changed from 4 to 8 provinces and from 33 to 55 kabupatens. Due to the breaking-up of provinces and kabupatens, those numbers changed to 9 and 73 respectively. Output targets in Km were revised as follows: PAD targets Revised targets Kabupaten maintenance (based on 8 provinces and 55 kabupatens) Routine (GoI funded) 19,800 24,750 Overlays (GoI funded) 2640 cancelled Periodic 6,600 8,250 Betterment and minor widening National/Provincial Kabupatens B2) Regional road network development. As a result of the 1997 Asian financial crisis, the sub-component on network extension (U$77.6 million) was dropped and the sub-component on capacity expansion was reduced from 130 Km to 80 Km. Cl) Enhance quality and technical efficiency of design and construction processes. The only revision in this sub-component was to the Mapping TA. This package was replaced after careful consideration of many factors including the estimated final cost of the TA, the limited coverage of the mapping (two provinces), project complexity, delays in tendering which would have resulted in a - 5 -

9 completion date near to or outside the loan completion date and the availability of alternative sources for project mapping. C2) trengthen road traffic and transport policies. The scope of the traffic safety subomcponent was reduced due to delays and budgetary limitations. Initially, blackspots were to be identified in all four northern provinces, remedial measures would be determined and designs completed. However, only two accident blackspots were included in the program. There was a change in the conceptual approach to controlling vehicle overloading. Realizing the limitations of existing practices to control overloading due to potential corruption at the weigh station, emphasis was changed from physical improvements and strengthening enforcement to involving the private sector in the operation of weigh stations and to the use of performance incentives. The new approach was piloted in four locations (three in West umatra and one in Aceh). 3.5 Quality at Entry: Quality at entry is rated satisfactory. The Project was consistent with both GoI priorities and CA objectives (see ection 3.1), and the implementation arrangements were well conceived as note above. The Bank s safeguard polices were followed. GoI conducted a social impact assessment for the Project and prepared a resettlement policy framework consistent with OD4.30 and a strategy to mitigate any negative impacts on isolated vulnerable people consistent with the GoI s requirements and the Bank s OD4.20. GoI also prepared an environmental impact assessment and an environmental management and monitoring plan based on the Environmental Assessment's (EA) recommendations. A project review by the Quality Assurance Group (QAG) in 1998 found quality at entry satisfactory. Among the more important risks identified by the team were: government commitment, counterpart funds and managerial capacity at the different Project levels. The general government thrust towards decentralization and the participation of BAPPENA in the Project served to mitigate both commitment and counterpart funding risks. The sound design of the overall implementation arrangements addressed the managerial capacity question well. However, government commitment did not appear very strong with respect to the vehicle weight and dimension sub-component. Polices to control overloading had not been successful in the past, and a 1989 GoI Action Plan addressing this issue was facing delays during preparation. While the quality at entry was satisfactory, there were several issues that could have been addressed better. Performance indicators were not well defined (were later changed) and no baseline was established. With "increasing efficiency and accessibility" being the broad development objective, there could have been indicators to measure these objectives more directly such as savings in travel time and cost for example. Among the other shortcomings, there were detailed implementation plans for two subcomponents only. In addition, the section in the PAD addressing the three TA programs of Component A on strengthening transport planning and the terms of reference for these assignments could have benefited from additional clarity in defining responsibilities and coordinating mechanisms, and in avoiding overlaps. Also, consistency in defining minor widening in the PAD was lacking. In certain cases, it was presented under betterment and in others it was presented as an independent category, with betterment referring to other activities. Finally, contractual arrangements made it difficult to figure out the cost of sub-component C1. The PMU was to be paid under sub-component A2 for all its tasks including those carried out under sub-component C1. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: - 6 -

10 The achievement of the Project s main objective of improving transport efficiency and accessibility within umatra was moderately satisfactory. This reflects the relative success in accomplishing each of the objectives as explained below. (a) trengthen and integrate planning and management of transport infrastructure: Unsatisfactory. Delays in implementing the TA packages for Component A, particularly the BAPPEDA TA, coupled with poor implementation coordination among these TAs at the different government levels precluded the development of the desired outcomes in a participatory manner that would ensure ownership and sustainability. Instead the TAs resulted in several high quality outputs that may or may not be utilized by the different levels and agencies of GoI. The performance indicators in Annex 1 show that while the availability of medium term plans for kabupatens increased from 39% in 2000 to 79% in 2003, it dropped to 39% in 2004 following the completion of the civil work component that required the preparation of these plans. If one were assess this objective excluding the 2004 data, the rating would have been moderately satisfactory, and outcome sustainability in question. (b) Develop and preserve road networks to environmentally sustainable and economically efficient levels: Moderately atisfactory The Project succeeded in meeting the network preservation objective. However, the network development objective was not met as the sub-component contributing to network development was cancelled (see B2 in ection 3.4). In addition, while the Project succeeded in imparting the skills necessary to apply economic efficiency criteria for project selection at the kabupaten level, these achievements were not sustained, as the performance indicators in Annex 1 show a decline by December The Project was not very successful in supporting kabupatens in developing and strengthening a road preservation culture. Eligibility of kabupatens to participate in the Project required commitments to certain targets for routine maintenance and overlays. The overlay condition was dropped as a result of the fiscal pressures facing the government following the Asian financial crisis. With the benefit of hindsight, it became clear that routine maintenance requirements were observed mainly to secure participation in the Project, and generally do not appear to be consistently followed by kabupatens any longer. While expanding the geographical region of the Project to the entire island of umatra reduced the intended impact of the physical improvements in promoting integrated strategic corridors in the provinces and kabupatens that were originally to be covered by the Project (northern umatra), it provided a larger set of roads for screening, which most likely resulted in overall higher economic rates of return to the physical improvements. This is however difficult to ascertain given that the screening and selection processes took place during implementation. The Project succeeded in strengthening the role of environmental screening and management in transport projects as evident by the more effective role that Bappedalda (Environmental Management Impact Agency) continues to play in some provinces. The performance indicators in Annex 1 also indicate that efficient planning is becoming less of a basis for network development and preservation. More importantly, the misprocurement of 32 contracts (see ections 5.2 and 5.3 below) whose prices had been prearranged among bidders, has clearly hindered efficiency in developing and preserving the road network. Even though the value of the 32 contracts accounted for a small share of the loan (2.5%) and might not have had a major impact on the achievement of the Project s objectives, the incident has raised concerns as to the possible prevalence of such practices in all procurement activities, which in turn would undermine the efficient development of the road network

11 (c) Improve resource use and service delivery: Moderately atisfactory This Project contributed to capacity development in project selection, design and supervision at the various government levels particularly for kabupatens where capacity was the most lacking. While the objective of achieving sustainable road safety programs was not fully realized, many positive steps were taken in that direction providing a sound base to build on. Notably, the reduction in fatalities per 10,000 vehicles exceeded planned targets. Vehicle overloading has been reduced on certain corridors on the strength of Project financing with some promising indications that current efforts will be sustained through government financing and the piloted model carried out nationally. 4.2 Outputs by components: Al) upport BAPPENA transport planning Rating: atisfactory After long start-up delays, a comprehensive set of decision criteria and guidelines for regional transport planning were developed. These covered 17 areas including: (i) development planning decision criteria; (ii) guidelines on environmental and social planning; (iii) guidelines on regional development planning; (iv) guidelines on evaluation and prioritization; (v) simulation of the regional transport planning process; and (vi) social environmental guidelines. A2) Develop provincial strategic transport planning capacity Rating: atisfactory At the kabupaten level, five and ten-year road network development plans were prepared for all kabupatens. At the kota (city district) level, guidelines for urban transport studies were developed. These included guidelines for urban transport development planning and for integrating urban transport plans into provincial plans. At the provincial level, key outputs included 20-year plans for roads and bridges, establishing transport planning groups, and developing simple methodologies for medium and long term planning. A3) Improve annual planning/programming Rating: atisfactory Although a large number of guidelines and manuals were developed under this sub-component, many were produced late in the Project and agencies were not fully exposed to their content. Key outputs included defining BANGDA s transport management roles and activities in provinces and kabupatens, preparing operational guidelines for BANGDA, developing guidelines and procedures for coordination among agencies, and institutionalizing the BAPPENA transport planning guidelines. B1) Regional road network preservation Rating: atisfactory Table 1 shows achievements against the revised targets. It was believed that the higher revised targets could be achieved based on the sharp depreciation of the Rupiah but this was not enough to offset the increased construction cost that turned out to be significantly higher than planned. Nevertheless, the table shows a relatively high rate of achievement varying from 85% to 235% of the original targets; and 69% to 176% of the revised estimates. There has been some variation in the overall quality of the implemented works, but problems have been rectified and with some exceptions particularly in kabupaten roads, quality is acceptable. B2) Regional road network development Rating: Moderately atisfactory Measuring against the revised target of 80 Kms of new capacity (capacity extension cancelled--see ection 3.4), the output for this subcomponent was substantially less than planned at only 42% of the revised target (see Table 1 below)

12 Table 1: Road Network Preservation and Development: Planned and Actual (Kms) COMPONENT B1: NETWORK PREERVATION Kabupaten Maintenance PAD/Revised* Actual Actual/Revised Core Program (GoI funded) Routine Maintenance 19,800/23,700 23, % Core Program (GoI funded) Overlays 2,640/cancelled cancelled Periodic Maintenance 6,600/8,250 5, % Betterment and Minor Widening National and Provincial 240/ % Kabupatens 990/1,650 1, % COMPONENT B2: NETWORK DEVELOPMENT Capacity Expansion National and Provincial 130/ % Network Extension National and Provincial 130/cancelled cancelled Kabupatens 825/cancelled cancelled * Figures per PAD/Revised figures after expansion of geographical scope. Land acquisition, resettlement and social impacts. Three subprojects required land acquisition and resettlement of about 560 households: Tabing-Duku in Padang, West umatra; Aek Nabara in Medan, North umatra; and Pekanbaru-Batas Kampar in Riau Province. Land acquisition was delayed in a few instances but was generally satisfactory. An October 2004 mission visited the capacity expansion project in West umatra and interviewed a number of project affected residents and businesses. They expressed their satisfaction with the compensation and with the land acquisition process and indicated that the project had improved the locale. everal mentioned that they are experiencing an increase in rents associated with their properties. Visits to Project sites in North umatra and Riau by GoI s Implementation Completion Report (ICR) consultant produced similar results. Therefore, implementation of social safeguards is considered satisfactory. Cl) Enhance quality and technical efficiency Rating: atisfactory An integrated implementation and quality assurance management team was established under the PMUC as envisioned and carried out the required tasks satisfactorily using both on-the-job training as well as formal training programs. This subcomponent was particularly beneficial in uncovering shortcomings in design and quantities used for tendering during the first phase of the Project. These problems were then addressed in ensuing phases. imilarly, poor construction quality was identified as a concern during site audits and was reasonably addressed resulting in significant improvements as the Project progressed through its three annual work programs. Multicolored road network base maps were produced for the region, its nine provinces and 73 kabupatens using digitized data. These were utilized in the preparation of 5 and 10-year plans. C2) trengthen road traffic and transport policies Rating: Moderately atisfactory Many of the outputs under this subcomponent were successfully produced. However implementation varied across provinces, and delays in procurement and implementation resulted in a reduction in the number of programs undertaken. Accident Blackspot Investigation Units (ABIUs) were established and - 9 -

13 equipped, and training and manuals were provided in the following areas: (i) accident reporting, (ii) accident investigation and development of mitigation measures, (iii) road safety auditing, and (iv) traffic signal design. However there was no effective ABIU work in North umatra. Blackspot programs were designed and improvement links were constructed in Riau only in 2004, with work programs in West and North umatra delayed till late Among other accomplishments, publicity campaigns were undertaken, legislation on traffic safety issues was drafted, and pilot road safety projects were developed but not undertaken due to lack of funds. Although delayed significantly and resulting in a one-year loan closing date extension, four pilot programs for managing overloading were established in West umatra and Aceh. The programs were built on performance-based incentives, management structures involving the private sector and community pressure through public awareness campaigns. 4.3 Net Present Value/Economic rate of return: As explained in the PAD the benefits of improved efficiency were difficult to quantify. Consequently the focus of the estimation was Bank-financed road works. This included both national/provincial roads and kabupaten roads. There should have been an estimate, even if just indicative, of the benefits associated with the GoI-financed core maintenance program in the kabupatens, as well as of the reduction in overloading. Moreover, to help increase the appreciation of planning and programming, it would have been useful to compare returns to annual work programs (AWPs) developed on the basis of the efficiency criteria promoted by the Project to the returns of AWPs as identified by kabupatens prior to the Project. National/Provincial Roads Inter-urban Road Management ystem (IRM) was used to identify the economic viability of the preservation and capacity expansion projects on national/provincial roads based on the Economic Internal Rate of Return (EIRR). This formed part of the screening process to define the specific investments in each AWP. When IRM data on road condition and traffic levels was suspect and where the proposed type of works differed significantly from the IRM output, better data on road condition and traffic level was sought during the screening phase of the Project. Due to lack of resources, traffic was not measured after implementation completion. The EIRR calculations in Table 2 below are based on the same traffic data and projections used at the time of screening but on actual construction costs. Consequently, and given that traffic projections have not been updated, there is a larger degree of uncertainty surrounding the projected returns. Annex 3 presents some details on the estimation and methodology used. Nevertheless, the returns are high enough to withstand significant drops in traffic volumes from what was projected. Table 2: Economic Internal Rates of Return (EIRRs) Pre-implementation and Post Completion* Phase 1** Phase 2** Phases 1 & 2** EIRR (Actual) EIRR (Actual) EIRR (Actual) Pre-implementation Post-completion Pre-implementation Post-completion Pre-implementation Post-completion Betterment 82% 81% 77% 88% 79% 85% Minor Widening 55% 50% 71% 86% 59% 58% Capacity Expansion 21% 23% 21% 23% All 61% 64% * Post completion EIRRs are based on pre-implementation traffic values and actual costs. ** The fluctuation in exchange rates over the life of the Project resulted in two phases for sub-project screening and identification. Initially, it was believed that only kms would be financed under the Project. When it was later realized that there would be sufficient funds to finance more works, an

14 additional Kms of preservation works and 33.8 Kms of capacity expansion works were implemented as phase 2. The initial Kms constituted phase 1. Kabupaten Roads: The Technical Guidelines for Planning and Programming of Kabupaten Roads (K77) were used to determine the economic viability of betterment projects based on a positive Economic Net Present Value (ENPV) for each project at a 15% discount rate. imilar to the case of national/provincial roads, the ENPV calculations provided in Table 3 use the same traffic data and projections utilized at the time of initial screening and the actual construction costs. It was not possible to estimate pre-implementation ENPVs because the pre-construction planned costs are no longer available. Table 3: Economic Net Present Value per Km for Kabupaten Betterment Projects ENPV (Rp millions/km) AWP 1999/ AWP AWP All AWPs Financial rate of return: N/A 4.5 Institutional development impact: Despite substantial contributions by the Project to institutional strengthening, its overall institutional development impact is rated modest. This is due to the unlikely sustainability of these impacts (see ection 6.1), which vary significantly from one area to another. The institutional impact of the Project in enhancing the quality of design and construction was high, while its impact on strengthening strategic planning, road safety and reducing vehicle overloading was modest. Extensive training was provided to help institutionalize the positive impacts of the Project. Almost 2,000 people received training for a total of about 12,000 person days. The design of the training programs was based on needs assessment and covered areas of planning, project design and implementation, quality control and contract management among others. Questionnaires were sent out to trainees and their employers to assess training effectiveness. About 92% of the respondents rated the programs between fair and very good. DGH played a guiding and advisory role to kabupatens (compared to the more hands-on executing roles on previous projects) to strengthen the capacity at the local level in technical planning, programming, implementation and monitoring of work packages. Through monitored supervision of civil works by field consultants, reinforced by formal training, project related staff gained valuable experience in implementing high quality works. This was reflected in an improved quality of design and construction progressively throughout the Project. The Project had a substantial impact on environmental management at the local level. The Project created greater awareness within BAPPEDA and MPW in the provinces and kabupatens for the need for consultation with Bapedalda during road works planning, programming, design, construction and maintenance. In addition, the guidelines that were prepared and the training on safeguard policies in

15 environmental aspects of projects that was offered helped strengthen the capacity of Bapedalda. There are positive institutional developments with respect to road safety but in very few locations. Members of the ABIUs have carried out presentations at four school locations identified as having high student accident casualties, ABIU staff has participated in radio talk shows to raise public awareness and the police participated in public broadcasts reminding listeners about the use of helmets and seat belts. The modest impact of the Project in developing strategic planning capacity at the local level is due to the fact that there was little ownership of the guidelines and manuals that were prepared. Local governments believe that they were not sufficiently consulted in the process. (see ection 6.1). 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: 1. East Asian financial crisis. The crisis affected the Project in three interrelated ways: First, the crisis shifted the attention of senior and key government officials to social safety nets and poverty alleviation policies. Moreover, the fiscal impacts of the crisis faced by GoI were massive. Debt service, fuel subsidies and restructuring of Indonesia s tate-run Electricity Company placed huge burdens on the budget. This limited any benefits to the network development sub-component as the network extension sub-component was cancelled and the network expansion sub-component was reduced substantially. In addition, the GoI-financed overlay sub-component for the kabupaten roads was also dropped. econd, the sharp depreciation of the Rupiah, coupled with the uncertainty regarding the magnitude of the fiscal impacts of the crisis, led to the expansion of the Project's geographical scope to cover all of umatra. The initial plan was for this Project to cover northern umatra and be later followed by a Project for southern umatra. This reduced the intended impact of the physical improvements in promoting integrated strategic corridors in provinces and kabupatens. Third, the uncertainty created by the crisis also resulted in effectively no disbursements for the first 12 months of the Project. The U$/IDR exchange rate was 3,500 IDR/U$ at appraisal, 14,200 IDR/U$ at effectiveness and 9,843 IDR/U$ at completion. The large fluctuations in the exchange rate throughout the life of the loan had implications for the achievable scope of work. The situation led the PMU to implement a conservative approach to the size of the AWPs for the kabupaten, provincial and national road works which in turn resulted in significant funds being available during the later stages of the Project. 2. Adverse security situation in Aceh. The security situation in Aceh worsened throughout the Project and delayed the implementation of several packages. This also resulted in the level of supervision and capacity development being less in Aceh than that in other provinces. 3. Vagueness in the Terms of Reference (ToR) for the TA packages addressing the strategic planning component To strengthen strategic planning at the local level, TAs were intended for four agencies: BAPPENA, BAPPEDA, BANGDA and DGH. The ToR were insufficiently specific in this regard leading to misunderstandings by both clients and consultants and requiring time and effort to clarify roles and responsibilities of each consultant in the overall structure of the Project. This vagueness contributed to delays which reduced the overall impact of this component. The situation was exacerbated by the

16 evolving roles following the decentralization law. While project preparation is the responsibility of the government, its implementing agencies and consultants, the Bank could have provided more guidance in particular areas where GoI may have little experience. 5.2 Factors generally subject to government control: 1. Lack of clarity in defining planning roles following decentralization. Decentralization created a plethora of agencies with transport planning and environmental management responsibilities. These agencies are widely dispersed in terms of geography, sub-sectoral scope and locus within the overall government structure. There was a lack of clarity within each agency regarding roles and responsibilities with respect to transport planning particularly under deconcentration, which left central government responsible for certain functions through their regional offices. Law 22 on regional autonomy did not provide for appropriate mechanisms to ensure that these agencies liaise closely so that transport planning, programming and budgeting are carried out in a consistent manner compatible with the development of an integrated transport system. 2. Availability and timing of release of counterpart funds. The annual programs and budgets were not always finalized by the start of the fiscal year (initially July and then January). Delays occurred while budgets were approved by government authorities. In addition, the available budgets for land acquisition and resettlement were limited, further contributing to implementation delays. 3. taff changes due to splitting of kabupatens. Transfer of key staff within the government deprived some kabupatens and provinces of critical personnel. During the life of the Project, 18 kabupatens were split up. This left some of the newly formed kabupatens with insufficient skills and capacity to manage contracts. 4. Weak commitment to managing the vehicle overloading and road safety component. DGLT s commitment to this component was not strong. There were continuous implementation delays (Project was extended one year just for this component), poor implementation performance, and no strong interest in exploring different institutional set-ups for sustaining the achievements in limiting vehicle overloading after Loan closing. imilarly, the ABIUs were not provided with budgets and qualified staffing to continue successful operation after Loan closing. ince closing, there have been some promising indications that local governments are contributing some funding to continue the programs for managing overloading that were piloted under the Project. 5. Misprocurement of 32 contracts. After undertaking a detailed review of 42 contracts for civil works for kabupaten roads in North umatra, West umatra and Jambi, the Bank found that the award of 32 of those contracts had been prearranged among bidders either with the active involvement of the tender committees of the kabupaten governments in the three provinces or in a manner that was easily detectable to the tender committees. These actions undermined the objectives of economy, efficiency and transparency intended in National Competitive Bidding processes required by the Bank's Procurement Guidelines and resulted in the award of higher, non competitively-priced contracts (also see para 2 in ection 5.3 below). 6. Lack of authority and independence of supervision engineers. While government-appointed project managers (PIMPROs) had the authority to reject works and withhold payments, supervision engineers did not have that authority. This resulted in low quality construction for some kabupaten roads and required remedial works. 7. Inadequacy of the implified Design approach. Proposals submitted for undertaking design

17 and construction works were based on implified Design standards per GoI s guidelines. These are indeed simple standards that do not take site-specific conditions into account. As a result, there would be design reviews later which would ultimately lead to increased costs and opportunities for corruption. The increase in cost made it difficult to develop a realistic sense of the size of the work program that would ultimately be implemented. Considerable efforts were directed at improving these standards and implified Design was replaced by Final Design in Phase 2 of the national and provincial works program. Design review was eliminated and the number and size of contract variations was reduced. 5.3 Factors generally subject to implementing agency control: 1. Land Acquisition/Resettlement Delays. Only four capacity expansion packages of work involved considerable land acquisition. The acquisition process caused delays with one package being cancelled due to insufficient time to complete the acquisition and resettlement process. It should be noted however, that limited budgets for compensation prolonged the implementation process as well (see para 2 in ection 5.2 above). 2. Compliance with Bank requirements. In some cases, provinces and kabupatens failed to comply with Bank requirements especially those relating to notification and advertising of prequalification/bids, bidding period and bid opening procedures. The PMU, with support from its consultant (PMUC), however, is responsible for day to day management of the Project. As such, even though the main responsibility for procuring goods and services lays with the local governments as indicated in the loan agreement, DGH, the lead implementing agency, also shares responsibility for the 32 misprocured contracts (see para 5 in ection 5.2 above and see GoI's comments on the Bank's ICR in ection 9). 3. Initial management of the trategic Transport Planning TA packages. While the PMU did a good job in managing the strategic transport planning TAs after the refocusing took place in mid 2002, the PMU does share responsibility with the Bank (see para 3, ection 5.1) for the vagueness of the initial TAs and more so for the resulting delays in implementation. In particular, the BAPPEDA TA had not been mobilized on time and the assurance of immediate continuity of assistance at the provincial level was lost, although BAPPEDA itself shares some of the responsibility for the delays. 5.4 Costs and financing: The cost of the Project at appraisal was estimated at U$369.3 million of which the Bank was to finance U$234 million and GoI to finance the balance of U$135.3 million. The total actual Project cost is estimated at U$219.3 million (59% of the appraisal estimate) of which the Bank financed U$178.9 million and GoI U$40.4 million respectively equivalent to 82% and 18% of the total. The Loan Agreement was amended three times and the closing date was extended two years to December 31, This first amendment in January 1999, cancelled U$50 million from the Loan. The second amendment in May 1999 changed the name of the Project and increased the Bank s share of financing of civil works and materials from 75% to 80%. The third amendment in July 2003 allocated the unallocated funds and made some changes in the existing allocations. Loan savings of U$5.1 million were cancelled at the end of the Project. The first extension in the loan closing date (to December 31, 2004) was to allow completion of the civil works and TAs. The second extension (to December 31, 2005) was to allow completion of the pilot programs to manage vehicle overloading and to afford DGLT sufficient time to effectively plan continuation of the program after the Bank loan closes. Expenditures on the regional road network development as a percentage of the estimate at appraisal was lowest at 12% due to cancellation and large reduction of the network extension and expansion sub-components respectively. Expenditures on the technical efficiency sub-component were also low at