ASIAN DEVELOPMENT BANK PCR: INO 28063

Transcription

1 ASIAN DEVELOPMENT BANK PCR: INO PROJECT COMPLETION REPORT ON THE EASTERN ISLANDS ROADS (SECTOR) PROJECT (Loan 1335-INO) IN INDONESIA April 2002

2 CURRENCY EQUIVALENTS Currency Unit Rupiah (Rp) At Appraisal At Project Completion (October 1994) (March 2001) Rp1.00 = $ $ $1.00 = Rp2,173 Rp9,800 ABBREVIATIONS BAPPENAS Badan Perencanaan Pembangunan Nasional (National Development Planning Agency) BINTEK Bina Teknik (Engineering Directorate of DGH) BME benefit monitoring and evaluation BMS bridge management system DBM Dinas Bina Marga (highways directorate of provincial public works department) DGH Directorate General of Highways DGLC Directorate General of Land Communications DGRI Directorate General of Regional Infrastructure (formerly DGH) DPUP Dinas Pekerjaan Umum Propinsi (Provincial Public Works Service) DTUA Direktorat Tata Usaha Anggaran (Directorate of Budget Administration) EIRR economic internal rate of return FIRR financial internal rate of return IRMS integrated road management system MOC Ministry of Communications MOF Ministry of Finance MOHA Ministry of Home Affairs MPW Ministry of Public Works MSRI Ministry of Settlements and Regional Infrastructure NPRMTP National/Provincial Roads Master Training Plan O&M operation and maintenance RBO regional betterment office Repelita Rencana Pembangunan Lima Tahun (Five-Year Development Plan) TA technical assistance NOTES (i) Before 2000, the fiscal year of the Government ended on 31 March. Since 2000, the fiscal year has ended on 31 December. FY before a calendar year denotes the year in which the fiscal year ends. For example, FY2000 begins on 1 January 2000 and ends on 31 December (ii) In this report, $ refers to US dollars.

3 CONTENTS Page BASIC DATA MAP ii vii I. PROJECT DESCRIPTION 1 II. EVALUATION OF DESIGN AND IMPLEMENTATION 2 A. Relevance of Design and Formulation 2 B. Project Outputs 2 C. Project Costs and Financing 5 D. Disbursements 6 E. Project Schedule 7 F. Implementation Arrangements 7 G. Conditions and Covenants 7 H. Related Technical Assistance 8 I. Consultant Recruitment and Procurement 8 J. Performance of Consultants, Contractors, and Suppliers 8 K. Performance of the Borrower and the Executing Agency 10 L. Performance of the Asian Development Bank 11 III. EVALUATION OF PERFORMANCE 11 A. Relevance 11 B. Efficacy in Achievement of Purpose 11 C. Efficiency in Achievement of Outputs and Purpose 11 D. Preliminary Assessment of Sustainability 12 E. Environmental, Sociocultural, and Other Impacts 13 IV. OVERALL ASSESSMENT AND RECOMMENDATIONS 14 A. Overall Assessment 14 B. Lessons Learned 14 C. Recommendations 16 APPENDIXES 1. Chronology of Milestone Events Status of Completion of Project Components List of Major Civil Works Contracts List of Road Sections that Used Lasbutag Appraised and Actual Project Costs Contract Awards and Disbursement Status Project Implementation Schedule Project Organization Schedule Compliance with Loan Covenants Consulting Services and Costs Number of Contracts and Mode of Procurement Methodology and the Results of Economic Analysis List of Roads with Construction Quality Problems Project Performance Rating Assessment 42

4 ii BASIC DATA A. Loan Identification 1. Country Indonesia 2. Loan Number 1335-INO 3. Project Title Eastern Islands Roads (Sector) Project 4. Borrower Republic of Indonesia 5. Executing Agencies (i) Directorate General of Highways (ii) Directorate General of Land Communications 6. Amount of Loan $180 million (ordinary capital resources) 7. PCR Number PCR: INO B. Loan Data 1. Appraisal - Date Started 1 August Date Completed 18 August Loan Negotiations - Date Started 26 October Date Completed 27 October Date of Board Approval 29 November Date of Loan Agreement 10 April Date of Loan Effectiveness - In Loan Agreement 20 June Actual 20 June Closing Date - In Loan Agreement 31 December Actual 31 March Number of Extensions Two 7. Terms of Loan - Interest Rate Variable - Maturity 24 years - Grace Period 4 years

5 iii 8. Disbursements a. Dates Initial Disbursement Final Disbursement Time Interval 14 July November years Effective Date Original Closing Date Time Interval 20 June December years b. Amount ($ million) Category Original Allocation Revised Allocation Amount Disbursed Undisbursed Balance A. Road and Bridge Works 1. Civil Works Supervision of Civil Works a. Core Team b. Field Team B. Vehicle Inspection and Testing 1. Civil Works and Equipment Advisory Services and Training C. Institutional Strengthening 1. East Indonesia Road Study Training D. Interest During Construction Total Local Costs (Financed) - Amount $7.6 million - Percent of Local Costs 39.0 percent - Percent of Total Cost 9.5 percent

6 iv C. Project Data 1. Project Cost ($ million) Item Appraisal Actual Estimate Foreign Exchange Cost Local Currency Cost Total Cost Financing Plan ($ million) (i) Implementation Cost Source Appraisal Estimate Actual Foreign Local Total Foreign Local Total Borrower-Financed ADB-Financed Total Cost (ii) IDC Cost Source Appraisal Estimate Actual Foreign Local Total Foreign Local Total Borrower-Financed ADB-Financed Total Cost ADB = Asian Development Bank. IDC = Interest During Construction

7 v 3. Cost Breakdown, by Project Component ($ million) Appraisal Estimate Actual Component Foreign Local Total Foreign Local Total Part A Road and Bridge Works Civil Works Supervision Consultant Subtotal Part B Vehicle Inspection and Testing Civil Works and Equipment Advisory Services and Training Subtotal Part C Institutional Strengthening East Indonesia Road Study Training Subtotal Interest During Construction ADB Loan Domestic Loan Total ADB = Asian Development Bank.

8 vi 4. Project Schedule Item Appraisal Estimate Actual Date of Contract with Consultants 31 August June 1996 Completion of Engineering Designs 1 August August 1995 Civil Works Contract Date of Award Completion of Works Equipment and Supplies Dates First Procurement Last Procurement Completion of Equipment Installation Start of Operations Completion of Tests and Commissioning Beginning of Start-Up 30 January January January November February March May July January February January 1999 D. Data on Asian Development Bank Missions No. of No. of Specialization Name of Mission Date Persons Person-Days of Members a Appraisal 1 August a, b, f, h, e Inception 27 June c, d Review 1 25 January a, h Review 2 16 September a, h Review 3 6 October a Review 4 21 September a Review 5 18 July a, g Review 6 2 June a, g PCR b 20 March a, h Total Person-Days 232 PCR = Project Completion Review. a a - engineer/project officer, b - transport development specialist, c - transport economist, d - financial analyst, e - counsel, f - program officer, g - project implementation officer, h - staff consultant. b The Mission comprised Soewartono HS, Sr. Project Officer/Mission Leader and Daud Ahmad, Transport Specialist/Staff Consultant.

9

10 I. PROJECT DESCRIPTION 1. The main objective of the Project was to support the Government s road sector development program and policies during Repelita VI. In particular, the Project would assist in funding a 3-year time slice (1995/ /98) of the Government s civil works program for national and provincial roads in the 13 provinces of eastern Indonesia. This would assist the Government in meeting the demands of rapid growth in road traffic by improving road conditions in order to enhance economic and social linkages and facilitate equitable access to the benefits of national development. Another objective of the Project was to improve road safety by upgrading facilities and procedures for vehicle inspection and testing. The Project also aimed to strengthen programming and planning of road network development in eastern Indonesia and to upgrade the professional qualifications of staff at the Directorate General of Highways (DGH) of the Ministry of Public Works The subprojects were located in 13 provinces in the eastern islands of Indonesia: West Nusa Tenggara, East Nusa Tenggara, former East Timor, Maluku, all provinces in Sulawesi, all provinces in Kalimantan, Bali, and Irian Jaya. 3. The Project consisted of three main parts: (i) (ii) (iii) Part A Road and Bridge Works, which included (a) civil works for (i) road betterment of about 980 kilometers (km) of selected road sections in deteriorated and congested conditions, (ii) periodic road maintenance of about 1,100 km, (iii) road upgrading of about 440 km, and (iv) bridge replacement, totaling about 3,750 linear meters; and (b) consulting services for supervision of civil works that provided a set of international and domestic consultants to assist and strengthen DGH and Dinas Bina Marga (highways directorates of provincial public works departments or DBMs) who were responsible for project implementation. The main tasks of the consultants were to undertake design reviews and construction supervision. The executing agency for this part was DGH under the Ministry of Public Works (MPW). Part B Vehicle Inspection and Testing, which aimed to enhance vehicle safety and compliance with environment standards, on a pilot basis, in the province of West Nusa Tenggara. The executing agency for this part was the Directorate General of Land Communications (DGLC) under the Ministry of Communications. This part included civil works for testing stations and offices, supply of equipment, and related consulting services. Part C Institutional Strengthening, which aimed to improve DGH s road network planning capacity and to upgrade the professional qualifications of its senior staff through (a) provision of consulting services to undertake a study on the future road network development in the project provinces; and (b) postgraduate training of about 20 DGH staff in highway engineering. The executing agency for this part was also DGH under MPW. 1 In April 2000, under the National Unity Cabinet, DGH was replaced by the Directorate General Regional Infrastructure (DGRI) of the Ministry of Settlements and Regional Infrastructure (MSRI).

11 2 II. EVALUATION OF DESIGN AND IMPLEMENTATION A. Relevance of Design and Formulation 4. The Project was designed to support the then strategic objectives of the Asian Development Bank (ADB) in Indonesia of supporting economic growth, human resources development, and sustainable utilization of natural resources. The road sector assistance strategy aimed at alleviating capacity constraints and raising the efficiency of existing assets through rehabilitation and upgrading. The Project was designed as a sector loan to finance a 3-year time slice of the Government s road sector program in selected provinces. The geographic focus of the Project on 13 eastern island provinces supported the Government s efforts to achieve balanced and equitable development throughout the archipelago. These provinces clearly lagged behind the western islands in infrastructure development, particularly in roads. The project design relied heavily on existing planning and design tools used in DGH and the construction management arrangements of earlier ADB projects. The subprojects were to be selected using integrated road management system (IRMS) and bridge management system (BMS) models. These two systems were designed to analyze the whole road network on a uniform basis to identify priority investments. During project appraisal, staff focus was on agreeing on the subproject selection criteria and methodology, the implementation arrangements, and the terms of reference for consulting services. A detailed government Policy Statement and Action Plan for the Transport Sector (PSAP2) was also agreed. No project preparatory technical assistance (TA) was used for this loan. However, after loan approval, some funds ($0.4 million) were used in 1995 from the ongoing Eleventh Road Sector Project (Loan 1115-INO) for field review of the proposed subprojects prior to their selection. 5. While the fundamental concept of the project design was sound, four aspects on whether (i) the project as designed was in fact a sector loan, (ii) the project as appraised was ready for timely implementation, (iii) the geographic coverage of the project in 13 provinces and 139 locations was overextended, and (iv) the original loan amount of $180 million was not overestimated, need to be revisited. B. Project Outputs 6. Appendix 1 shows a chronology of milestone events of project implementation. A comparison of project components at appraisal and completion is made in Appendix 2. The Project was generally implemented in line with the scope envisaged at appraisal, with some modification resulting mainly from the financial crisis of 1997/98. Due to prevailing budgetary constraints, the third phase of the civil works program (18 subprojects, $24.5 million) was cancelled, although some of its packages were shifted to the second phase of this Project as well as to another ongoing World Bank project. The civil works, equipment, and consulting services targets of the Project were mostly accomplished, and were even exceeded for the main road betterment activity. The significant devaluation of the rupiah against the dollar contributed heavily toward savings in the loan amount; a total $87.9 million, or nearly 50% of the original loan amount, was cancelled at different stages. 2 2 The first partial loan cancellation amounting to $53.4 million was made in July 1998 during the portfolio review; the second partial loan cancellation of $10.0 million was made in September 1999 due to a shortfall in counterpart funds; and the third partial loan cancellation in the amount of $24.5 million was made in December 2000 because of the cancellation of the third phase of implementation.

12 3 1. Part A: Road and Bridge Works 7. The Project s civil works were planned to be implemented through three phases, covering about 130 contract packages. The first phase package list, agreed by ADB in January 1996, included 67 packages for road works (1,404 km betterment, 615 km periodic maintenance, and 15 km upgrading) and 27 bridge replacement sections (3,077 m) amounting to Rp366.0 billion or 59% of the total civil works cost. The second phase included 18 packages for road works (600 km) and bridge replacement (701 m), amounting to Rp261.0 billion or 41% of the total civil works contracts. The third phase was supposed to consist of 18 packages for road works (818 km) and bridge replacement (1,213 m). During the financial crisis, almost all ongoing project packages suffered from lack of local counterpart funds and rupiah price escalations. The Government and ADB agreed to cancel phase 3 of the Project; two packages from this phase were shifted to the restructured phase 2. In 1998, the Ministry of Public Works, to cope with virtually stagnant progress, offered all contractors of ongoing projects voluntary unconditional termination of contracts, if they were unable to complete their project on the terms of stipulated price escalation. Twelve contractors chose to do so and these works were subsequently re-bid, with some contracts subdivided into smaller packages. The contractors who chose to stay on the job had to essentially absorb the price escalation cost on their remaining works. However, the final costs of all remaining contracts turned out to be significantly higher than the original contract prices. The list of major civil works contracts, original and after cost escalation is in Appendix Delay in Project Start-Up. Project implementation had a slow start in Selection of phase 1 subprojects, preparation of detailed designs, and selection of supervision consultants were the main activities planned for the first year of implementation. The Government submitted its proposal for phase 1 packages to ADB in July After various clarifications, this package was approved in January 1996 and the first contract of phase 1 was made in February However, construction works for road betterment and upgrading could not start immediately after bid awards as the contractors were supposed to undertake some necessary site investigations, on the basis of which the supervision consultants were to review and verify the designs. 9. Construction works started in late Except for the simpler periodic maintenance and some bridge replacement works, progress on almost all other contracts was stalled in the middle of 1997 due to the financial crisis. A number of contracts had to be retendered, others restructured. The government budget allocation for counterpart funds in FY1998 dropped by nearly 60%, prompting ADB to raise its disbursement percentage to 80% for that year to mitigate the difficult situation. About 60 contract packages (44% of total works) were completed around the end of 1998 and another 43 (31%) were completed by the original loan closing date of 31 December The remaining 34 contracts, mostly retendered, and subprojects with construction or contractual problems, extended into 2000 and All contract packages were completed by the revised loan closing date of 31 March 2001 but final payments to contractors were extended to 15 November 2001 pending agreement on price escalations. The final outcomes of the Project are summarized in Table 1 below, which shows that while the physical targets for the road betterment component were exceeded by 66%, 3 the accomplishments for the other components were slightly below the original targets. In the case of the upgrading components, which required a feasibility study, the approvals were slow in coming and little upgrading work was included in the first phase. 3 Most of the roads to be periodically maintained were found structurally defective, therefore parts of the periodic maintenance works were shifted to road betterment.

13 4 Table 1: Planned and Actual Civil Works Component Component Unit Planned Actual Change (%) Betterment Periodic Maintenance Upgrading Bridge Replacement (km) (km) (km) (m) 980 1, ,750 1, , km = kilometer, m = meter. Source: Project Completion Report, Directorate General of Regional Infrastructure, February Simplified Design Approach. All contracts were tendered on the basis of a simplified design, generally produced by the regional betterment offices (RBOs), based on DGH s simplified road design program. No site topographical, drainage, or geotechnical survey data were available for doing the designs. The final design and estimate of construction quantities were undertaken during design reviews, supposed to be carried out during the first 3 months of the construction contract. Approval for these designs was required from the subproject manager, project manager, core team consultants, RBOs, DGH, and ADB. This was a timeconsuming and cumbersome process, and was further complicated by the fact that the core team consultant did not come on board until December 1996, over 2 years after loan approval. 11. Use of Cold-Mixed Lasbutag. 4 Several approved phase 1 designs provided for the use of cold-mixed lasbutag for wearing as well as binder courses, as part of the government policy for promoting local materials. A total of Rp131 billion, or 20% of the total cost for road improvement, was allocated for using the lasbutag. However, lasbutag turned out to be unsuitable and it failed in most applications even before the final handing over of the works. Part of the improved roads that used lasbutag were reoverlaid with asphalt hot-rolled sheet. The repairs were financed entirely by government funds. The list of road links that used cold-mixed lasbutag is in Appendix Part B: Vehicle Inspection and Testing 12. Construction of the four inspection and testing centers and procurement of fixed vehicletesting equipment as well as mobile roadside spot-check vehicles took place in four pilot cities in West Nusa Tenggara Mataram, Selong, Sumbawa, and Bima and was completed in July The upgrading of the Regional Training Center in Bali was completed in September Consultant advisory services for improved vehicle inspection and enforcement of policies were completed in June 1999; the final report was submitted to ADB in July Part C: Institutional Strengthening 13. The Study for Road Network Development the Eastern Islands was conducted by a group of international consultants during October 1997 to January 1998; the final report was submitted in March The study area included 13 provinces in five main islands. The study was designed to review the existing road network, review the medium-term future requirements of road network development in the context of the Government s overall development program for the region, prepare proposals for provincial road master plans, and identify priority 4 Lasbutag is natural asphalt mineral from Buton Island in South-East Sulawesi that consists largely of natural rock asphalt, aggregate, and modifier.

14 5 investments and conduct prefeasibility analyses on them. The consultants, after undertaking an overview of the road network in the study area, identified four priority provinces for road network investments West Nusa Tenggara, North Sulawesi, South Sulawesi, and Maluku. Detailed work on Maluku could not be undertaken due to lack of reliable data. The consultants thus only focused on the remaining three provinces to conduct road viability analysis, i.e., data collection/analysis, screening/prioritization of road corridors, and recommendations. The main recommendation of this study was in the form of a set of investment proposals for external financing. Under a new ADB loan (Loan 1798 INO: Road Rehabilitation Project, November 2000), only one of the above provinces (North Sulawesi) was included in the new project since the rest had already been financed by other external agencies. 14. Under the Project, 27 DGH staff obtained a Masters Degree (nine from France and 18 from Australia) in Highway Engineering, exceeding the targets at appraisal of 20 staff. The average training cost was about $37,000 per participant. All of them are now involved in the technical and implementation directorates of DGH and provincial highway offices. This component was implemented successfully. 4. Sector Policy Matters 15. As a sector project, the Government and ADB were supposed to have close consultation and to monitor the progress of highway and road traffic and transport policy implementation, in particular further road sector policy reforms beyond the project time frame. The Government was also supposed to keep ADB informed of other significant programs and studies related to road and road transport sector development. During the project period, both ADB and the Government did follow up on policy matters but on occasions either the Government s response was slow or issue resolution took some time. This is due to the fact that, as policy issues, there was caution in resolving them and decisions were required from higher levels of government. C. Project Costs and Financing 16. The total project cost estimated at appraisal was $332.9 million equivalent. The ADB loan of $180.0 million was to finance all foreign exchange costs ($160.6 million), including interest during construction and $19.4 million of local costs, and the Government was intended to provide $152.9 million as counterpart funds. The total actual project cost amounted to $168.6 million equivalent for which the ADB loan financed $79.8 million and the Government provided $88.8 million equivalent. Appendix 5 shows a detailed comparison of planned and actual project costs and financing arrangements. A summary of the appraisal and actual project costs is in Table 2 below.

15 6 Table 2: Project Financing ($ million) Appraisal Estimate Actual Change Foreign Local Total Foreign Local Total Foreign Local Total Item Exchange Currency Cost Exchange Currency Cost Exchange Currency Cost Borrower (64.1) (64.1) Financed % change (42.0) (42.0) ADB (88.4) (11.8) (100.2) Financed % change (55.0) (60.8) (55.7) Total (88.4) (75.9) (164.3) % change (55.0) (44.1) (49.4) ADB = Asian Development Bank. Source: Staff estimates. 17. While the overall project targets were largely met, the final dollar-based project costs were much lower than the appraisal estimate due to a fourfold devaluation of the rupiah. As most of the project contracts involved local costs, a comparison in rupiah costs is in order. Overall, the initial signed contracts amount for about 120 contracts was Rp605.3 billion, while the final completion cost for these packages was Rp651.3 billion, an increase of 8%. Most of the local cost increases were encountered on nearly half of the betterment contracts implemented during FY1997 and after. Twenty-six betterment contracts totaling Rp137.3 billion were completed in the first year with only 4.5% of total increases in price (both physical and price escalations). Twelve contracts were terminated, of which 10 were retendered. Another 14 contracts, which were implemented during the financial crisis, registered a 77% price hike on their total value. Overall, the price escalation for the betterment component amounted to 6%. For periodic maintenance, road upgrading, and bridge replacement components, the price escalations were 11%, 6%, and 12%, respectively. The ADB appraisal cost estimates for civil works turned out to be conservative. It is estimated that, if the crisis had not occurred, the loan amount would still have registered a saving of about $30.0 million. Given the spiraling price increases in construction, the overall cost performance of the Project is rated satisfactory. D. Disbursements 18. Total ADB disbursements amounted to $79.8 million, or 87% of the revised loan amount of $92.1 million after cancellation of $53.4 million in July 1998, $10.0 million in September 1999, and $24.5 million in December Appendix 6 shows actual contract amounts and the disbursement status as of 31 March A special account was established to facilitate the timely release of loan proceeds. In general, payments to contractors, suppliers, and consultants were made on time and in accordance with government and ADB procedures, except for those payments to 11 contractors that needed extended time for clearance from the Government for the calculation of price escalation. However, the submission of withdrawal applications from the Direktorat Tata Usaha Anggaran (Directorate of Budget Administration) to ADB was substantially delayed. 19. As a result of the delay in implementation during the first 2 years, the accumulative disbursement made in 1995 was only $5.0 million against a target of $21.0 million, and $10.8

16 7 million against a target of $81.0 million in In 1998, ADB agreed to increase its financing ratio to 80% for 1 year to cope with the counterpart funding shortages. This resulted in an extra disbursement of about $1.75 million. E. Project Schedule 20. The overall project implementation schedule as appraised and implemented is shown in Appendix 7. The Project was approved on 29 November 1994 and became effective on 20 June The loan closing date (31 December 1999) was extended twice due to delays in completion of the civil works resulting from a slow start-up and the impact of the 1997 crisis, which stalled most of the contracts for some time. The loan was closed on 31 March 2001, about 15 months after the original closing date. It should be noted that the first 18 months of project implementation were mainly devoted to project preparation activities. In view of the difficult circumstances faced by the Project, such as weak preparation, a slow start, and the financial crisis, overall implementation was satisfactory. The physical targets were largely achieved, albeit behind schedule. The related institutional development components were also implemented within a reasonable time frame after the Project got under way. F. Implementation Arrangements 21. At appraisal, it was agreed that DGH would be the executing agency for the Road and Bridge Works (Part A) as well as Institutional Strengthening (Part C), while DGLC would be responsible for Vehicle Testing and Inspection (Part B). In April 2000, the Government ministries were reorganized: the public works functions were transferred to the Ministry of Settlements and Regional Infrastructure (MSRI). Most of the DGH staff and functions were shifted to the Director General Regional Infrastructure (DGRI), responsible for inter-urban roads, and water and sanitation infrastructure. This change did not have any material effect on project implementation as staff continuity was maintained. Appendix 8 shows the broad implementation arrangements for the Project. The Directorate of Planning in DGH was responsible for overall coordination and approval for planning, programming, funding, design, and supervision. The design and supervision functions were delegated to the RBOs and, in the case of national roads, to the National Roads Planning and Programming Office (P3TNAS), which recruited consultants for support. All civil works for roads and bridges were coordinated by the provincial governments DBMs (and sub-dbms) through their respective appointed project and subproject managers. All design changes produced by RBOs or P3TNAS were subjected to approval by DGH s Directorate of Engineering (Bintek). DGLC, responsible for the vehicle testing and inspection component, appointed its Subdirectorate of Road Transport Technical Standards as the executing agency. G. Conditions and Covenants 22. The Loan Agreement included a total of 33 covenants, of which 23 were project specific. While, overall, a large number of the covenants were complied with, a number of conditions were not fully complied with, including: (i) there was no timely provision of adequate counterpart funds; (ii) a number of employed contractors turned out to be incompetent; (iii) subprojects were designed using DGH design standards (the simple design criteria themselves turned out to be inadequate and had to be modified during implementation); (iv) DGH did not carry out a regular field-monitoring program to ensure environmental measures; and (v) ADB was not regularly informed of the progress of PSAP2 (annual status reports were not submitted). Appendix 9 summarizes the loan covenants and compliance.

17 8 H. Related Technical Assistance 23. No accompanying TA related to the Project was provided. I. Consultant Recruitment and Procurement 24. The major consultancy contract under the Project was for supervision services, which required expatriate consultants as the core team and seven groups of domestic consultants as field teams. The core team consultants were selected through international competition in which six short-listed firms participated. The evaluation of the submitted proposals was completed in January 1996, but contract approval and award were delayed to December The mobilization of the core team was on the critical path of the civil works construction, as the consultants had to review the designs prior to start of construction on betterment and upgrading subprojects. Most of the civil works contracts for the phase 1 packages were awarded in March/ April 1996; the field teams were also selected at about the same time. However, the process of selecting the expatriate consultants turned out to be rather slow. The core team was finally mobilized in around December 1996, nearly 12 months later than planned. This turned out to be a major contributing factor toward implementation delays. In retrospect, it would have been better if critical steps such as advance recruitment action of consultants had been completed prior to, say, loan negotiations. The consultants for the vehicle testing program and road sector study were involved only minor delays. In the end, the international consultant inputs totaled 189 person months against 187 planned, and those of the domestic consultants were 4,150 person months against 4,265 planned. Summary of the consulting services is given in Appendix There were a total of 155 contracts under the project: 139 for civil works, 8 for supervision consulting services, and 8 for the rest of the components. Procurement of civil works was undertaken mostly through local competitive bidding; 10 civil works contracts were procured through international competitive bidding along with all expatriate consultants and equipment. The seven domestic consultants for field supervision teams were selected through local competition. Further details are given in Appendix 11. J. Performance of Consultants, Contractors, and Suppliers 26. Consulting Services Arrangements. The Project used the formula that was adopted for earlier ADB projects and was deemed successful, in which a centrally based team of coordinating consultants provides advisory services to DGH and the RBOs, and directs the activities of teams of field-based consultants organized by the respective RBOs. The terms of reference and other arrangements for the supervision consultants were to be similar to the ones used in earlier projects. One of the crucial tasks of the supervision consultants was to review the subproject designs, prepared by RBO staff, before the start of construction. Given the dispersed location of project components, it was decided to have four core teams, one located in Jakarta, and three regional teams based in Surabaya, Banjarmasin, and Makasar. Seven groups of fieldbased supervision teams were located in 11 provinces, working under the guidance of the core teams, but reporting to the relevant local RBOs. Each team included a site engineer, a chief inspector, a quantity surveyor, inspectors, and technicians. The recruitment of supervision consultants faced delays until January 1997 when the four core teams were mobilized. The field teams were fielded in August The original contract period for the consultants was 3 years, ending December As construction works were extended into 2000, a number of adjustments in the staffing of expatriate consultants were made during implementation to extend the time period, but with a reduction of staff inputs to manage costs. The contract for three of

18 9 the core teams was thus extended by 1 year to December At the time of the demobilization of the core teams, 11 contract packages were ongoing. The key functions of the supervision teams can be summarized as follows: (i) (ii) (iii) the lead core team, based in Jakarta, was responsible for overall project monitoring and coordination, and provided support to the central DBM; three core teams, based in Surabaya, Banjarmasin, and Makasar, assisted RBO chiefs in design reviews and DBM project managers in construction supervision, coordinated the work of field teams, and served as advisors to project managers; and the seven field supervision teams, hired by RBOs under separate contracts, were responsible for site inspection and contract management. 27. Performance of Consultants. The performance of the core team supervision consultants was generally satisfactory. However, the performance of field teams was less than satisfactory, as these teams were not always staffed adequately and the field team staff were not located with the project managers. The district offices responsible for project implementation did not have the capacity to undertake the required construction management. The core team consultants provided overall coordination and support to DGH, field teams, and the local project offices. The field teams were carrying out the day-to-day construction supervision and field management. These were elaborate and complicated arrangements, which have been used on other ADB projects as well. The overall cost of construction supervision under this project was 7% of the total civil works cost, compared to 8% in the original estimate. While the overall results and costs of these arrangements were satisfactory, one obvious weakness of this set-up is that the consultants seem to undertake the supervision activities on their own, rather than working in a supporting role to the clients. For example, the consultants issued all project progress reports independently with little contribution from the government agencies. There is not much evidence of capacity building as a result of these services at the local level. It is recommended that the supervision consultant services should be structured to support the local implementation agencies rather than work instead of them. 28. Performance of Contractors. All civil works contractors involved in the Project were domestic contractors. There were 139 contracts for civil works involving about 80 domestic contractors. A number of contractors could not mobilize their staff and equipment on time, thus causing delays in the overall project implementation. Poor construction quality was found in some visited road sections 5 in Central and East Kalimantan, and road settlements and land sliding 6 in South-East Sulawesi and East Nusa Tenggara. However, the performance of a large number of these contractors was generally satisfactory. 29. Performance of Suppliers. Two suppliers were involved in the Project: (i) supplier for the training center in Denpasar, Bali; and (ii) supplier for vehicles inspection and testing for Mataram, Selong, Sumbawa, and Bima. In general, performance of both suppliers was satisfactory without any notable delay in delivering the equipment. However, the supply contract did not have sufficient provision for spare parts as required by the Project. 5 The mission found total road damage under contract BF 4 in Pulang Pisau KM35 in Central Kalimantan and Bontang Sangata in Kutai in East Kalimantan. 6 Land slides and subgrade resettlement were found under packages of BE7: Kefamanu Maubesi and Lepo Lepo Ambesia in South-East Sulawesi.

19 10 K. Performance of the Borrower and the Executing Agencies 30. Overall, the performance of the borrower and its executing agencies is rated satisfactory. DGH has been the executing agency for a number of ADB projects; this was the 13 th road sector loan in Indonesia. The past performance of DGH has generally been rated satisfactory. In this project, the overall planning and coordination was with the central Government and the actual project implementation was delegated to the provincial offices. The Government generally lacked the staffing and skills to manage this large project on its own; it had to rely heavily on consultant support for design and supervision. DGLC as the executing agency of part B has implemented its component without delay. However, the operation and maintenance of the facilities were still limited as a result of limited operational incomes because of only commercial vehicles have to undergo testing. The performance of the executing agencies as well as the Government could have been improved by the following possible measures: (i) (ii) (iii) (iv) DGH should have undertaken more project preparation activities, such as selection of subprojects, field investigations, and designs for the first-year program, prior to loan approval. Selection of supervision consultants should have been expedited; more efforts toward timely provisions of counterpart funds in the government budget should have been made; there should have been less reliance on consultants for project support. Consultants should have been hired with a specific objective of gradual capacity building and should work closely with the implementing agencies; and Ministry of Finance should have adopted a more proactive approach toward cancellation of the loan savings and submitting withdrawal applications in a timely manner. L. Performance of the Asian Development Bank 31. Although the Project was inadequately prepared for timely implementation since the first 12 to 18 months after loan approval were essentially devoted to project preparation activities, the performance of ADB during implementation is rated satisfactory, as the ADB staff managed to identify and address various implementation problems. The loan administration responsibility for this Project was transferred to the Indonesia Resident Mission in January 1997, a move that the government side welcomed. Interaction between ADB and the executing agencies became more frequent and easier. The following possible actions could have further enhanced ADB interventions in the Project: (i) Advancement in the project cycle of key project actions, such as selection of phase 1 subprojects, design reviews/approvals, and consultant selections should have been prerequisites for loan negotiation. ADB has to convince the Government that implementation of advance procurement action provided for in the Loan Agreement is the key for implementing the project on schedule. Although it implies a risk to the Government, since the project should start before signing the loan, it would benefit the executing agency to implement the project as early as possible.

20 11 (ii) A more balanced focus is needed between physical components and road sector policy issues and developments. The Project was designed as a sector loan to support the Government s road sector investments and policies. ADB tended to act as though it was a project loan without providing adequate follow-up actions for the Government s other road investments and policies. III. EVALUATION OF PERFORMANCE A. Relevance 32. The project design a sector loan to support the Government s road sector development programs and policies under the sixth Five-Year Development Plan (Repelita VI) in the eastern islands was sound and relevant. The Project was in line with the focal points of ADB s recently developed country strategy and programs of (i) targeting less developed islands, (ii) generating direct impact to economic growth in cities that are linked by improved and upgraded roads, (iii) supporting the development of the eastern islands economic integrated zone (KAPET) 7 that is intended to alleviate poverty by generating job opportunities in the community, and (iv) human resources development through the provision of support for institutional development and capacity building. The main changes made during implementation related to improvements in design criteria and an increase in the ADB financing share for 1 year to cope with the Government s tight budgetary constraints. B. Efficacy in Achievement of Purpose 33. The Project achieved its stated physical objectives of improving road sections and bridges, despite delays and difficulties during implementation, at acceptable cost and within a satisfactory time frame. Some minor adjustments in project scope were made following the financial crisis. Had these difficulties not occurred, the Project would have achieved much higher physical implementation targets within the approved loan amount. On the policy front, the Project aimed to support implementation of PSAP2. The Project included a covenant to annually review progress in implementation of PSAP2 and to hold consultations for further road policy measures. Overall, implementation of PSAP2 was slower than planned, but the main accomplishments were the issuance of a number of pending decrees or implementation regulations relating to the 1992 Road Law and the undertaking of a study to further reform the road user charges system. PSAP2 was updated in 1996 to become PSAP3, covering implementation of sector policies for the period However, implementation of PSAP3 was suspended during the crisis. Its revival is now proceeding under the new Roads Rehabilitation Project (1798-INO). The institutional development objectives relating to road safety and environment have been partly achieved, since only the provision of a training center and equipment in Denpasar and vehicle inspection and testing in Mataram are fully operational. C. Efficiency in Achievement of Outputs and Purpose 34. Financial analysis is not applicable for this Project as no cash generation is involved. Economic analysis was undertaken using well-established integrated road management system 7 Most of the roads selected for improvement are located in the KAPET area (South Kalimantan and South Sulawesi).

21 12 and bridge management system models. 8 The main benefits anticipated from the Project were in time and in vehicle operating costs resulting from improved road conditions. The subproject selection criteria stipulated that only projects with an economic internal rate of return (EIRR) higher than 15% (and 12% for upgrading components) could be included. DGH has a national program of collecting annual traffic and road condition data for all national and provincial roads. Although these data are supposed to be collected four times a year, they are only collected once a year due to budget constraints. The main problem with this analysis is reliability of the available data. Using the actual traffic and road condition data available at the DGRI, and the actual construction costs, the economic analysis was done for all major links in the project using the same methodology. Appendix 12 provides a comparison of the estimated EIRR at appraisal and those based on the current actual data. The traffic growth in a number of areas was sluggish, mainly due to the difficult economic situation in the country since Of the reevaluated sections, 51% show EIRRs less than 15%. In terms of the objectives of the overseas Masters programs, these have been achieved efficiently with an average cost of $37,000 per participant, lower than the cost at appraisal of about $60,000 per participant. D. Preliminary Assessment of Sustainability 35. Part A Road and Bridge Works. Sustainability of the road and bridge components will largely depend on the quality of completed works and subsequent required routine and periodic maintenance by the Government. The latter requires the Government to commit funds and resources to operation and maintenance (O&M). The allocated routine maintenance budget by the provincial government of about Rp10.0 million to Rp15.0 million per kilometer per year was adequate only for the maintenance of road surface layers and shoulders. Consequently, other supporting facilities such as side ditches and road drainage have not been properly maintained. Although sustainability of the project benefits may be hampered by the lack of a routine maintenance budget, the risks are minimized by the fact that the Government also provides another program called road periodic maintenance for those subproject components that have reached their 5 years of economic life, in the form of hot-mixed asphalt overlay and rehabilitation of supporting facilities on a selected basis. The final report of the core team consultants as well as selected field visits by the Project Completion Review Mission confirm that the quality of completed subprojects was generally satisfactory. It was found that 16 road links have been identified with construction deficiencies, which DGRI needs to address. In 2002, the Government started rehabilitating part 9 of these roads. The list of roads with construction deficiencies is in Appendix Part B Vehicle Inspection and Testing. While the construction of the testing facilities, procurement of equipment, and training of local staff by consultants proceeded satisfactorily, the actual operation of these facilities has not been without problems: (i) (ii) as part of the Government s deregulation program of 1998, the operational responsibility for these facilities has been shifted to the district government level (kabupaten). Due to the limited operational incomes from the facilities, the district governments remain subsidized for O&M costs; the present regulations (PP-44) require only commercial vehicles to undergo testing, which limits the coverage available for these facilities; and 8 These systems prioritize and plan work programs for maintaining and improving roads and bridges. They are the main tools for evaluating and justifying new project proposals. 9 Road sections at Kefamanu Maubesi in West Timor and Pulang Pisau in Central Kalimantan.

22 13 (iii) the spare parts of the facilities were not available in a timely manner. 37. Currently, only one facility in Mataram is operating satisfactorily; the other three are only marginally operational. DGLC is reviewing the current regulations with the possibility of expanding the scope of vehicles to undergo testing, which will not be limited to commercial vehicles. 10 At this stage, only the facility in Mataram is likely to be sustainable. 38. The training center in Denpasar, Bali is supposed to provide training facilities for provincial and district land communications staff in the eastern islands. The training center was put into operation in September 1999 to provide four training courses a year, each of three months duration. The training participants have substantially gained from the training provided by the Project. In late 2000, the training center was moved from the provincial land communications office to land owned by the training division of DGLC. Since the new training center is still being completed, four training courses scheduled in FY2002 will have to be moved to Jakarta. The new training center is scheduled for completion in October From observations during the Project Completion Review Mission, the training facilities appear to be sustainable. E. Environmental, Sociocultural, and Other Impacts 39. Road betterment, periodic maintenance, and bridge replacement works were not expected to have any major environmental and social impact as these works were intended to improve the existing infrastructure. Similarly, the right-of-way requirements were estimated to be minimal as the existing alignments were to be used. The project selection criteria stipulated that all subprojects to be funded by the Project have to be subjected to appropriate environment and social impact screening criteria. It was also agreed that a mechanism would be established to strengthen the institutional capacity of DGH and DBMs to screen subprojects and monitor mitigating measures. Except for two road links (BF 53 and BD 29), most of the completed subprojects did not have much of an environmental or social impact, although compliance with these agreements was marginal. There is little evidence that the required environment and social impact analysis was done prior to subproject selection. ADB review missions repeatedly requested DGH to put in place monitoring and mitigating measures, but DGH was unable to comply since it was of the view that the Project is based on existing roads rather than developing new ones. 40. Two road links were identified as having serious potential for negative environmental impacts BF 53 (Bontang Sangata in East Kalimantan), a 55.8 km road passing through a national park preservation area, and BD 29 (Lususua Lapai II, in South-East Sulawesi), a 41.0 km section passing through a conservation forest. The BF 53 road link was completed without information on the environmental issue by DGRI. The issue was noted by ADB during review mission visits. This road link also suffers from very poor construction quality; many land slides and pavement structural failures have already occurred. At the request of ADB, DGH asked the core team consultant to prepare environmental monitoring and management reports for these two road sections. These reports recommended a series of monitoring and mitigating measures. Due to delay in the submission of an environmental mitigation assessment report by DGRI, BD 29 in South-East Sulawesi was finally taken out of the program. 10 Law No. 14 (UU No. 14, 1992) requires that all types of motorized vehicles have to undergo testing. The law has been gradually implemented taking into account users affordability. PP-44, which requires only commercial vehicles to be tested, is now being reviewed.