IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-71190, IDA-42980) ON A LOAN IN THE AMOUNT OF US$40 MILLION AND ON A CREDIT

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Document of The World Bank Report No: ICR2490 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-71190, IDA-42980) ON A LOAN IN THE AMOUNT OF US$40 MILLION AND ON A CREDIT IN THE AMOUNT OF XDR 24.9 MILLION (US$77.5 MILLION EQUIVALENT) TO THE INDEPENDENT STATE OF PAPUA NEW GUINEA FOR THE ROAD MAINTENANCE AND REHABILITATION PROJECT (RMRP) December 20, 2012 Sustainable Development Department Timor-Leste, Papua New Guinea & the Pacific Islands East Asia and Pacific Region

2 CURRENCY EQUIVALENTS (Exchange Rate Effective, November 15, 2012) Currency Unit = Papua New Guinea Kina (PGK) PGK 2.06 = US$1.00 US$1.00 = PGK 0.47 FISCAL YEAR January 1 to December 31 ABBREVIATIONS AND ACRONYMS AGO AusAID AWP BIBMS BMS BOQ CAS CSTB DoW EMMP EPM ERR GoPNG ICR ISR MTR NCB NEC NPV NRA OEC OED PAD PDO PEPM PGK PNG PNGSDP PWM QAG RAMS RMRP SEA SIS USD VAT Attorney General s Office Australian Agency for International Development Annual Work Plan Bridge Inventory and Bridge Management Systems Bridge Management System Bill of Quantities Country Assistance Strategy Central Supply and Tenders Board Department of Works Environmental Management and Monitoring Plan Employer s Project Manager Economic Rate of Return Government of Papua New Guinea Implementation Completion and Results Report Implementation Status and Results Report Mid-Term Review National Competitive Bidding National Executive Council Net Present Value National Roads Authority Office of Environment and Conservation Operations Evaluation Department Project Appraisal Document Project Development Objective Provincial Employer s Project Manager Papua New Guinea Kina Papua New Guinea Papua New Guinea Sustainable Development Program Provincial Works Manager Quality Assurance Group Road Asset Management System Road Maintenance and Rehabilitation Project Sectoral Environmental Assessment Socio-Economic Impact Study United States Dollars Value-Added Tax i

3 Vice President: Pamela Cox Country Director: Ulrich Zachau Sector Manager: Michel Kerf Project/ICR Team Leader: Jim Reichert ICR Principal Author: Scott Wilkinson ii

4

5 INDEPENDENT STATE OF PAPUA NEW GUINEA Road Maintenance and Rehabilitation Project (RMRP) Datasheet... iv 1. Project Context, Development Objectives And Design Key Factors Affecting Implementation And Outcomes Assessment Of Outcomes Assessment Of Risk To Development Outcome Assessment Of Bank And Borrower Performance Lessons Learned Annex 1: Project Costs And Financing Annex 2: Outputs By Component Annex 3: Economic And Financial Analysis Annex 4: Bank Lending And Implementation Support/Supervision Processes Annex 5: Beneficiary Survey Results Annex 6: Summary Of Borrower's Icr And/Or Comments On Draft ICR Annex 7: List Of Supporting Documents Map iii

6

7 A. Basic Information Country: Papua New Guinea Project Name: Project ID: P L/C/TF Number(s): Road Maintenance and Rehabilitation Project IBRD-71190,IDA ICR Date: 12/23/2012 ICR Type: Core ICR Lending Instrument: SIL Borrower: Original Total Commitment: Revised Amount: USD 76.26M Environmental Category: B THE INDEPENDENT STATE OF PNG USD 40.00M Disbursed Amount: USD 77.52M Implementing Agencies: Department of Works Cofinanciers and Other External Partners: AusAID Papua New Guinea Sustainable Development Program Ltd Government of Papua New Guinea Provincial Governments - Papua New Guinea New Britain Palm Oil Ltd (NBPOL) B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 01/20/1995 Effectiveness: 02/07/ /07/2003 Appraisal: 05/02/2001 Restructuring(s): Approval: 06/06/2002 Mid-term Review: 07/03/ /15/2011 Closing: 12/31/ /30/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Satisfactory Moderate Satisfactory Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Moderately Satisfactory Quality of Supervision: Satisfactory Implementing Satisfactory iv

8 Overall Bank Performance: Satisfactory Agency/Agencies: Overall Borrower Performance: Moderately Satisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project Yes at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: Yes Satisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration Rural and Inter-Urban Roads and Highways Sub-national government administration 9 9 Theme Code (as % of total Bank financing) Decentralization Infrastructure services for private sector development Micro, Small and Medium Enterprise support Participation and civic engagement Rural services and infrastructure E. Bank Staff Positions At ICR At Approval Vice President: Pamela Cox Jemal-ud-din Kassum Country Director: Ulrich Zachau Klaus Rohland Sector Manager: Michel Kerf Jitendra N. Bajpai Project Team Leader: James A. Reichert Hatim M. Hajj ICR Team Leader: James A. Reichert ICR Primary Author: Scott Jared Wilkinson v

9 F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To promote an efficient, safe and reliable roads transport system in the participating provinces through: (a) the improvement of selected road segments; b) strengthening strategic planning and management of the road sector, and (c) strengthening the institutional arrangements for road maintenance, and (d) including private sector participation. Revised Project Development Objectives (as approved by original approving authority) Not Applicable. (a) PDO Indicator(s) Indicator Indicator 1 : Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Percentage of national roads in good or fair condition based on RAMS condition survey is sustainably maintained each year through Value quantitative or Qualitative) 20% 70% 74% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % achievement) Indicator 2 : GoPNG was keen to improve road conditions so made this a priority and coordinated with the funding agencies to provide needed funds for road maintenance. Between 2002 and 2009, reduction by 20% of the percentage of households in sample villages with household income under K100/month by Value quantitative or Qualitative) 25% 5% 5.5% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % This target was marginally achieved. achievement) Between 2002 and 2009, reduction by 30% in percentage of students in sample Indicator 3 : villages who report having to spend more than one hour travel time to get to nearest primary school each day. Value quantitative or Qualitative) 35% 10% 18.8% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % achievement) Indicator 4 : This target was met based largely on students traveling to schools that were located along the roads that were improved. Between 2002 and 2009, overall reduction of 30% in percentage of respondents in sample villages who report travel time to nearest health facility of one hour or vi

10 more. Value quantitative or Qualitative) 40% 10% 37.5% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % achievement) This target was met based largely on patients traveling to clinics that were located along the roads that were improved. Indicator 5 : RAMS generated program budget for periodic maintenance on national roads increases by 30% by Value quantitative or Qualitative) 70% 100% 55% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % achievement) While GoPNG accepted and implemented the RAMS as a management tool, funding was too low to make it fully operational. Indicator 6 : Percentage of DoW budget on maintenance and preservation in the participating provinces is maintained. Value quantitative or Qualitative) 30% 95% 55% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % achievement) This target was not achieved because scarce financial resources were utilized for emergency purposes i.e., bridge and road washouts and landslips, etc. Indicator 7 : Percentage of program budget for routine maintenance of national roads based on acceptable maintenance management system increases to 75% by Value quantitative or Qualitative) 10% 75% 67% Date achieved 05/10/ /31/ /30/2012 Comments Although the target was not met, it is a big improvement as funding for routine (incl. % road maintenance is chronically underfunded. achievement) Indicator 8 : Percent of maintenance expenditure bid competitively. Value quantitative or Qualitative) 40% 100% 90% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % achievement) The ten percent shortfall can be attributed to the single force account contract on Manus Island, and a couple of sole source contracts. Otherwise, a big improvement. Indicator 9 : Number of provinces that submit annual work programs in each year. Value quantitative or Qualitative) 50% 100% 100% Date achieved 05/10/ /31/ /30/2012 vii

11 Comments (incl. % Provinces were trained to submit annual work programs and all subsequently did. achievement) Number of provinces that submit management claims in a timely and accurate Indicator 10 : way. Value quantitative or Qualitative) n/a 100% 100% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % Provinces were trained to submit management claims and subsequently did so. achievement) Percent of maintenance expenditure bid competitively in the project provinces is Indicator 11 : increased over time. Value quantitative or Qualitative) 40% 95% 95% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % achievement) The target was achieved with increased number of maintenance contractors trained under the project. Dow's policy to combat the spread of HIV/AIDS is included in all major contracts by 2009, as measured by the inclusion of either FIDIC Technical Indicator 12 : Specifications on HIV/AIDS, or the Bank's performance based contracts by Value quantitative or Qualitative) 0% 100% 100% Date achieved 05/10/ /31/ /30/2012 Comments (incl. % achievement) The target was met after special provisions were included in the standard bidding documents for works under the project. (b) Intermediate Outcome Indicator(s) Indicator Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Indicator 1 : Total kilometers of national roads with restoration/specific maintenance. Value (quantitative or Qualitative) Date achieved 05/10/ /31/ /30/2012 Comments (incl. % achievement) The target was not achieved because: (i) construction costs increased after the LNG project began; and (ii) more gravel roads were upgraded to seal than originally planned. Indicator 2 : Total kilometers of provincial roads with restoration/specific maintenance. viii

12 Value (quantitative or Qualitative) Date achieved 05/10/ /31/ /30/2012 Comments Provincial governments were keen to restore roads, but their budgets were very (incl. % tight, which limited the amount of kms that could be rehabilitated. achievement) Indicator 3 : Maintenance/rehabilitation of national bridges. Value (quantitative or Qualitative) Date achieved 05/10/ /31/ /30/2012 Comments A lack of qualified bridge contractors limited the ability to restore bridges (incl. % throughout the country. achievement) G. Ratings of Project Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD millions) 1 12/26/2002 Satisfactory Satisfactory /06/2003 Satisfactory Satisfactory /17/2003 Satisfactory Satisfactory /28/2004 Satisfactory Unsatisfactory /08/2004 Satisfactory Satisfactory /20/2005 Satisfactory Moderately Satisfactory /30/2005 Moderately Moderately Unsatisfactory Unsatisfactory /06/2006 Moderately Unsatisfactory Moderately Satisfactory /02/2006 Satisfactory Satisfactory /26/2007 Satisfactory Satisfactory /28/2008 Moderately Satisfactory Moderately Satisfactory /19/2009 Satisfactory Moderately Satisfactory /31/2009 Satisfactory Satisfactory /10/2011 Satisfactory Satisfactory /25/2011 Satisfactory Satisfactory /25/2012 Satisfactory Satisfactory H. Restructuring (if any) Restructuring Date(s) Board Approved ISR Ratings at Restructuring Amount Disbursed at Reason for Restructuring & Key Changes Made ix

13 PDO Change DO IP Restructuring in USD millions 07/03/2007 N S S /15/2011 S S Additional Financing through an IDA Credit of SDR The reasons for restructuring were limited to the extension of the project closing date from December 31, 2011 to June 30, 2012 and reallocation of remaining funds among existing project categories. I. Disbursement Profile x

14

15 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Strategic Context. At the time of preparation of the Road Maintenance and Rehabilitation Project (RMRP), Papua New Guinea (PNG) was ranked as a middle income country in terms of GNP per capita (~US$1,000/capita per annum). However, it has a dualistic economy dominated by the enclave capital intensive resource sectors, while 85 percent of the population subsists on subsistence agriculture and fishing. Following a severe financial crisis in 1994, driven by fluctuations in primary exports, an Economic Recovery Program was put in place with support from the International Monetary Fund and the World Bank. Through the late 1990s and early 2000s, substantial economic progress was made, but basic social and service delivery indicators continued to lag. The gains of the commodity-led recovery were not converted into positive development outcomes for the country s predominantly rural population. 2. Transport infrastructure underpins the efficient flow of production inputs and goods and ensures access to health, education and other basic services and means of livelihoods for both rural and urban communities. The provision and maintenance of transport infrastructure in PNG are complicated and expensive due to the country s rugged mountainous topography, unstable geology, high torrential rainfall, wide dispersal of population centers, unconnected modal and road networks, and fragmented and duplicative administrative systems. 3. There are around 30,000 km of roads in the country. The central government (Department of Works) is responsible for the national roads network (around 9,600 kms), while 19 provincial and 86 local government levels are responsible for the remainder of the network. Through the late 1980s and 1990s, central funding for provincial and local government services fell from 30 percent of the national budget to 20 percent as service delivery responsibilities were increasingly decentralized. At the central government level, budget priorities were skewed in favor of the construction of new roads and the central Department of Works (DoW s) annual budget for road maintenance and restoration was only 20 percent of calculated needs. The condition of the road network rapidly deteriorated: much of the paved network was potholed and in sections reverted to gravel and the condition of the unpaved network was rough to impassable. As a result, travel times and operating costs increased and local communities were increasingly isolated. 4. The Country Assistance Strategy (CAS) in place at the time the Project was prepared and appraised had the overarching goal of supporting poverty reduction through improved governance and focusing development on delivery of basic services assistance to the poorest and most vulnerable, particularly in rural areas. The Bank's assistance strategy ranged from macroeconomic and structural reforms, donor co-ordination and partnerships with nongovernmental organizations to a lending program aimed at improving private sector activity, public sector reform, and general service delivery. The Project specifically supported the CAS objective to improve the quality and reliability of the country's infrastructure, particularly the provincial and rural road network, to allow for more cost-effective movement of goods and services and promote economic development in rural areas". It was also consistent with the Government of Papua New Guinea s (GoPNG s) Medium-Term Development Strategy, that recommended improved quality of local services and devolution of responsibilities and resources to lower tiers of government for more efficient and accountable service delivery. 1

16 1.2 Original Project Development Objectives and Key Indicators 5. The original project development objective (PDO) was to promote an efficient, safe and reliable roads transport system in the participating provinces. Specifically, the PDO would be achieved through: (a) the improvement of selected road segments; (b) strengthening strategic planning and management of the road sector; (c) strengthening the institutional arrangements for road maintenance and (d) including private sector participation. 6. This was elaborated as key objectives with associated performance indicators as follows: a. Restore satisfactory physical condition and safety to parts of the network (improve selected road segments) of national and provincial roads and bridges in six of the country s nineteen provinces based on priority needs as measured by: i. percentage of national roads in fair or better condition; and ii. percentage of provincial roads in fair or better condition. b. Strengthen strategic planning and management of road sector assets as measured by: i. percentage of expenditure on maintenance and preservation; and ii. percentage of budgeted work program based on Road Asset Management System (RAMS). c. Strengthen institutional arrangements for road maintenance, including through an expanded provincial government role in line with the Government s decentralization policy and the New Organic Law as measured by: i. timeliness - number of provinces who submit Annual Work Programs in each year; ii. number of unified Departments of Works (DoW)/Provincial Maintenance; and iii. number of staff trained. d. Increase private sector contracting of road maintenance works, including through the strengthening of small and medium-sized contractors and participation by village communities as measured by: i. percentage of maintenance expenditure bid competitively. 7. The key outputs were: cumulative kilometers of national roads receiving restoration/specific maintenance; cumulative kilometers of provincial roads receiving restoration/specific maintenance; and cumulative number of bridges replaced. 1.3 Revised PDO 8. Project Development Objectives. No formal revisions were made to the PDO over the duration of the project. 9. Performance Indicators. When the Additional Financing operation was approved on April 26, 2007, the performance indicators were revised for clearer output and impact monitoring (quantitative and qualitative indicators), and to better measure project progress and achievements, as summarized in the following sections. It is important to note that the Project files comprise two separate files, one for the original Loan (P004397) and a separate file for the RMRP 2

17 Supplemental (Additional Finance P102792). For unknown reasons, the Task Team did not modify the approved amended indicators at the time of the Additional Financing and the indicators generated in the Implementation Status and Results Reports (ISRs) remained as per the original Loan. The approved amended indicators are identified in paragraph 10 below, while targets and end of project values are shown in the Data Sheet of the ICR. 10. Additional Financing Performance Indicators (Amended). PDO (a) Improve selected road segments, as measured by: Percentage of total national roads in Good and Fair condition based on RAMS condition survey is sustainably maintained 1 each year through 2009 (modified definition); Between 2002 and 2009, reduction by 20% of the percentage of households in sample villages with household income under PGK100/month (~US$1/day) by 2009 (new); Between 2002 and 2009, overall reduction of 25% in percentage of students in sample villages who report having to spend more than one hour travel time to get to nearest primary school each day (new); and Between 2002 and 2009, overall reduction of 30% in percentage of respondents in sample villages who report travel time to nearest health facility of one hour or more (new). PDO (b) Strengthen strategic planning and management, as measured by: RAMS-generated program budget for periodic maintenance on national roads increases by 30% by 2009 (modified definition); Percentage of DOW budget on maintenance and preservation in the participating provinces is maintained ( modified definition); Percentage of program budget for routine maintenance of national roads based on acceptable Maintenance Management System increases to 75% by 2009 (new); and Percentage of maintenance expenditure bid competitively (new). PDO (c) Strengthen institutional arrangements, as measured by: Timeliness: number of provinces that submit Annual Work Programs in each year (unchanged); and Timeliness: number of provinces that submit management claims in a timely and accurate way (new). PDO (d) Increase private sector contracting, as measured by: Percentage of maintenance expenditure bid competitively in the Project provinces is increased over time (modified definition); and 1 Roads in Good or Fair condition are those roads with an International Roughness Index of between 5-8 and 8-15 respectively in the project area. 3

18 DOW s policy to proactively combat the spread of HIV/AIDS is fully institutionalized on all major contracts by 2009, as measured by the inclusion of FDIC Technical Specification on HIV/AIDS OR paras (h) and (i) of the Bank s Output and Performance Based Road Contracts in all major contracts by 2009 (new). 11. Outputs. No changes. 12. Discarded Indicators. The following indicators were dropped under the Additional Financing: (a) percentage of provincial roads in fair or better condition (from PDO (a)); (b) number of unified Provincial Works Managers (PWM)/Provincial Maintenance (from PDO (c)); and (c) number of staff trained (from PDO (c)). 13. Reasons for Changes. The bulk of the original performance indicators were retained with the introduction of the Additional Financing, with revisions to the target values made based on the status of the project at the time. Six new indicators were included, new targets set and three indicators were dropped so as to: Reflect the scaling up of the project, both in terms of additional funds (IDA US$37.31 million equivalent), and the inclusion of two further provinces (Gulf and Western). Better capture several positive results found by the 2006 Socio-Economic Mid-Term Impact Study that found an increase in visits to towns and markets, and roadside sales because of more traffic and passengers on the roads, as well as improved access to markets and suppliers. Indicators to monitor socio-economic impacts (benefits and negative impacts) were therefore included for more regular monitoring and where possible, disaggregated by gender to monitor the impacts for men and women who would have different transport and security needs and priorities. Support the Government of Papua New Guinea s (GoPNG) policy with respect to HIV/AIDS. An indicator was added to reflect the adoption of HIV/AIDS mitigation measures under major road contracts funded under RMRP. Take into account that in some provinces, more concessional bi-lateral grant funds had become available under other donor programs and had been utilized to complete works, resulting in the need to revise downwards some of the Project-funded target values. The indicator regarding the percent of provincial roads in fair or better condition was therefore dropped and replaced by two indicators to monitor the robustness of the provincial work programming and contract management systems. The indicator measuring the unification of DOW/Provincial maintenance units was dropped as DOW no longer saw this as a priority given the dispersed nature of the country and the need for isolated provincial units to be strengthened rather than be reliant on distant central government support. 14. Core Indicators. Roads in good and fair condition as a share of the classified roads was already being monitored prior to the introduction of Core Indicators. A further core indicator Size of the total classified network was subsequently included and monitored. 1.4 Main Beneficiaries 15. The expected benefits arising from the improvement works were a reduction in vehicle operating costs, and improved accessibility to markets, health facilities, schools, social amenities 4

19 and employment opportunities for local road users. An additional anticipated benefit was fostering wider overall economic recovery and growth. Improved all-weather accessibility was (and still is) considered to be particularly important in PNG because owing to the terrain, there are very few substitute routes or modes of travel. 16. The total affected population in the six provinces at the time of project appraisal was estimated to be 1.3 million (27 percent of the country's population). Because road services, including haulage and public passenger transport, were - and still are - reasonably competitive, the benefits of improved road maintenance were expected to be passed on to local road users. 1.5 Original Components 17. The original loan and additional finance credit components and associated costs were: Component 1: Road Maintenance and Rehabilitation of National Roads (US$40.79 M) - Annual routine maintenance on about 2,200 km and restoration of about 440 km of national roads. Component 2: Maintenance and Rehabilitation of Provincial Roads (US$7.39 M) - Annual routine maintenance on about 900 km and restoration of about 200 km of provincial roads. Component 3: National Bridges (US$5.42 M). This component consisted of bridge maintenance and rehabilitation and replacement of bridges on national roads, as well as purchase of emergency bridges and bridge parts. Component 4: Provincial Bridges (US$0.36 M) - maintenance and rehabilitation and replacement of bridges on provincial roads. Component 5: Implementation Support (US$7.56 M) - consulting services and purchase of equipment to support project implementation, namely: a) Project management (Employer's Project Manager - EPM); b) Designs and supervision of roads to be restored and bridges to be replaced; c) Incremental operating costs by DOW, Provincial Work Managers, and Provincial Administration Staff to manage the project (including supervision of the civil works and monitoring compliance with environmental management plans); d) Laboratory equipment, traffic counting equipment, and environmental monitoring equipment; e) Provision of auditing services for DOW and participating provinces project accounts; and f) Strengthening the financial management and operational capacity of DOW. Component 6: Road Sector Support (US$2.45 M) - consulting services and training to: a) Strengthen small- and medium-sized contractors for road maintenance operations; b) Improve the institutional capacity of DOW (Headquarters and Provincial level) and provincial administrations for road maintenance; c) Update and improve the bridge inventory and bridge management system to be more effective as a management tool for DOW; and 5

20 d) Carry out a study of the socio-economic impacts of roads and bridges under the proposed project. 1.6 Revised Components 18. No revisions were made to the project components when the project was scaled up to two new provinces (Additional Finance). 1.7 Other Significant Changes 19. Amendments to Loan Agreement. Following early delays due to GoPNG s inability to provide counterpart funds in a timely way, in September 2004 an Amendment was made to allow for the special (later designated) account to be opened in local currency, six monthly external auditing of project accounts, establishment of a project steering committee and some streamlining to government procurement approval procedures. 20. First Extension to Loan Closing Date and Reallocation. The closing date was extended for twelve months to December 31, 2007 to compensate for earlier delays in project implementation as a result of slow counterpart funding provision, lack of familiarity with Bank procedures, and a slow response from some participating provinces. 21. Second Extension to Loan Closing Date. The closing date was extended for two years to December 31, As part of the Additional Financing, a second extension to the loan closing date was processed to December 31, 2011 to accommodate delays caused by the disruption of the work program, while the National Executive Council (NEC) was approving the new EPM contract (nine month approval period). 22. Additional Financing. The Additional Financing Credit of US$37.5 million was approved by the Board on April 26, At the time, the Additional Financing did not involve any formal restructuring of the original project or reallocations of the original loan. It primarily provided on-going interim funding for road maintenance in the original six provinces, while GoPNG programs and AusAID Transport Sector Support Program activities came on-line, to provide gap financing to complete work on two national roads, and to provide scaling-up of the original project activities to two new provinces (Gulf and Western). There was no change to the PDO or Project Components. Changes made to the Performance Indicators and associated targets have been described in Section First Extension of Credit Closing Date. The closing date of the Credit (only) was extended by two years to December 31, A new contract was negotiated with the EPM following the approval of the Additional Financing and scaling up of activities. Government approval procedures for this contract, which had to go through the NEC, took longer than the expected nine months. DOW financed an interim bridging contract to support scaled-back interim activities. This delay, which was exacerbated by lengthy visa approval times for consultants, delayed the work program under the Additional Financing. 24. Second Extension of Credit Closing Date. The closing date of the Credit was extended by the Regional Vice President for a further six months to June 30, 2012 in order for the government to fully disburse all available credit funds. At the same time, a reallocation of funds was approved for the unallocated funds to be used for rehabilitation of national roads and 6

21 bridges. Other minor changes reflected evolving priorities and respective financing needs among various project components. 25. Co-Financing. There was no formal co-financing to the project. However, in order to support its own counterpart funding requirements (varying across different categories from 30 percent to 10 percent of expenditures), GoPNG secured support from various other parties to meet its obligations, in the following amounts: (i) 2003 and 2004: the Australian Agency for International Development (AusAID) (US$4.05 million); (ii) 2007: New Britain Palm Oil funded a portion of an access road near its plantation (US$1.02 million); (iii) : the participating provinces (US$5.2 million); (iv) : the Papua New Guinea Sustainable Development Program s Development Fund for development projects (US$22.61 million). 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 26. Soundness of the Background Analysis. The background analysis to support the need for RMRP was sound even though the project was designed and first became effective during a period of significant macro-economic instability and so not without risk. Reviews of previous land transport sector projects in PNG were carried out to better understand potential problems and issues arising during project implementation, and successful approaches that could be adopted to ensure that outcomes and objectives could be achieved were identified. Fundamental to the design of RMRP was the fact that its development objectives were not limited to physical infrastructure investments; equal emphasis was placed on developing long term planning and budgeting capabilities for road maintenance within the government s own systems and on fostering a competitive, local contracting market that might service the sector in the years to come as well as generating much needed local employment opportunities. 27. Experiences from earlier World Bank-funded highway and transport projects in PNG showed that while implementation of physical components had generally been satisfactory, sustainability had been less assured and economic rates of return had been lower than at first estimated due to ineffective contract management by implementing agencies and cost overruns during implementation, lower than expected traffic growth, and delays in starting and completing works. Based on these analyses, robust project preparation and realistic risk assessments would be required to underwrite the Project design. Other risks requiring attention were also identified, such as poor contractor performance and deficient supervision by consultants, persistent problems in land acquisition and security, and disputes over rights-of-way (common for any projects involving new construction or major widening of existing roads). 28. Positive lessons of experience drawn from the Bank-funded Gazelle Restoration Program and the AusAID-funded National Roads Regravelling and Resealing Project and the Bridge Replacement and Upgrading Project were the establishment of a professional and competent implementing authority; establishment of a specialized supply and tenders board that could approve contracts of up to PGK 1.5 million; minimizing works outside of existing rights of way to avoid the need for land acquisition; and recognizing the lack of technical capacity in provincial offices of DOW to carry out surveys, testing and environmental management. 29. Assessment of Project Design. The design of RMRP is rated satisfactory. It was not overly ambitious or too complex given the volume of need. Project design identified a number of high risks to implementation and proposed mitigating measures. One aspect that was under- 7

22 estimated was the ability and willingness of GoPNG to provide counterpart funds in a timely way. The government elected to provide up to 30 percent of expenditures from its own resources and so reduce the size of the required IBRD loan. However, in retrospect, this was too high a requirement and the different percentages against each of the different categories made project financial management onerous. 30. Project objectives and performance indicators (Section 1.2) were straightforward and were further clarified and expanded during the additional finance to better monitor all aspects of the project investments. Six project components were proposed, which were straightforward and provided support not only to project works, but also for implementation support by strengthening the capacity of both the implementing agency and the private sector through dedicated training activities and the EPM. 31. To minimize the potential for social and environmental safeguards concerns, it was intended that no project works would require land acquisition and the works were designed to occur within existing rights-of-way although safeguards documentation on Involuntary Resettlement was prepared and disclosed in the event that it was needed. 32. Adequacy of the Government s Commitment. During project preparation, Government commitment was strong, as evidenced through discussions during preparation missions. However, there was a delay of some eight months between Board approval and effectiveness, while the necessary government approvals were obtained. 33. Assessment of Risks. A number of significant risks were identified during project preparation, resulting in an overall project risk rating of High. 34. Risks from Outputs to Objective: Inadequate and/or late receipt of counterpart funds (rated as High). To be mitigated through the establishment of accounts for the national Government and six provincial governments and making initial deposits; The potential for political interference in the drawing up of Annual Work Plans (AWPs) (rated as Substantial). The use of RAMS and the Bridge Inventory and Bridge Management Systems (BIBMS) was to be used to reduce this risk; Lack of local support for proposed works (rated as Moderate). This would be mitigated by encouraging involvement of village groups in routine maintenance, training activities for local contractors and employment creation for local villagers on the maintenance contracts; and Knowledge drain from DoW and provincial administrators seeking better positions in the mineral resources sector (rated as High). This was offset through training programs and implementation of contract management systems. 35. Risks from Components to Outputs: Inability of GoPNG to acquire any additional land required (rated as High). Only minor land purchases were considered likely. Voluntary land donations and consent to use communal properties when land must be required were cited as mitigation measures; 8

23 Selection of inappropriate contractors and/or not awarding contracts in a timely manner (rated as Substantial). This was to be mitigated by the EPM, Bank s prior review on contracts and use of Central Supply and Tenders Board (CSTB) procedures; Inability to recruit competent works supervisors (rated as Moderate). To mitigate this, the EPM had authority to reject works and supervisors assigned by PWMs and provincial administrations and recruit replacements; Inability to ensure that participating communities doing routine maintenance would comply with contract conditions (rated as Moderate). The employment of works supervisors with suitable and appropriate communication skills would be used to mitigate this; and Possibility of inappropriate procurement and misuse of funds (rated as High). To mitigate this, the EPM was to be involved in the procurement process with the relevant authority to endorse vouchers for payments under the project. 36. The project identified and put in place measures to mitigate the risks. This included the establishment of a strong EPM team with appropriate experience and competent staff. The EPM provided strong managerial and technical input to works, assisted DoW and provincial DoW staff, provided valuable inputs into the procurement process and prepared regular reports on implementation, including the AWPs. EPM staff was employed in the provinces, which acted to decentralize knowledge and experience and provided support to provincial authorities. Additionally, RAMS and BIBMS were utilized to ensure that technically and economically sound investments were made during the project. 2.2 Implementation 37. Factors that contributed to successful implementation and problems encountered. The relative success of the project can be attributed to strong ownership exhibited by DOW and to professional and decentralized management by the EPM. As a result of the good working relationship that was established between the DOW and EPM at the central and provincial levels and the collegial approach of the Bank s project teams, PNGSDP interest in the project grew. Understanding the process of identifying and prioritizing sub-projects for funding based on technical, social and environmental assessments convinced PNGSDP that channeling funds through the project would be money well spent. It also benefitted GoPNG, since the funds PNGSDP provided were used to meet their counterpart funding responsibilities. 38. Through careful packaging of proposed works and strong contract management by the DOW and EPM, the Project was able to let a large number and range of contracts for routine, periodic and restoration maintenance of national and provincial roads. In total, some 39 major contracts, including bridges, valued between US$2.5 million and US$5.0 million (two of which were on the order of US$10 million), 120 national competitively bid contracts of up to US$150,000, and 559 small contracts around US$2,500 each were let through the life of the project. Following a training needs analysis, from December 2005 an 18-month training component was prepared and implemented by specialist consultants to provide practical road and bridge maintenance training to 79 small and medium domestic contractors (identified by the Department of Trade and Industry) and 45 central and Provincial DOW staff in six provinces. The training covered subjects such as construction project management, cost control and financial management, materials management, estimating quantities, understanding contract documents, claims management and environmental training. A set of more advanced training was also provided on dispute resolution, road maintenance line and level checks, drainage, and machine 9

24 maintenance. In addition to the above, DOW staff were provided specific training in administration of contracts procedures and site records, bid evaluations, and survey and piling. Training provided was largely classroom based although decentralized to all six provinces. However, the planned use of on-the-job- training through demonstration contracts did not eventuate. Ideally, such training would have been provided at regular intervals through the extended life of the project to facilitate market entry by new small contractors and for refresher and hands-on training for contractors once they had gained some additional experience. To be sustainable, the training could have been accompanied by active publication of forward work programs and funding envelopes to demonstrate the longer term business opportunities that would be available to local firms. 39. In 2008, after the firm contracted to rehabilitate the East-West Highway on Manus Island encountered problems due to logistics in the remote location and prolonged wet weather, and was released under the frustration provisions of the contract, a force account contract was awarded to the DoW to complete the Works. The use of force account was considered because of the challenging conditions, including the limited availability of suitable construction materials, adverse weather conditions, and the works were located in remote areas, which required equipment and supplies to be shipped in from other provinces. Despite these difficulties, the works were completed within the estimated costs and in a timely manner. 40. The use of force account to complete this critical section of civil works in the absence of effective private contractors was considered a success. Several factors contributed to this success including: (i) the keenness of the departmental and provincial authorities to take on the job; (ii) the presence of a skilled technical motivator and mentor in the form of the provincial EPM; (iii) DoW staff with past Force Account experience; (iv) staff assigned to the rehabilitation works with a sound technical understanding of the challenges at hand and the work required; (v) knowledgeable and experienced supervisory staff; and (vi) a solid organizational structure in place to appraise and approve initial cost estimates and subsequently to track expenditures, make timely payments, and submit reports. 41. Annual audit reports in the early years of RMRP ( ) were not submitted on time to the Bank for its review. In part this was due to lack of staff in the Auditor General s Office. From 2007 onwards, the project audits were carried out by independent auditors and only required certification by the Auditor General. Thereafter, from mid-2009 onwards, as the initial arrangements for external audits did not take into account the statutory mandate of the Auditor General to conduct the audit, the procurement of external auditing services was implemented through the Auditor General s office, resulting in more timely approvals. 42. Effects of Project Restructurings. The additional financing was primarily sought to allow for a scaling-up of activities. This saw an extension of benefits to two additional provinces. The restructuring also brought about the revision of existing, or inclusion of new, performance indicators to reflect the addition of the two provinces. In addition, the restructurings allowed small scale re-allocation of funds as the need arose. No changes were made to the PDO during the restructuring. At the time of restructuring, the project was rated as Satisfactory, both in terms of the overall implementation progress, and for progress toward achievement of the PDO. Following restructuring, project implementation progress and progress toward achieving development objectives continued to be rated as satisfactory through the end of the project. 43. Mid-Term Review. A specific mid-term review (MTR) was not carried out for RMRP. Initially, an MTR was proposed for the end of 2003, however, due to significant delays in implementation, the MTR was pushed back to a later date. With the timing of GoPNG s request 10

25 for Additional Financing, a detailed review of project performance was undertaken as part of the appraisal for the Additional Financing in lieu of an MTR. 44. Response to Problems/Issues. No major problems were encountered during the project although issues arose which saw the project ratings being downgraded on several occasions. 45. The downgrading in implementation progress ratings was the result of the lag in provision of counterpart funding (ISR #4), particularly provincial support (and the associated unsatisfactory rating for Component 2 - Provincial Roads Maintenance and Rehabilitation). 46. To address the moderately unsatisfactory status of overall implementation progress and progress towards achieving the PDO, PNGSDP provided funding to improve the availability of counterpart funding (in addition to the financial support provided by AusAID), long overdue audits were finally provided to the Bank and found to be satisfactory. In turn, this allowed for an increase in the Special Account from US$1.5 million to US$5.5 million to help address payment delay problems, and strong supervision support ensured that works being undertaken were carried out to a high standard and in a timely manner. 47. Quality Assurance Group Ratings. This was not applicable and at no time was RMRP considered an At Risk Project. It was considered a Potential Problem Project from project effectiveness until January 2006 largely due to the government shortfalls in providing counterpart funding. By requiring funds to be deposited at the start of each financial year and with the PNG Sustainable Development Fund taking out some of GoPNG s counterpart funding commitments, both steps facilitated by the Bank s project team, counterpart funding was made available and, towards the end of the project, government counterpart contributions were increased so as to extend some contracts (e.g., Manus). 2.3 Monitoring and Evaluation Design, Implementation and Utilization 48. Monitoring and Evaluation. DoW, with assistance of EPM, prepared Quarterly Project Management Reports describing physical progress in implementation and achievement of the development objectives. In addition, DoW annually updated the project monitoring indicators, describing physical, procurement and financial expenditure progress, updated estimates of project costs versus available funding, and explained variances between actual and previously forecast implementation targets. Monitoring and evaluation activities were summarized through the Performance Indicators, which were either linked to the development objective (Project Development Objective Indicators), or reflected physical outputs achieved through the project works. 49. The EPM maintained data on the supervision of all road and bridge maintenance and/or rehabilitation works and worked closely with DoW in reviewing budget allocations for road maintenance and rehabilitation works, assigning contracts and signing off on works. 50. Baseline data was prepared and reported by DoW and used in project design and to assess progress during project implementation. A DoW Environmental officer was responsible for monitoring and reported on compliance with environmental management plans. 51. Socio-Economic Impact Studies were launched in 2005 and completed in 2011, and covered all eight provinces where RMRP applied. The studies quantified the socio-economic 11

26 impacts of the project with results disaggregated by gender. The GoPNG presented the results of the socio-economic studies at a number of international fora. 52. Issues with Data Collection. During implementation, data collection was not always accurate and was at times problematic given that maintenance was being carried out over the same length of roads each year. In limited instances, where routine maintenance occurred more often than annually, works were counted more than once, adding up to an inflated estimate of the length of roads being routinely maintained. In addition, during the period that AusAID counterpart funding was being provided, works outside of RMRP, funded by AusAID in the RMRP project provinces, were mistakenly included in the monitoring data but this was brought to the attention of the government by a supervision mission and appropriate corrections were made. 2.4 Safeguard and Fiduciary Compliance 53. OP4.01 on Environmental Assessment, and OP4.12 on Involuntary Resettlement were triggered by RMRP. A Sectoral Environmental Assessment (SEA) and an Environmental Management and Monitoring Plan (EMMP) were prepared and approved in However, due to the delayed effectiveness, the SEA and EMMP subsequently had to be revised in This revision also covered a social assessment, a framework for land acquisition and resettlement, and community participation. 54. Social Safeguards. Under RMRP, OP4.12 on Involuntary Resettlement was triggered as even existing road corridors are on customary land in PNG. Only minimal land acquisition was anticipated during preparation, and the ESIAMMP was the document to be used in the event that land acquisition was needed during implementation. A consultative process was put in place by appointing provincial government land officers to be in-charge of consultation and communication with all stakeholders to ensure there was a common understanding and local ownership of project goals, to the advantage of all parties. There were limited instances of community resistance which resulted in delays to the contractors. These were resolved via the consultative process, with facilitation by DoW staff. 55. Environmental Safeguards. OP4.01 on Environmental Assessment was triggered, and the project was classified as Environmental Category B. Because the works associated with road rehabilitation and/or maintenance were located within existing rights-of-way, the environmental impact of the project was minor and limited mainly to effects associated with rehabilitation works, such as the production of dust and noise, use of bitumen, disposal of limited amounts of solid or hazardous waste, erosion control, quarrying, silt control and labor camp construction and management. No changes were deemed necessary under the Additional Financing, and the existing environmental safeguards were considered adequate. Further, no sub-projects were assessed by the environmental authorities to require a full Environmental Impact Assessment. 56. As part of the project, an Environmental Officer was appointed to DoW, specifically to assist with monitoring environmental safeguards issues arising on sub-projects, and to train and support provincial DoW staff and contractors on environmental issues. The work performed by the Environmental Officer generally was reported as satisfactory with closer supervision carried out when the work program appeared to be slipping. Training of Community Development Officers, who were charged with improving relations with local communities along the roads to be restored and also to assist with carrying out Socio-Economic Impact Assessment studies, also took place. 12

27 57. Financial Management. The recipient complied with the financial management (FM) conditions outlined in all financing agreements and satisfied the Bank's requirements under OP/BP Following the initial FM assessment of substantial risk due to various deficiencies identified, the project was structured to include mitigating strategies to address these risks, including a full time international project financial controller to implement and oversee additional internal controls and reporting. These mitigating strategies were successfully implemented and the financial management was initially rated "satisfactory". From June 2005 through June 2009, the rating was reduced to "moderately satisfactory." The lower rating was due in part to delays in submission of audit reports and to the build-up of an eight months backlog of data entry by the EPM. While the latter did not impact on payments to contractors, it did mean that withdrawal applications could not be processed and FMR reporting to the Bank was not completed on time. Modifications were made to the financial reporting software and the backlog was cleared, resulting in an improvement of rating to satisfactory from July 2009 to the close of the project. 58. All audits of the project were received and accepted by the Bank. A number of audits were not received by their due dates as the initial arrangements for external audits did not take into account the statutory mandate of the Auditor General to conduct the audit. From mid-2009, the procurement of external auditing services was implemented through the Auditor General s office, resulting in more timely approvals. All but one audit was unqualified; the one qualified audit was qualified because of a specific issue related to the implementing agencies internal accounting and reporting of counterpart funding and expenditure, which did not relate to IBRD/IDA funds/expenditures. The final audit of the project s financial statements is not due until 30 June All interim financial reports were received, the majority on time, and all financial reports were accepted by the Bank. Withdrawal Applications were submitted regularly and there were no cash flow issues relating to IBRD/IDA funds other than the backlog in late 2008/2009 (above). However, this project had a large counterpart funding requirement and there were significant problems in the receipt of counterpart funds, both in terms of time and amount in the early years of the project, which caused not only cash flow problems, but delays to the project implementation. These problems were progressively addressed and in the second half of the project no longer existed. Advances to the Designated Account were fully accounted for or refunded to the Bank. No ineligible expenditures were identified. 60. The project progressively increased the use of country systems in a number of FM areas, with the project's planning and budgeting, accounting, and external auditing all using country systems. Full use of country systems for internal controls, internal audits, treasury, and reporting was not adopted due to a combination of deficiencies identified in the FM assessment and borrower and Bank preferences, but in some of these areas, the base country systems were used and simply added to in order to provide the controls required. 61. Procurement. In a challenging set of circumstances the procurement of works, goods and services was generally carried out satisfactorily by Department of Works, which was assisted by the EPM. With no foreign interest in the mostly small-scale works involved in relatively remote areas and an overstretched and relatively small domestic construction industry in PNG, contracts typically drew a limited number of bids. However, the project introduced a number of innovative approaches in order to facilitate the involvement of small- and medium-sized local contractors, while maintaining fiduciary oversight. These included processing of progress claims within two weeks (although 28 days from certification was allowed in the contract) to assist the cash flow of the small businesses. Also, dedicated pre-bid small contractor training in both technical aspects, as well as contract documentation was provided. 13

28 62. DOW had to contend with significant delays during implementation because of lengthy GoPNG approval processes for major contracts (above PNGK10m), requiring approval by the National Executive Council that typically can take between six to nine months. A particular issue arose in relation to the EPM assignment. In order to ensure a smooth transition between the end of the original Loan and the commencement of the Additional Finance Credit operations, DOW had planned a bridging contract with the EPM for a six-month period. However, approval processes through to the NEC finally took 12 months to complete. Several parallel and subsequent tasks also had to be adjusted because of the time needed to process the EPM contract. This required the DOW itself to fully-fund the EPM interim contract and related costs to ensure continuity of activities, which had to be scaled down until the NEC approval was forthcoming. Two other factors in the administration of the EPM contract were significant challenges for both DoW and the EPM, these being (i) the need to downsize the EPM staffing during these periods of delay in order to minimise costs; and (ii) higher turnover of EPM key personnel (PEPMs), due to the extension and delays. The EPM cooperated with the DoW in address these challenges. 63. Deviations or Waivers. Throughout implementation, no deviations or waivers under either of the two safeguard policies were triggered. 2.5 Post-completion Operation/Next Phase 64. RMRP II. At the request of GoPNG, the Road Maintenance and Rehabilitation Project II (RMRP II), was prepared, and subsequently approved, by the World Bank Board on April 7, RMRP II built on the achievements of RMRP, and lessons learned from RMRP (reported on in Section 6 of this report) were taken into consideration in the design of RMRP II. GoPNG requested that RMRP II continue working in the provinces that were included under RMRP, with the addition of two other provinces (Madang and Milne Bay). 65. Transition Arrangements. Prior to 2003, DoW was responsible for the maintenance, restoration and rehabilitation of all roads and bridges in PNG. With the creation of the National Roads Authority (NRA) in 2003 and full transfer of responsibilities by 2007, roads that have been rehabilitated, or that are already in satisfactory and maintainable condition, have been transferred to the NRA. NRA is responsible for raising funding for road maintenance, preparing maintenance plans, and overseeing road maintenance to ensure that road assets are preserved to a satisfactory standard. 66. DoW maintains responsibility for bridges throughout PNG, and is also responsible for operating the RAMS and BIBMS and ensuring that the inventory and asset register is fully maintained. Limitations on funding have often impacted its ability to gather data, store the data and make full use of these asset management systems. 67. Performance Indicators. Under RMRP, most of the performance indicators were output based, such as cumulative kilometers of roads maintained and numbers of bridges maintained, and so were pertinent only to RMRP. For RMRP II, new performance indicators were developed, but some of the performance indicators under RMRP will continue to be monitored. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 14

29 68. Objectives. The Papua New Guinea Medium-Term Development Plan states that 25,000km of national roads are to be in good condition by The document cites the need to provide ongoing funding to the sector to encourage private sector participation and growth, in order to achieve this goal. 69. The World Bank s Country Assistance Strategy (FY2008 to FY2011) defines seven propoor expenditure priorities: (i) rehabilitation and maintenance of transport infrastructure; (ii) promotion of income earning opportunities; (iii) basic education; (iv) development-oriented informal adult education; (v) primary health care; (vi) HIV/AIDS prevention; and (vii) law and justice. 70. Based on the objectives of the Medium-Term Development strategy and the Country Assistance Strategy, the objectives of the RMRP remain highly relevant overall. 71. Design. Individually, the project components and activities included in project design were relevant to the needs of GoPNG and people of PNG. The project design included rehabilitation and maintenance of roads and bridges to improve their condition to good, and to provide continuous linkages of the road network. The design also included allowance for building capacity within the sector, and for providing assistance to growing the private sector, which remains a key part of GoPNGs goals and objectives. 72. Implementation. During implementation, there were no significant changes that affected the relevance of objectives or project design. 3.2 Achievement of Project Development Objectives 73. The following is an evaluation of project outputs and how they relate to project objectives. More details on component outputs are presented in Annex Rating: Satisfactory. 75. Overall, progress in achieving project development objectives is considered Satisfactory and reliably so in the last five years of implementation. 76. Key Objective 1: restoring satisfactory physical condition and safety to parts of the network of national and provincial roads and bridges in six of the country s nineteen provinces based on priority needs, was achieved. This assessment is based on RMRP final outputs, which included nearly 450 km of national roads maintained and 600 km of national roads restored annually, and 46 national and provincial bridges rehabilitated, maintained and/or replaced. Annex 2 provides detail pertaining to the outputs achieved under this objective. 77. Key Objective 2: strengthen strategic planning and management of road sector assets. The RAMS was adopted as a means to more systematically identify and prioritize roads requiring maintenance or rehabilitation. This was widely used throughout implementation and continues to be used. There are, however, some concerns around its ongoing use due to funding limitations. The percentage of budgeted work program based on RAMS fell short of the project target. Similarly, the percentage of expenditure on maintenance and preservation did not meet the targeted amount. Nonetheless, because the RAMS system was utilized under the RMRP and is effectively producing data, this objective is considered to have been substantially achieved, despite outputs not meeting the project targets. 15

30 78. Key Objective 3: strengthen institutional arrangements for road maintenance, including through an expanding provincial government role in line with the Government s decentralization policy and the New Organic Law. Outputs pertaining to this objective were the number of provinces submitting AWPs, and the number of provinces that submit management claims in a timely manner. The outputs for these indicators had met the target values at the time of the Additional Financing. The project also contributed to the establishment of the NRA, which assumed responsibility in 2007 for road maintenance. The objective of strengthening institutional arrangements for road maintenance was achieved. 79. Key Objective 4: increase private sector contracting of road maintenance works, including through the strengthening of small and medium-sized contractors and participation by village communities. Based on figures provided by GoPNG, under RMRP, 457 and 100 contracts were let to local contractors for roadside and pavement maintenance on national roads respectively, while another 39 contracts were awarded for national road and bridge rehabilitation. For provincial roads, 102 and 20 contracts were awarded for roadside and pavement maintenance respectively. These outputs went a considerable way to reaching the objective of increasing private sector contracting in road maintenance. It also supported village communities in road maintenance works as many of the maintenance activities were undertaken by local villagers and community groups. Greater impact may have been attained had the preparation and approval of simplified bidding documents for small- and medium-sized contractors and training on their use been offered at the start of the project, rather than after a number of contracts had been awarded. This may have encouraged more small-contractors to compete and avoided crowding out of smaller firms by larger more experienced companies. 80. Another limiting factor was that during the course of the RMRP, the country experienced growth in the mineral resources sector, which drew contractors and private firms to the lucrative opportunities in the mining, oil and gas sectors. This created challenges in terms of attracting qualified contractors to carry out road works, and exacerbated efforts to build local capacity within the roads departments. The DoW and other relevant Governmental institutions bemoaned the loss of key staff to the private sector, particularly to companies working in the resources sector, which diminished their capacity. 3.3 Efficiency Rating: Satisfactory. 81. The objective of the project s maintenance and improvement program was to minimize the total life-cycle costs for roads in terms of both agency and user costs, given the existing traffic volumes and road conditions. The project road works program was developed by DOW using the RAMS and was subjected to economic analysis by consultants using the Highway Development and Management Model - 4. Periodic and specific maintenance requirements were determined from the RAMS in a coarse screening process for sub-projects based on net present values (NPV) divided by costs. Upgrading of gravel roads to sealed roads and pavement reconstruction were determined from a fine screening process. 82. The ex-post evaluation calculated the economic rates of return (ERR) of the representative roads, adopting the actual average costs of each road work type. The following table presents the results of the ex-post economic evaluation. 16

31 Ex-Post Economic Evaluation Road Work ERR (%) Reconstruction 60 Regravelling 22 Sealing 27 Upgrading 21 Considering the actual average road works unit costs per km, the ex-post ERR is higher than 12 percent for all road work types. The ex-post ERR ranges from 21 percent for upgrading works to 60 percent for reconstruction works. Thus, the program remains economically justified considering the actual road works costs. 83. The RMRP is more than ten years old. Part of the delayed implementation was due to the lack of counterpart funding, which was a big reason for the need to extend the closing date of the original Loan by twelve months. Implementation was further hampered by the torturous national approval process of the EPM contract, which took nine months before it was approved, as well as a general lack of familiarity with Bank procedures. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory. 84. The rating of Satisfactory is based on the relative achievement of objectives, efficiency, and relevance. The project objectives remain highly relevant to current day objectives and goals of GoPNG and are consistent with the CAS available at the time of the ICR. Overall, the RMRP s development objectives were achieved. Significant lengths of national and provincial roads and bridges were restored, while the use of a systematic approach to prioritizing road maintenance activities was adopted. There were some shortcomings in building private sector capacity as a result of many contractors moving to the mining, oil and gas sectors where more lucrative work exists. However, this was something beyond the control of the RMRP. This also had an effect on institutional strengthening, with the decrease in capacity of the DoW and other relevant Governmental institutions being put down to staff moving into the private sector, particularly to companies working in the resources sector. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 85. The Socio-Economic Impact Assessment study played a clear role in assessing the effects and impacts on poverty and social development. An executive summary of the findings from the study is presented in Annex 5. One of the objectives forming the PDO was to increase private sector involvement in the participating provinces, particularly that of small- and medium-sized contractors. With hundreds of minor maintenance contracts being awarded to local community groups, income was generated in areas with high rates of unemployment. The RMRP helped to strengthen institutional capacity in attempting to ensure the sustainability of road maintenance and ongoing contracts for such works in areas where small- to medium-contractor involvement is needed. 17

32 86. The impact on beneficiaries of one road project funded under RMRP was captured in a short documentary film, The Road to Kerema, commissioned and completed in 2011, which looked at the issues facing the population of Kerema, the capital of Gulf Province, which until the recently-completed works under RMRP, had no continuous road access to Port Moresby. 87. Generally, women are pleased with the road improvements, particularly on the Kerema Road, as it has made access to Port Moresby easier for them. Women now travel more often to Port Moresby to shop, market their goods, and bank. Due to better road access, prices at Kerema market for goods like sago have increased, which benefits village women. It was also reported being easier to travel to special events like the church basket exchange. Women also reported finding travel safer as a result of road improvements. Earlier the road had been in such a bad condition during the wet season that they had to walk some parts of the way or travel by boat, which was expensive. As a result, the road improvements have also decreased the price for travelling to Port Moresby. Some women with children studying in Port Moresby say it is easier to visit them. 88. One drawback has been increased speeds and reckless driving along improved roads, often under the influence of alcohol. Many indicated that it was safer to walk close to the village, but not on the main road as the driving was unpredictable. (b) Institutional Change/Strengthening 89. RMRP had a positive impact on institutional development, primarily through the above training inputs and also through the day to day assistance the EPM provided to DoW, and the use of Force Account in Manus Province. The latter allowed DoW staff to experience firsthand the difficult conditions facing road contractors. It also required DoW to provide detailed costings to carry out road rehabilitation works, and to gain a greater understanding of the challenges associated with this exercise. 90. Non-quantifiable benefits included strengthening the capabilities of the Department of Works (DoW), and the six (and later eight) provincial administrations, which were responsible for the planning, design, financing, supervision, construction, operation and maintenance of roads and bridges. Furthermore, the project assisted in strengthening small- and medium-sized contractors, and with the exception of the works that were undertaken by the Government to complete the limited road rehabilitation activities on remote Manus Island, reduced the amount of works done through force account. The project also fostered improved practices in environmental protection, land acquisition, and participatory approaches to project preparation and implementation by the provincial authorities. 91. The appointment of an EPM was critical to the relative success of RMRP, and as identified during project design, DoW on its own would have lacked the capacity and resources to fully implement the project. The EPM supplemented and facilitated the transfer of skills and knowledge to DoW staff in areas such as technical knowledge, contract management and financial reporting. (c) Other Unintended Outcomes and Impacts (positive or negative) 92. An unintended outcome was the use of force account to complete the East-West Highway on Manus Island after the firm contracted to rehabilitate it encountered problems. The use of 18

33 force account in this instance was pursued because of the challenging conditions, which included the remote location and prolonged wet weather. The contractor was released under the frustration provisions of the contract, a force account contract was awarded to the DoW to complete the Works. While not appropriate for most road works, force account does have a role to play under certain circumstances, such as for remote locations, when there are adverse (weather) conditions, the need to expedite works, or when small contracts are unlikely to attract a number of private contracting firms. Another critical factor is the need for an implementing agency with in-house and/or access to suitable project management skills, staffing resources, equipment and plant to carry out the work and ensure expenditure and reporting discipline. Based, in part, on the success of this contract, at least one other donor in PNG is planning to fund periodic road maintenance using force account. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 93. A longitudinal socio-economic survey was carried out to evaluate the impacts of the road rehabilitation on the households' income and expenditure, on access to health and education services and the reduction of poverty in two sampled villages located along main roads under the maintenance program and two control villages at some distance from the roads in each of the six participating provinces of Manus, East and West New Britain, Oro, Central, and Morobe. The baseline survey and report was carried out in , a mid-term survey and report was carried out in 2006, and a final survey and report was undertaken in Each year, the same households in the same villages were surveyed, allowing for longitudinal analysis over time. Under the Additional Financing, a baseline socio-economic impact survey was carried out in 2008 in the two new provinces, Gulf and Western, and included more in-depth, qualitative analysis of the impacts, including poverty, vulnerable groups, and specific disaggregation of the data by gender to differentiate between the needs of men and women in Central and East New Britain provinces. The results of the surveys were presented to a National Workshop in 2009 and distributed to each of the provinces in The purpose of the socioeconomic impact study was to evaluate the impacts of the road improvements on the households' income and expenditures, on access to health and education services, and the reduction of poverty. The results of the surveys were unambiguous regarding the positive and widespread benefits for those villagers living alongside the improved roads. After the rehabilitation of the main roads, visits to towns and markets increased 1.7 times in the sample villages by the rehabilitated roads, and 1.4 times in the control villages. The increases were less for the non-rehabilitated roads, where the visits increased 1.2 times in the sample villages and 0.7 times in the control villages. Although transportation costs increased in all villages, the increases were greater in villages on non-rehabilitated roads. Related to the improved levels of access to markets, while incomes increased over the time period in all villages, they increased more in villages close to rehabilitated roads. In particular, roadside sales increased because of more traffic and passengers on the roads. This was of particular benefit to women who have been able to earn money near the home and at the same time can take care of children and other domestic tasks. Roadside sales also have helped unemployed villagers to earn cash income. Somewhat unexpectedly, the rehabilitation of roads was found to have had little impact on the travel time to health or education facilities as villagers typically used social services that were provided within the village, or made use of shortcuts through the bush 19

34 and on tracks, rather than walking along the main road. However, it was reported that improvements to the roads did benefit the secondary school students by permitting more convenient and faster public motor vehicle rides. Comparing the 2007 and 2009 data, households living below the poverty line along the rehabilitated roads decreased from 57 percent to 40 percent, based on a monthly income of PGK500 or less. 95. Details of the Socio-Economic Impact Assessment carried out under the project are presented in Annex Assessment of Risk to Development Outcome 96. The risk to development outcome at the time of the project closing is rated as Moderate, due to a number of factors discussed below. 97. Political and institutional risks are moderate. Important preconditions for sustainable road maintenance programs in the participating provinces are in place. For example, roads have improved over the last decade due to sustained financing and technical assistance from AusAID, the Asian Development Bank, Japanese International Cooperation Agency and the World Bank. GoPNG has recognized the importance of adequate transport infrastructure in its strategic documents 2 and is currently finalizing its National Transport Strategy plan, which calls for significant investments to rehabilitate and adequately maintain the roads transport infrastructure in the country. Following rehabilitation of roads carried out by DoW across the country, NRA is increasingly assuming the responsibility to maintain National Roads of PNG as prescribed under the NRA Act of Nevertheless, NRA is a relatively new institution that needs support with developing appropriate maintenance modalities (planning, specific contractual arrangements, etc). 98. Technical risks are moderate. The planning and setting of priorities for roads is currently constrained by lack of data, inadequate planning tools and limited engagement by provincial governments. Roles and responsibilities of DoW and NRA are well defined legally and there is common understanding about the scope of their activities. Nevertheless, because of chronic lack of adequate funding, both institutions have been facing difficulties in attracting and keeping quality staff. The capacity situation at regional and local offices is challenging and can cause delays in implementing works programs. A further risk is that of private sector contractors seeking higher paid contracts in the booming mining sector. Nonetheless, the predictable, medium-term road maintenance funding allowed for the emergence of numerous small- and medium-sized companies to enter the market and several communities to enjoy casual day labor employment. 99. RMRP II, which follows on the achievements made under RMRP and takes into account lessons learned, helps minimizes the risk that development outcomes under RMRP will not be maintained. 2 Papua New Guinea Vision 2050 (November 2009); Papua New Guinea Development Strategic Plan (March 2010); Medium-Term Development Plan

35 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory The project team drew lessons of experience from previous highway and transport projects in PNG and throughout the Pacific Region and provided a well-balanced approach to project design to include both tangible physical investments, as well as institutional strengthening, socio-economic impact studies that in time might build political support for the need for maintenance and a proactive training approach for small- and medium-scale local contractors to participate in the economic opportunities of the project Potential risks were realistically identified and measures put in place to mitigate them (see Section 2.1). The design of RMRP limited works and involvement to six provinces, in an effort to ensure that the staffing and resources of an institutionally weak implementing agency were not overstretched and overcommitted. Project design was also consistent with the country s development strategy and key objectives to be achieved under the project tied in with the country s objectives Care was taken in design to mitigate any potential safeguards issues. The overall principle was to only conduct works within existing rights-of-way, and to avoid the need for land acquisition. Documentation was prepared to mitigate both environmental and social impacts, and the consistent Satisfactory ratings for safeguards throughout implementation are testament to good consideration of safeguards aspects in project design The project design required the implementing agency to regularly submit progress reports, as well as Annual Works Programs, which were viewed as a good option for keeping implementation on track. The design of the project included allowance for the EPM to implement the project on the government s behalf. This was a critical decision in the design and made a significant contribution to the project s success Considerable delays in implementation were experienced during the early years of the project. The design called for counterpart contributions of 33 percent, which GoPNG wanted because this was an IBRD loan, not IDA Credit. However, changing economic conditions made it difficult for GoPNG to finance its contribution, and it wasn t until PNGSDP intervened with funding that the project took off. Although these types of changes cannot be anticipated, the Bank Team could have been more proactive when it became apparent that counterpart funding was not forthcoming. (b) Quality of Supervision Rating: Satisfactory Implementation support was provided throughout the project, not only in project management matters, but also specifically to financial management, procurement, and environmental and social safeguards. Regular supervision missions were carried out by the Task Team, which was consistently staffed with qualified staff and consultants. 21

36 106. During supervision when difficulties arose, the Bank provided adequate support when needed. For example, ISR #7 of June 2005 rated both progress towards achievement of the development objectives and overall implementation progress as Moderately Unsatisfactory. This was largely due to issues with counterpart funds, audit reports not being provided, and letters of credit not being signed. To improve implementation, the Bank provided close supervision support to the implementing agency, and also later on prepared Additional Financing to account for increased project costs, a less favorable exchange rate, and to further progress of works undertaken under the project. The close supervision resulted in the upgrading of ratings, as reported in ISR #9 (November 2006). From that point on, the ratings remained Satisfactory with the exception of one occasion when a Moderately Satisfactory rating was given for progress towards achievement of development objectives and twice for overall implementation progress The indicators generated in the ISRs remained as per the original Loan throughout implementation, which was a system error. The Bank Team should have made efforts to correct this, but the regular project reports did provide information on the revised indicators. While this is a shortcoming, it is not enough to override the Bank s satisfactory performance over the RMRP s ten-year implementation period. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory The Bank conducted supervision missions at least two times per year, with additional special visits made when needed. During these missions, Bank staff regularly met with various stakeholders, periodically inspected physical works in the field, and carried out rapid appraisal interviews with community groups. In addition, at key junctures, the task team reacted proactively to address implementation delays as they arose, for example by: (i) facilitating the Government s access to supplementary counterpart funds from AusAID and PNGSDP; (ii) processing a number of extensions to the closing date that were accompanied by time bound action plans for the Government to make full use of the available funds; (iii) encouraging DOW to employ a full-time Environmental Management Officer to verify compliance by the contractors with approved environmental management plans; and (iv) incorporating good practice international examples of HIV/AIDs information education and communication campaigns to strengthen DOW s initiatives in introducing these measures on all major contracts (even those below US$10 mil.). Over the course of the project, there were only three task team leaders, thus providing a degree of continuity to the Borrower. In late-2004, responsibility for the project passed from a headquarters-based task manager and team to a Sydney-based task manager and team, which had an appropriate skill mix for the task. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory Although performance improved over the life of the project, the lack of counterpart funding, particularly in the early years of implementation (2002 to 2005), contributed to significant delays. It wasn t until the PNGSDP provided funds to GoPNG that implementation took off. Inter-ministerial support to DOW during design was, at times, lacking. For example, the Ministry of Communications, which was responsible for identifying local contractors to 22

37 participate in the project, was lacking, so oversight for the entire training component was given to DOW. The timeliness of project audits by the Auditor General s Office was not well appreciated, and the length of time for NEC approvals of large contracts led to implementation delays and higher contract costs. Despite the lack of funding in the early years, the Government s commitment to the RMRP s development objectives, particularly in the last five years of the project when tangible improvements in the condition of some of the main roads in the participating provinces materialized, was strong The Government actively participated during project preparation and was proactive in identifying practical solutions to overcome challenges during implementation. (b) Implementing Agency or Agencies Performance Rating: Satisfactory DOW demonstrated its commitment to the project by being responsive to project needs throughout implementation and by providing strong, experienced and dedicated direction and management at Head Quarters via the Project Director for the full duration of the project period, and consistently providing qualified counterpart staff in all provinces Recognizing the need to have additional help to implement the project, DOW contracted a firm to serve as its project manager. Throughout the duration of the project, the EPM provided both technical and fiduciary support. Additionally, an Environmental Officer from within DoW was appointed following project effectiveness (2002) to provide environmental input on project works and training to provincial DoW staff. This assisted in the project meeting environmental safeguards obligations throughout implementation During preparation, the DOW played an active role in identifying roads and bridges to be improved under the project, and ensured effective project design and contract documentation during implementation. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory The rating for Overall Borrower performance is Moderately Satisfactory, based on the ratings for performance of the Government and Implementing Agency. 6. Lessons Learned 115. The RMRP was implemented in a challenging operating environment and was at times constrained by circumstances beyond the immediate control of those implementing the project. The country environment risk was flagged for the duration of the project and yet the project was able to maintain satisfactory or moderately satisfactory achievement of the development objectives in all but ten months of its 9.5 years duration, and satisfactory or moderately satisfactory implementation progress for all but around 14 months. The reasons for this were fourfold and inter-related Firstly, the project design was rigorous and realistic in its identification of potential risks. It accurately pre-identified a number of high risks that surfaced during the course of 23

38 implementation and as such, could be addressed by the project team in a timely manner. Among the highest risks identified were inadequate or late counterpart funding, potential political interference in drawing up annual work programs, inability of the government to acquire additional land, if required, selection of inappropriate contractors, and inability to motivate participating communities to comply with contract conditions. The open and frank early identification of the risks created an entry point for their solutions, as explained below The second reason for the project s success was that recognizing the small size of the civil service in PNG, and the relative inexperience with Bank guidelines and procedures, the project funded a long term, full time Employer s Project Manager to supplement (not supplant) DOW s staff numbers and capabilities. The EPM comprised not only the usual central project management unit responsible for project fiduciary functions, but also Provincial EPMs, who worked alongside the Department of Works own provincial officers in the participating provinces. The professional and collegial approach of the EPM staff at both central and provincial levels, supplemented by periodic formal work-shopping of common issues and concerns between provinces, resulted in the engagement of government counterpart staff in meaningful learning-by-doing project implementation. Whereas early risks to the project had been identified as the potential inability to recruit competent works supervisors and the possibility of inappropriate procurement and/or misuse of funds, throughout the life of the project, procurement performance was consistently rated as satisfactory, and in the last two years, DOW itself was enforcing strong contractual measures against under-performing contractors. This learning-by-doing in turn inculcated a strong sense of local ownership and, for example, whereas shortfalls in counterpart funds was the cause of delay for the first two years of the project, by the last two years, the government (through the PNG Sustainable Development Fund) was contributing more than the agreed contribution to make up for shortfalls resulting from the appreciation of the local currency against the US dollar, and to extend some contracts beyond the project work program Thirdly, recognizing the large number of potential risks, the Bank project team was proactive in closely supervising the project through regular missions and site visits with government counterparts to verify implementation progress against agreed work plans, the quality of remote works, and compliance with safeguards. This resulted for example, in the Bank s project team recognizing the needs of the client and being able to flexibly support a range of procurement methods to cover the requested large and inclusive range of maintenance contracts from international competitively bid major contracts on the country s national highways to the dozens of community-based maintenance contracts to the use of force account by DOW in remote and inaccessible provinces, such as Manus. It also resulted in the government obtaining a better understanding and honoring the principles behind safeguards and, for example, DOW staff assuming responsibility for monitoring contractors in the field to ensure they were following the provisions of approved environmental management plans and HIV/AIDS IECC outreach on all major contracts Fourthly, the socio-economic impacts of the project works were systematically monitored across beneficiary and control (non-beneficiary) communities in all provinces. The results of these surveys were presented to DOW and other technical staff during the course of the project, highlighting the educational, health and local economic development benefits of improved access, and also identified some of the negative impacts of the program. This not only allowed for adjustments to the work program to include not only road sections with high economic rates of return, but also to better target communities most in need. The public dissemination of some of these results (for example, a popular video that was released) served to build public support for the project and helped mitigate against the earlier identified risks of lack 24

39 of local support for the proposed works and inability to motivate communities to comply with contract conditions The project was one of the first in the EAP region to benefit from (at the time) the recently introduced additional finance instrument. The availability of significant additional finance and its rapid processing meant that the project could be scaled-up from six to eight participating provinces. This scale-up and the subsequent increase in the number of small contracts that were awarded, minimized the potential for conflict between communities that benefitted and those that did not Finally, RMRP was a large-scale infrastructure project, implemented by the DOW, but the local economic benefits and improved access to markets, health and education that resulted from the maintenance of roads and bridges have brought positive and quantifiable benefits to rural communities and groups within the communities, notably women, children and the poor The overriding lesson to emerge from RMRP is that even when both country and project risks may be high, where the risks are robustly identified and the project is implemented in a close, but collaborative way at a pace appropriate to uptake by local institutions and with a focus on sustainability and not asset building, the risks, by and large, can be managed for positive outcomes. As a post-icr activity, together with the Borrower, the Bank team intends to conduct an ex-post risk validation exercise, in which the risk framework outlined in paragraphs will be examined by asking three questions in hindsight: (i) did this risk materialize; (ii) if so, what mitigation measures were used (by any parties, and specifically did the Bank and Borrower use the mitigation measures identified ex ante); and (iii) what other risks eventuated that were not flagged in the project design, and how were those mitigated? The results of this ex-post risk validation exercise will be shared internally and with partners, along with the findings of this ICR and the specific results of the lessons learned exercise on the Manus force account pilot Recommendations. Extrapolating from RMRP s lessons learned generates the following operational principles for successful implementation in fragile institutional settings and low-capacity operating environments: (i) Avoid overly ambitious timetables or complex project designs, and sustain engagement with Borrower and other stakeholders during long delays (whether caused by effectiveness conditions or delayed key actions in implementation). In fragile operating environments, aspirational timetables that may be inspirational in other settings end up having divisive and discouraging impacts on implementing partners and can risk reputational damage to the Bank. Allowing for a more realistic timetable, while sustaining constant monitoring and encouragement to take small steps and make intermittent progress allows for a sense of joint positive engagement. While dealing with lengthy delays, use other activities (low-key workshops on implementation topics, periodic VCs with stakeholders) to sustain engagement and build rapport. (ii) Customize the approach and tools used to engage with the private sector so as to facilitate the robust involvement of small- and medium sized firms and level the playing field for domestic firms while not missing out on the technical expertise offered by international firms. In addition to aggressive outreach to the local 25

40 market and training in IDA/IBRD processes/documents/procedures, this may include explicit choices about bid sizes and sequencing of bids, consider fasttracking payments for small contractors against valid progress reports, and enhancement of the standard bidding document for small works. (iii) Anticipate delays in areas where weak or low-capacity national functions will impede the project (central tenders board, national audit authority, cash-flow management for release of counterpart funding), and combine pragmatism with proactivity. Budget for supplementary inputs (funds for external auditors in project budget, albeit supervised by national audit office), plan more time for the project to meet key benchmarks, embed early warning signals and assertively use Bank relationships with central economic agencies to unblock obstacles, and develop alternatives for critical path items like counterpart funding (e.g., third-party cofinancing in lieu of Government budget releases). (iv) Explore the unexpected benefits that might be created when alternatives are activated in managing risks, and actively build upon those choices. In RMRP, supporting GoPNG s search for a solution to delayed counterpart funds resulted in co-financing from several key development partners. Rather than leaving this as a purely financial arms-length relationship, the project team actively cultivated the quadrilateral relationship between the Bank, the Borrower, a key bilateral donor Australia, and PNG Sustainable Development Program. Through shared field visits and workshops these co-financiers were converted into champions for more effective and higher quality implementation support to major civil works projects, and the aggressive use of PNG media allowed the stakeholders to demonstrate these effective partnerships to a generally skeptical public, which in turn built more support for and trust in the DoW. (v) Continually review and update the risk framework during implementation with attention to which risk mitigation measures are most effective. Also, review the entire implementation experience with the Borrower after the ICR is complete in order to validate the risk framework and formalize lessons learned. Cultivate a shared approach that emphasizes that risk is not inherently damaging, but unmanaged risk is. (vi) Close the loop using post-icr workshops and other mechanisms to proactively feed operational learning into Borrower institutional processes, design of new Bank operations, and design of non-bank operations (through collaborative events with other sector donors), and create upstream and downstream project-specific lessons to other projects in the portfolio (just as the RMRP team used lessons from the earlier Bank-financed Gazelle Restoration Program in PNG and fed RMRP experience into the design of RMRP-2). (vii) The transport, access and personal security needs of men and women may be different. Disaggregated data collection was not a focus of the 2007 socioeconomic survey, but it had a more prominent role in the 2008 survey. Early disaggregation by gender of the results of socio-economic surveys of the benefits and negative impacts of project activities and outcomes should be carried out early on. It is important to do so because good baseline data and information is essential in formulating policies and measures that effectively addresses women s issues, and for ensuring that project impacts are socially and economically inclusive. Robust participation by a wide range of stakeholders in reviewing and learning from those data will build a broader shared understanding of both the differential 26

41 needs of men and women, but can also nurture a shared approach for placing a higher priority on inclusion across all partners projects. (viii) The business needs and capabilities of large and small contractors are markedly distinct in small countries with limited private sector contracting capacity. To encourage the participation of small- and medium-sized contractors, the early preparation and approval of simplified bidding documents and training on their use should be offered to smaller firms at the start of the project, rather than after a number of contracts have been awarded in an effort to avoid crowding out of smaller firms by larger more experienced companies. Specific actions during contract management (as discussed in paragraph 61) can also further support smalland medium-sized contractors. (ix) In countries with limited capacity, measures should be structured into a project s design to ensure that counterparts have adequate training on Bank processes and procedures, and have regular access to qualified staff and consultants, especially on fiduciary matters. Best results will be obtained by not relying only on one-off classroom training events (which generally focus on information delivery), but rather including semi-formal problem-based solution sessions as part of regular supervision/implementation support missions. These project-specific investments in client capacity can be complemented by having project staff join periodic CMUsponsored portfolio-wide capacity building workshops that are emerging as good practice because of their ability to build a community of practice across Bankfinanced implementing agency teams in a country. 27

42 Annex 1: Project Costs and Financing (a) Project Cost by Component (in USD million equivalent) Original Loan Components Appraisal Estimate (US$ mil.)* Actual/Latest Estimate (US$ mil.)** Percentage of Appraisal 1: Maintenance & Rehabilitation of National Roads : Maintenance & Rehabilitation of Provincial Roads : National Bridges : Provincial Bridges : Implementation Support : Road Sector Support Total Baseline Cost: Physical Contingencies 4.96 Price Contingencies 4.24 Total Project Cost: Front-end fee PPF Front-end fee IBRD 0.4 Total Financing Required: * These amounts do not include VAT and are both IBRD and Borrower amounts combined. ** These amounts include VAT and are both IBRD and Borrower amounts combined. 28

43 (b) Project Cost by Component (in US$ mil. equivalent) - Additional Financing Credit Components Appraisal Estimate (US$ mil.)* Actual/Latest Estimate (US$ mil.)** Percentage of Appraisal 1: Maintenance & Rehabilitation of National Roads : Maintenance & Rehabilitation of Provincial Roads : National Bridges : Provincial Bridges : Implementation Support : Road Sector Support Total Baseline Cost: Physical Contingencies 3.49 Price Contingencies 3.49 Total Project Cost: Front-end fee PPF Front-end fee IBRD 0.4 Total Financing Required: * These amounts do not include VAT and are both IBRD and Borrower amounts combined. ** These amounts include VAT and are both IBRD and Borrower amounts combined. 29

44 (c) Financing Original Loan: Source of Funds International Bank for Reconstruction and Development Type of Cofinancing Loan (IBRD ) Appraisal Estimate (USD mil.)* Actual/Latest Estimate (USD mil.)** Percentage of Appraisal Borrower: (GoPNG) Provincial Governments Counterpart New Britain Palm Oil Ltd funds PNGSDP AusAID Sub-Total for Original Loan Additional Financing: IDA Credit (Additional Financing) (IDA-42980) Borrower: (GoPNG) Counterpart Provincial Governments funds PNGSDP - - Sub-Total for Additional Financing Total Financing: * Borrower amounts include VAT/GST (10%) of total financing ** These amounts include VAT/GST and are based on exchange rate of USD 1.0 = K 2.96 Note: At project appraisal, the exchange rate was PGK1.0 = US$0.27 and US$1.0 = PGK At project closing, the exchange rate was PGK.1.0 = and US $1.0 = PGK

45 Component/Sub- Component Major Outputs, with Targets (dates) Annex 2: Outputs by Component Major Outputs at Completion (latest data) Project Component 1: Road Maintenance and Rehabilitation of National Roads Routine Maintenance of National Roads Restoration of National Roads 750 km (annual target) - revised under AF from 543 km 640 km (revised under AF from 440 km) 447 km (annual average from 2003 to 2007) Comments Roadside maintenance contracted three times annually. Pavement maintenance also carried out annually, involved some 100 contracts from 2003 to The reasons cited for target shortfall included the following, as reported by the Borrower: Funds provided by GoPNG and AusAID through their own programs for routine maintenance in some of the RMRP provinces. Some road sections to be maintained had deteriorated so badly they required rehabilitation and not maintenance as planned. Limited number of contractors available to perform maintenance works. From 2007, NRA assumed responsibility for maintaining roads in Central Province, Moroboe, and West New Britain effectively limiting the roads for maintenance under RMRP. 600 km Reasons provided by the Borrower for the shortfall include: The cost of construction increased significantly from 2007 on. Contractor capacity weakened due to lucrative work in the mining and oil/gas sector, plus larger contracts in other provinces financed by other Donors. DoW opted to reseal selected sections of roads which were originally anticipated to be finished with an unsealed surface. This sub-component was fundamental to achieving the project development objective through contributing to the promotion of an efficient, safe and reliable roads transport system. 31

46 Project Component 2: Maintenance and Rehabilitation of Provincial Roads Routine Maintenance of Provincial Roads Restoration of Provincial Roads Project Component 3: National Bridges Bridge Maintenance and Rehabilitation Bridge Replacement Emergency Bridges and Parts 50 km annually 166 km (annual average from 2003 to 2008) There was significant appetite among provinces for better maintenance of provincial roads supported by community involvement. This helped to increase the level of routine maintenance about three times over what was anticipated. 220 km 174 km Reason for shortfall involves lack of provincial counterpart funding. This subcomponent was fundamental to achieving the project development objective through contributing to the promotion of an efficient, safe and reliable roads transport system and for building private sector participation Reasons for the shortfall provided by the Borrower include: DoW lacked a bridge engineer who was needed to scope bridge works. The EPM Director filled this void at times, but was unable to commit enough time to oversee achievement of the target amount. Bridge works contractors are very limited in PNG. This resulted in three cancelled tenders in West New Britain alone, due to lack of responses. Necessary bridge parts were not readily available in PNG. Yumiyet Bridge Program" financed by the GoPNG led to construction/replacement of bridges in six provinces, which reduced the urgency for bridge replacement under RMRP. Further, construction of some of the planned bridges was cancelled due to inaccessibility to the proposed sites, or low economic rates of return caused by high construction costs and/or relatively lower than expected traffic volumes. 6 6 Replacement of national bridges ensured good road connectivity and supported a safer and more reliable road system. 6 0 Cancelled. The Borrower reported that the contract for the supply of six bridges was awarded in 2004, and it was thought that this would be fulfilled within 12 months. Delays were encountered, the main reason cited being the establishment of the letter of credit by GoPNG and twelve months passed, and no objection for a contract price variation was not granted by the World Bank. 32

47 Project Component 4: Provincial Bridges Bridge Maintenance and Rehabilitation Bridge Replacement Project Component 5: Implementation Support Employers Project Manager Design Incremental Operating Costs Purchase of Equipment 2 2 Fundamental to achieving the project development objective through contributing to the promotion of an efficient, safe and reliable roads transport system. 1 1 This bridge restored reliable transport along an important provincial road. EPM established EPM established EPM established as agency to implement RMRP on behalf of GoPNG. The EPM consisted of a Project Manager/Team Leader, Financial Controller, Provincial Engineers to supervise works in RMRP provinces, and other shortterm staff. Provision of necessary assistance to Implementation Agency. Provision of funding to support incremental operating costs as necessitated under the project. To assist with project implementation, equipment was acquired to support the following: Laboratory; Traffic Counting; and Assistance provided throughout implementation. Funding provided throughout implementation. Equipment acquired. A number of consultants were engaged to carry out feasibility studies, design, preparation of bidding documents and alike. Difficulties reported by the Borrower were: DoW design office lacking in experience design engineers to check the consultants work and approve. There were delays in providing the services by design consultants on time. These covered necessary costs associated with travel/incidentals for the DoW environmental monitoring and road supervision activities. Purchase of laboratory equipment Basic laboratory facilities were set up in East New Britain, Manus, Oro, and West New Britain to facilitate the testing of material; grading, compaction, liquid limit, etc., in the provinces as a mechanism for quality control. Personnel were trained to operate the laboratories and this worked well. Major tests such as California Bearing Ratio for example, were carried out at the Lae and Boroko regional laboratories. Traffic counting equipment Three automatic traffic counters were purchased 33

48 Environmental Monitoring for DoW maintenance Coordination Branch to facilitate the traffic counts for the seal roads more accurately and efficiently. The Borrower has reported that a lack of funding support for RAMS limits the use of traffic counters, and installing and protecting this equipment from vandalism and theft is costly. Also, staff trained in the use of this equipment have mostly been transferred to other departments or have resigned to join the private sector workforce. Audit Services Strengthening the Financial and Operational Capacity of DOW Independent auditing of project accounts. Limited capacity in DoW for carrying out Financial and operational tasks. Audits carried out. DoW capacity strengthened. Purchase of Environmental Monitoring Equipment Cancelled. DoW decided to remove this sub-component due to the equipment being considered sensitive and requiring special storage and maintenance. An alternative option to loan equipment from the Department of Environment and Conservation was cited. One of the financing conditions for RMRP was for GoPNG to appoint independent auditors to audit the project accounts. The Borrower reported that the Auditor General s Office is the mandated agency for the management of audits for all PNG funds. The arrangement of an annual independent auditor had to be done through the AG. The Attorney General s Office (AGO) then was to contract out to external companies the auditing of the project accounts. The AGO insisted on using their standard tender and contract documents for engaging an independent auditor to audit the project accounts. The World Bank requested that their standard bidding documents be used and no progress was made over this conflict for some time. Eventually, DOW decided to fully fund the external auditor fees from the GoPNG counterpart funds to enable the audits to proceed. Discontinued. The Borrower has reported that initially this item was to be undertaken by the original RMRP Project Financial Controller who worked with the Departments audit committee to address serious issues raised by the AGO s office after undertaking a full audit of the Departments operations. By all accounts, this proved to be less than satisfactory and eventually was discontinued with the commencement of the AusAID funded Transport Sector Support Program. This project saw the appointment of a Financial Management Advisor whose role was to provide financial management and direction and to assist in strengthening the capacity of the senior management to effectively control the resources of the Department. 34

49 Project Component 6: Road Sector Support Strengthening Small- and Medium-Sized Contractors Capacity Building of the DoW and Provincial Administrations Bridge Inventory and Bridge Management Systems (BIBMS) Study of Socioeconomic Impacts of Road and Bridge Improvements Intention to improve capacity amongst S&M sized contractors to bid and carry out works. Strengthening of DoW and Provincial Administrations. Establishment of BIBMS/BMS to allow data collection and usage for sustainable planning and management of bridge assets. Study to assess benefits arising from project works to local communities and individuals. Practical road and bridge maintenance training provide to 79 small and medium domestic contractors. 45 DoW central and provincial level staff trained in six provinces. BIBMS established. Study launched in 2005 and completed in This sub-component was used to strengthen private sector participation in road and bridge maintenance and rehabilitation. The training was undertaken by two consultants between 2004 and This sub-component was used to build capacity of DoW and Provincial Administrations, in order to strengthen the strategic planning and management of the road sector which would contribute to promoting an efficient, safe and reliable roads transport system in the RMRP participating provinces. A Bridge Management System (BMS) was developed by a consulting firm engaged under RMRP to cover all the provinces in PNG. Since the funding was limited, the updating of bridge data and condition surveys was limited to the project provinces. The BMS was developed to strengthen the strategic planning and management of the road sector. The study was commissioned with the aim of identifying the socio-economic changes resulting from the road and/or bridge improvements under RMRP. 35

50 Annex 3: Economic and Financial Analysis 1. The objective of the project s maintenance and improvement program was to minimize the total costs over the life cycle for a road for both agency and users considering the existing traffic volume and condition of a road section. The project road works program was developed by DOW using the Road Asset Management System (RAMS) and was subjected to economic analysis by consultants using the Highway Development and Management Model. Periodic and specific maintenance were determined from the RAMS in a coarse screening process for sub-projects based on net present value (NPV) divided by cost. Upgrading gravel roads to sealed roads and pavement reconstruction were determined from a fine screening process. 2. Based on the above, a three-year works program with a value of US$52.1 million (in March 2001 prices) was developed comprising routine maintenance for national and provincial roads and selected upgrading or pavement rehabilitation. The project appraisal document presented a list of 56 roads showing the province, road name and length of sections requiring restoration, together with the NPV divided by costs and the economic rates of return (ERR) and the proposed year of implementation. Under the most likely traffic growth scenario, the NPV estimated at appraisal was US$36.5 million and the corresponding ERR was estimated at 30 percent. The ERR for the sub-projects ranged from 14 percent to over 100 percent. Those rates were calculated by comparing the most effective treatment with the do minimum case. 3. The table below presents the average road work costs per km given in the 2001 project appraisal document, and corresponding 2011 prices. The road works correspond to the planned 2002, 2003, and 2004 road works program that totals 377 km, of which 22%, 49%, 23%, and 7% are for pavement reconstruction, regravelling, sealing and upgrading works, respectively. Table 1: PAD 3 Average Costs (US$ per km) Road Work 2001 prices 2011 prices Reconstruction 134, ,031 Regravelling 20,323 25,810 Sealing 58,866 74,760 Upgrading 97, , The evaluation of the actual road works financed with IBRD and IDA funds, show that around 788 km of road works were rehabilitated or upgraded from 2003 to 2012, of which 29%, 28%, 31%, and 12% are for pavement reconstruction, regravelling, sealing and upgrading works, respectively. Most of the road works (67%) were executed during the 2004 to 2007 period. The following table presents the average actual road work costs per km (in 2011 prices). 3 Project Appraisal Document. 36

51 Table 2: Actual Average Costs (US$ per km) Road Work 2011 prices Reconstruction 131,767 Regravelling 64,642 Sealing 58,152 Upgrading 283, The comparison of the actual and estimated costs at appraisal show that the actual reconstruction and sealing costs are around 78% of the estimated costs at appraisal, while the regravelling an upgrading costs are around 230% to 250% of the estimated costs at appraisal. Considering the percentage of road works executed under the project per road work type, the actual cost of the overall program is 13% higher than the one estimated at appraisal. 6. The following table shows, for each type of road works, the average traffic, in vehicles per day, and the average ERR, in percent, as estimated at appraisal. Regravelling road works carry the least traffic, while the sealing works yielded the lowest average ERR. Road Work Table 3: PAD Average Traffic and ERR Traffic (vehicles per day) ERR (%) Reconstruction Regravelling Sealing Upgrading The ex-post economic evaluation first replicated the ERR results given in Table 3, for each road work type, simulating a representative road carrying the given daily traffic and rehabilitated or improved with the given road work type. The ex-post evaluation then recalculated the ERR of the representative roads, this time adopting the actual average costs of each road work type. The following table presents the results of the ex-post economic evaluation. Table 4: Ex-Post Economic Evaluation Road Work ERR (%) Reconstruction 60 Regravelling 22 Sealing 27 Upgrading Considering the actual average road works unit costs per km, the ex-post ERR is higher than 12 percent for all road work types. The ex-post ERR ranges from 21 percent for upgrading works to 60 percent for reconstruction works. Thus, the program remains economically justified considering the actual road works costs. 37

52 Annex 4: Bank Lending and Implementation Support/Supervision Processes (a) Task Team Members Lending Names Title Unit Responsibility/ Specialty Supervision/ICR Jim Reichert Task Team Leader EASNS Laura E. Bailey Country Manager EACGF Demetrios Papathanasiou Sector Leader AFTSN Infrastructure Economist Cristiano Costa e Silva Nunes Sr. Procurement Specialist EAPPR Procurement Miriam Witana Procurement Specialist EAPPR Procurement David Michael Chandler Sr. Financial Management Specialist EASOS Christopher Robert Fabling Sr. Financial Management Specialist EASFM Thakoor Persaud Consultant ENV Economist Colleen M. Gollach Consultant EASNS Infrastructure Finance Hatim M. Hajj Consultant MNSSD Bruce M. Harris Consultant EASTE-HIS Social Development John A. Lowsby Consultant EASNS Civil Engineering Lawrence William Carlson Consultant EACGF Civil Engineer Sudesh Ponnappa Operations Analyst EACNF Paul Richard Wansbrough Consultant ENV Civil Engineer Scott Jared Wilkinson JPA, Transport and Infrastructure EASNS Daniel Jason Toga JPA, Transport and Infrastructure EASNS Fiona Doube ACS EACNF Transport and Infrastructure Transport and Infrastructure 38

53 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel & consultant costs) Lending FY FY FY FY FY FY FY FY FY Totals: Supervision/ICR FY FY FY FY FY FY FY FY FY FY FY FY FY FY FY FY FY FY Totals: Lending (Additional Financing) FY Total:

54 Annex 5: Beneficiary Survey Results ROAD MAINTENANCE AND REHABILITATION PROJECT CONSULTANCY SERVICES FOR SOCIO-ECONOMIC IMPACT STUDY Contract No: CS/DOW/IBRD/07 Prepared by Finnroad, Helsinki, July 2010 Background 2009 Evaluation Report Executive Summary The Road Maintenance and Rehabilitation Project is expected to speed-up national development by providing better roads and transportation system in the country. The roads are also seen to reduce poverty. Some economic changes can be seen quite soon, such as increased employment possibilities and incomes, or after several years or even after one generation, for example, as a healthy and educated labor force. The purpose of the Socio-economic Impact Study is to evaluate the impacts of the road rehabilitation on the households' income and expenditure, on access to health and education services and the reduction of poverty in the selected sample and control villages in the six participating provinces of Manus, East and West New Britain, Oro, Central, and Morobe starting in 2004 and ending in This report includes also the Western province, where the baseline study was made in The study villages were divided into the sample villages by the main roads and the control villages away from the main road. This report covers the final round of the study and combines and compares the previous data, and the data collected in Survey Population In 2009, 387 households were interviewed in 22 villages, 172 households in ten sample and 215 households in the twelve control villages. The same households were interviewed also in the previous study rounds. In the survey households, subsistence farmer was reported as occupation by 70 percent of the respondents and 30 percent report some other occupation than subsistence farmer. The main income sources of the households are vegetable and cash crop sales and salaries and wages. In the sample villages, the vegetable sales bring 27 percent, cash crop sales 26 percent and salaries and wages 29 percent of all incomes. The respective percentages in the control villages are 41, 29 and 13. Main Findings Impacts on road use to towns and market visits: After the rehabilitation of the main roads, visits to towns increased in 5 out of 8 sample villages, but only in 3 out of 10 control villages. Also the market visits increased. The trips to markets are not identical with the town visit, but are overlapping, because people use several market places. On average, the trips increased 1.7 times in the sample villages by the rehabilitated roads, and 1.4 times in their control villages. The increases were less for the non-rehabilitated roads, where the visits increased 1.2 times in the sample villages and 0.7 times in their control villages. Although people refute that the rehabilitation is the primary reason for their trips, nevertheless the trips increased according to the location of the villages and the road status. People do admit that the rehabilitation makes it easier to travel 40

55 to the vegetable markets, when they use the main roads, especially because the rehabilitation has increased the road services. Impacts on the PMV fees: The rehabilitation has not had any positive impacts on the PMV fees, which on average have increased percent. The increases have been bigger on the non-rehabilitated than on the rehabilitated roads. It is clear from the previous study rounds that people did not expect the fees to go down either, mainly because of the minimum fees set by the authorities. Despite the increases in the PMV services, according to the responses, there are still too few PMVs operating and no competition affecting the fees. Additionally, people face problems to get access to PMVs, and sometimes they have to wait for hours to get in. Impacts on income sources: Impacts on the vegetable market income. Independent of the number of visits, the single trip incomes have increased in all villages, except in one. These increases mainly follow the general trend of price increases of all goods and services. However, combined with more market trips, the average monthly income from the vegetable sales has increased 1.6 times in the sample and 1.7 times in the control villages. Roadside and village store sales. The roadside sales have increased and clearly benefited from the rehabilitated roads, because of more traffic and passengers on the roads. This is seen as a good income generating activity especially for women who can earn money near the home and at the same time can take care of children and other domestic tasks. Roadside sales are also seen to help unemployed to earn money. Additionally there is no need to pay transportation fees. Six villages have no roadside sales due to their location. Store owners in the villages can more easily transport needed commodities from the towns. However, in some villages store keepers have fewer sales because people prefer to buy store food and goods in the towns where the prices are less. Cash crop sales. Impacts on cash crop sales are modest because the prices are usually fixed, or fluctuate for other than rehabilitation reasons, but if the road is good, it is easier for the farmers to take their produce to factories and storehouses in the towns. From some villages, the company drivers collect rubber and oil palm produce and this collection system was there already before rehabilitation. The bad or missing feeder roads to the control villages show how important the good roads are. Presently in the control villages, many households could sell much more vegetables and/or cash crops in the markets if they could transport them at least to the main roads. Presently many cash crop sites lie idle because of the combined problem of no markets and bad feeder roads. Impacts on salaries and wages. The rehabilitation has had no impact on the people's salaries, which are fixed and decided by the employers. However, the rehabilitation is making the travel more convenient and faster. Impacts on the monthly incomes of the survey household On average, monthly incomes of the survey households have increased. 73 percent of the respondents say that their household income has increased. The share is bigger in the sample villages by the rehabilitated roads, where the percentage is 82. The road rehabilitation mainly has increased the incomes from the sales of vegetables and other agricultural products. The share of households with an income of K100 or less has decreased from 17 percent to 1 percent comparing the 2007 and 2009 data. For the share of households with an income of K500 or less, the decrease has been from 62 to 47 percent. Impacts on access to health services The rehabilitation of roads has had very little impact on the travel time to the nearest and most often used health services. Either people have services in their villages or their walk to the other villages, 41

56 but they do not usually use the main roads, but short cuts and bush tracts. From 18 villages people walk, less than half an hour from 12 villages, from 3 villages about one hour, although from one village walking takes 2 hours without a possibility to use PMVs. From 4 villages, people use PMVs to reach the health services. The rehabilitation has a positive impact on the PMV travel times, and people use them when they have to travel to the hospitals. On average, only ten percent of all health service visits are to the hospitals. Nobody mentions that they have no access to health services because of the distance or costs. Impacts on access to education services The rehabilitation of the roads has no positive impacts on access to schools, either because the schools exist in the villages or children walk to the schools in the near-by villages, using short cuts, bush tracks and river banks. Only in three villages children partly use the main road. For most, the walking time is from a few minutes to about half an hour, but also over one hour from a few villages. The biggest obstacles to education in some villages are too long walking distances for small children and in all villages the shortage of money to cover the ever increasing school fees. A surprisingly common reason was also that parents do not see the importance of schools and education. In some villages also children are needed to work on the family gardens and cash crop blocks. Road rehabilitation is benefiting the secondary school students by more convenient and faster PMV rides, but the access to secondary schools is better explained by the parents'/fathers' education level and attitudes and by the households' income level. Impacts on poverty According to most responses, very few households are poor in the villages. This was mainly based on the food security; people have food from their gardens. However, according to their own definition, respondents included the shortage of money stronger than food, to the poverty features/indicators: Food is no problem, the school fees are. Comparing the 2007 and 2009 data, there was a decrease of the share of households below the poverty line, from 57 percent to 40 percent, based on the monthly income of K500 or less. In 2007 the poverty line for a household was K500 according to the international definition, now that poverty line shows severe poverty, and the new definition of the poverty line requires at least K800 as a monthly household income. According to this poverty line, 67 percent of the households were below the poverty line. Only Manus, as a province, had more people above the poverty line (of K800) in 2009 than in 2007 (K500). In other provinces, only five sample villages, all by good rehabilitated roads, and one control village, had fewer households under the poverty line than in The comparison among the sample and control villages shows that the villages by the rehabilitated roads have better incomes and less poor, and have benefited more than the villages by nonrehabilitated roads. The importance of the good roads is becoming especially clear, when the sample and control villages are compared. The bad or missing feeder roads are negatively affecting the people's lives and incomes, by preventing people to visit more often markets and/or sell more their produce, because of no transportation to markets. In some villages cash crops are not sold for the same reason. In these villages the shares of the poor have increased. It is clear, based also on this study data, that the good roads are necessary preconditions for socioeconomic development. 42

57 Annex 6: Summary of Borrower's ICR and/or Comments on Draft ICR PAPUA NEW GUINEA ROAD MAINTENANCE AND REHABILITATION PROJECT Project Completion Report- Summary (Covering both Loan 7119PNG and Credit 4298PNG) Financing: Implementing Agency: Management Consultant: World Bank and Government of Papua New Guinea Department of Works, PNG Cardno Acil Pty Ltd (Cardno Emerging Markets) Issued: 15 December

58 NOTE; This Document is a Summary of the Project Completion Report dated July 2012, prepared for the Department of Works by the Employer's Project Manager, Cardno Acil Pty Limited. (Cardno Emerging Markets Pty Limited), and reference should be made to that Document for full details of all issues discussed in this Summary. ADB AusAID BMS DoW CSTB EPM FIDIC FMS GoPNG GST IBRD IDA km K LNG M MDB NBPOL NRA NEC PEPM PFC PMV PNG PNGSDP PWM RAMS RMRP RMRP II SIS SDR LIST OF ACRONYMS Asian Development Bank Australian Agency for International Development Bridge Management System Department of Works Central Supply and Tenders Board Employer s Project Manager International Federation of Consulting Engineers Financial Management System Government of Papua New Guinea Goods and Services Tax International Bank for Reconstruction and Development International Development Association Kilometres Kina Liquefied Natural Gas Million Multilateral Development Banks New Britain Palm Oil Limited National Roads Authority National Executive Council Provincial Employer s Project Manager Project Financial Controller Public Motor Vehicle Papua New Guinea PNG Sustainable Development Program Provincial Works Manager Road Asset Management System Road Maintenance and Rehabilitation Project Road Maintenance and Rehabilitation Project II Socio-economic Impacts Study Special Drawing Rights 44

59 1: INTRODUCTION AND ACKNOWLEDGEMENTS The Road Maintenance and Rehabilitation Project (RMRP) was National Government initiative funded by the World Bank; the National Government; and co-funders; and was implemented by the Department of Works (DoW) with assistance by Cardno Acil Pty Ltd as the Employer s Project Manager (EPM). This successful project had duration of more than nine years commencing in February 2003 and finishing in June The National Government provided funding of Kina (K) Million (M), and the World Bank financing was K94.16M. Co-funders consisted of PNG Sustainable Development Program (PNGSDP) which provided K48.26M, Provincial Governments which provided K11.10M, the Australian Agency for International Development (AusAID) which provided K8.64M, and New Britain Palm Oil Limited (NBPOL) which provided K2.17M. The total expenditure under RMRP was K358.67M. Residual funds from the National Government s contribution to RMRP will be transferred to the follow on project, RMRP II. Some highlights from RMRP include: Rehabilitated 788 kilometres (Km) of roads. Repaired and maintained 5,980 Km of roads. Constructed 6 new bridges and rehabilitated/maintained 47 bridges. Set the pattern for sealing the roads which cut down on maintenance costs. Capacity building via on the job training. Conducted a study on the impacts of the road improvements on socio-economic which is the first of its kind in PNG. A good deal of the success of RMRP can be contributed to the National Government, which, after a slow start, provided sufficient and timely counterpart contributions; and the excellent working relationship between DoW as Executing Agency, the World Bank, and the EPM team. This project highlights how much can be achieved through dedication and commitment to provide substantial and sustainable benefits to the people of Papua New Guinea. RMRP was hailed as a model of development expenditure by key stakeholders specifically the Ministerial Economic Committee and it was the only project considered appropriate to be extended under the same arrangements in Acknowledgment is also made for the excellent support provided by the Department of National Planning and Monitoring; Department of Treasury; Central Supply and Tenders Board (CSTB); and the World Bank Team headed by the respective Task Team Leaders (Hatim Hajj, Thakoor Persaud and Demetrios Papathansiou). Special thanks and appreciation goes to the PNGSDP which pro-actively assisted by providing cofunding to support the National Government. The PNGSDP commitment to provide substantial funding, and to support the outcomes of RMRP were greatly appreciated. The achievements and successful outcomes under RMRP have resulted in a follow-on project RMRP II being strongly supported and approved by both the National Government and World Bank. 45

60 2: PROJECT BACKGROUND The Government of Papua New Guinea (GoPNG) launched the medium term development strategy with emphasis on maintenance of the existing infrastructure in Due to changing priorities, both political and social, implementation of the medium term development strategy concept was held in abeyance for some five years. In the meantime the quality of the existing road network deteriorated drastically as the result of lack of funding due to the country s financial constraints. The National Government remained committed to restore the national and provincial road network and the World Bank was requested to provide funds for the road maintenance and rehabilitation. Consequently the World Bank agreed to fund road maintenance in specified areas of PNG under a loan agreement named the Road Maintenance and Rehabilitation Project RMRP. The RMRP was designed to cover maintenance and rehabilitation of roads and bridges both national and provincial, in Central, East New Britain, Manus, Morobe, Oro and West New Britain Provinces. The remaining provinces (excluding Gulf and Western) were to be shared between AusAID and the Asian Development Bank (ADB) with the objective of bringing the entire national road network to an acceptable condition within a reasonable time frame. DETAILS 1. Country Papua New Guinea 2. Project Title Road Maintenance and Rehabilitation Project 3. Borrower Independent State of Papua New Guinea 4. Executing Agency Department of Works LOAN DATA Loan Number 7119PNG Loan Amount US$40million (K125million) Loan Signed Date 12 August 2002 Loan Effectiveness Date 7 February 2003 Original Closing Date 31 December st Revised Closing Date 31 December nd Revised Closing Date 31 December 2009 CREDIT DATA (ADDITIONAL FINANCING) Credit Number 4298PNG Credit Amount SDR24.9million (US$37.3million, K98.2million) Credit Signed Date 3 July 2007 Credit Effectiveness Date 19 September 2007 Original Closing Date 31 December st Revised Completion Date 31 December nd Revised Completion Date 30 June : PROJECT OBJECTIVE AND COMPONENTS The objective of the project was to assist the GoPNG in promoting an efficient, safe and reliable roads transport system in the participating provinces through: (a) the improvement of selected road segments; (b) strengthening strategic planning and management of the road sector; and (c) 46

61 strengthening the institutional arrangements for road maintenance, including private sector participation. The above objective was achieved through the following six components: (A) maintenance and restoration of national roads; (B) maintenance and restoration of provincial roads; (C) maintenance, rehabilitation and replacement of national bridges; (D) maintenance, rehabilitation and replacement of provincial bridges; (E) implementation support and (F) road sector support. 4: IMPLEMENTATION 4.1 EXECUTING AGENCIES The two Executing Agencies responsible for implementation and their respective areas of responsibilities were as follows: 1. Department of Works (DoW) all project components other than the loan category Training and Support of the Construction Industry ; and 2. Department of Trade and Industry the Training and Support of the Construction Industry component. 4.2 IMPLEMENTATION ARRANGEMENTS Department of Works, assisted by the EPM, was responsible for: a) Plan, design, and implement the component activities and initiatives outlined in this scope of service. b) Together with other GoPNG agencies involved, procure the works, goods and services in accordance with GoPNG regulations and guidelines and to the approval of the World Bank. c) Providing administrative, procurement, personnel, operational, contract management support and implementation of program activities. d) Ensuring that all project-related documents and records are maintained efficiently to enable ease of access during post reviews by authorized GoPNG and World Bank staff, and stored securely to ensure access to only authorized personnel. e) Stringent financial controls were necessary to ensure all funds were managed and controlled within both the World Bank guidelines and GoPNG legislative parameters. A financial management system was designed and implemented to meet the Bank s specific requirements in regards to financial reporting and management including withdrawal applications. 5: PROJECT LOANS, FINANCIALS AND DISBURSEMENTS 5.1 INTRODUCTION One of the loan conditions under covenant was for GoPNG to provide counterpart funds in a timely basis. There were some delays in the early stages up to and including 2003 due to unavailability of sufficient counterpart funds for this project which resulted in financial payment cash-flow difficulties. This was addressed by GoPNG in arrangements with AusAID initially in 2004, and then PNGSDP from 2005 to 2012, to provide additional required counterpart funds on behalf of GoPNG. NBPOL provided some funds in 2007 for a section of the New Britain Highway in which they had an interest. Overall the GoPNG co-funding performance has been satisfactory in meeting the needs and goals of the project. 47

62 5.2 PROJECT FINANCING Funded By IBRD Program IDA Program Total World Bank 118,410,508 94,164, ,575,113 GoPNG 50,249,065 36,284,541 86,533,606 AusAID 8,639, ,639,485 PNGSDP 18,284,164 29,980,727 48,264,891 Provincial Governments 18,997,318-1,466,296 17,531,022 New Britain Palm Oil Ltd 2,170, ,170,547 Totals: 216,751, ,963, ,714, DISBURSEMENTS PROJECT COST BREAKDOWN BY COMPONENT/CATEGORY IBRD LOAN AND IDA CREDIT PROGRAMS COMBINED Category Civil Works National Roads Description World Bank Financing IBRD IDA Total Kina (million) 1A Routine Maintenance 37% 11,666,405 12,037,828 23,704,234 1B Restoration 70% 95,679,366 91,677, ,356,367 Provincial Roads 1C Routine Maintenance 37% 3,699, ,117 3,800,083 1D Restoration 70% 24,733,813 7,510,972 32,244,785 National Bridges 1E Maintenance and Rehabilitation 70% 3,068,438 6,003,938 9,072,376 1F Replacement 70% 11,446,660 4,007,191 15,453,851 Provincial Bridges 1G Maintenance and Rehabilitation 70% 205, ,425 1H Replacement 70% 1,150, ,150,383 Goods Sub-total 151,650, ,337, ,987,504 2 Emergency Bridges and Parts 100% Laboratory Equipment 100% 324, ,103 2 Traffic Counting Equipment 100% 32, ,341 Environmental Monitoring 2 Equipment 100% Sub-total 356, ,444 Training 3 Training Workshops Conferences 91% 287, ,785 Consultants Services Sub-total 287, ,785 48

63 Category Description World Bank Financing IBRD IDA Total Kina (million) 4A EPM/PEPM 83% 16,392,805 26,455,111 42,847,916 Design, Bid Documents and 4B Supervision 75% 820,965 1,142,794 1,962,000 Bridge Inventory and Management 4B System 100% 2,155, ,155,452 4B Socio-economic Impact Study 100% 876,256 1,985,894 2,862,150 4C Construction Industry Training 100% 1,756, ,756,281 4D Strengthening FMS 100% E Auditing and Accounting Services 100% 856,641 1,689,348 2,545,989 Incremental Operating Costs Sub-total 22,858,400 31,273,138 54,131,538 5 Incremental Operating Costs 80% 28, , ,902 Other 6 Fee 100% 1,481, ,481,481 7 Unallocated 69% GST 16,852,032 12,032,586 28,847,118 9 Other 34,542 17,268 51,830 Total 193,551, ,799, ,351,132 6: PROJECT TARGETS AND ACHIEVEMENTS 6.1 COMPONENT A - MAINTENANCE AND RESTORATION OF NATIONAL ROADS A.1 - Routine Maintenance of National Roads Target (annually) km Achieved km (average over 2003 to 2007) Expenditure (M)... K23.70 A.2 Restoration of National Roads Target km Achieved km Expenditure (M)... K COMPONENT B - MAINTENANCE AND RESTORATION OF PROVINCIAL ROADS B.1 - Routine Maintenance of Provincial Roads Target (annually)... 50km Achieved (average) km (2003 to 2008) Expenditure (M)... K3.80 B.2 - Restoration of Provincial Roads 49

64 Target km Achieved km Expenditure (M)... K COMPONENT C MAINTENANCE, RESTORATION OF REPLACEMENT OF NATIONAL BRIDGES C.1 - Maintenance and Rehabilitation of National Bridges Target Achieved Expenditure (M)... K9.07 C.2 - Replacement of National Bridges Target... 6 Achieved... 6 Expenditure (M)... K15.45 C.3 Supply of 6 Emergency Bridging Sets The Government had started the Umiyet bridge construction program to supply and construct bridges nationwide and thus DoW elected to cancel this component from the RMRP Program. 6.4 COMPONENT D MAINTENANCE, REHABILITATION AND REPLACEMENT OF PROVINCIAL BRIDGES D.1 - Maintenance and Rehabilitation of Provincial Bridges Target... 2 Achieved... 2 Expenditure (M)... K0.21 D.2 Replacement of Provincial Bridges Target... 1 Achieved... 1 Expenditure (M)... K1.15 7: LESSONS LEARNED 7.1 ENHANCED BIDDING DOCUMENT The DoW standard contract which uses the World Bank s standard bidding document for small works is general in form and content, and created a number of problems with contractors during supervision of the contract works. The standard bidding document was enhanced with the following new sub-clauses added under the specification, special provisions during the implementation of the IDA program. (i) (ii) (iii) quality assurance system requirements; environmental management plan requirements; HIV/AIDS prevention program requirements ( program delivered by an accredited agency); 50

65 (iv) (v) health and safety of workers requirements; and prior performance within the bid documents qualification criteria. The new clauses placed a clear obligation on the contractor to produce management plans with defined methodology to achieve the expected outcomes which are listed under these clauses, as guidance for the contractor to prepare its plan. All plans, including the implementation of same, are pay items in the contract bill of quantities, with payment divided between as a lump sum for the plan and progressive payment for implementation period. These additional provisions have proven effective in achieving the social objectives. 7.2 PROCUREMENT OF WORKS AND EXECUTION OF CONTRACTS Village Groups The RMRP initially procured routine maintenance via village groups. The issues faced in implementing routine maintenance by village groups were many: a) Payments were demanded immediately after completion of each cycle of maintenance which put extra pressure on DoW work supervisors and account section to process claims and issue cheques within a short period of time, and demands for payments to be made in cash. b) Long delays in the reconciliation of accounts and application for reimbursement of expenditure by the provinces. c) The village groups often did not having bank accounts and thus found it difficult to cash cheques and therefore demanded cash payments, which was an unsafe method of payment. d) In number of instances, an office bearer absconded with the cash. e) Opinion as to the quality of output differed and this lead to disputes. f) Claims of villagers demanding toll from road users for them maintaining the roads. Solution: In the later stages of the project, larger sections of the road were procured under one contract through national shopping (minimum of three quotes) with the provision that the contractor would employ and pay the villagers to perform tasks. Minor Works Contracts (Up to K300,000) The issues faced with minor works contractors were: a) Small contractors experienced cash flow problems in purchasing the materials such as premix for pothole repairs, gabion baskets etc, leading to delays in the work. b) There was a provision in the contract to receive 10% advance payment, but most contractors could not use this facility as their bank credit rating was poor, preventing them from being able to secure an advance payment guarantee from a bank. Solution: The cash flow problem was addressed to certain extent by paying the progress claims within a short period of time, i.e. in most cases less than two weeks against 28 days in contract. Major Contracts a) There is a limited number of national contractors able to carry out major contracts, therefore a small pool of would bid the same projects. b) Delayed delivery of the completed works by the contractors, even though liquidated damages were applied, impacting on the RMRP program for implementation. c) Contracts processed within the CSTB system must be less than K10M, and this low limit discouraged some of the better resourced contractors. Propose limit be lifted to K. 51

66 d) The alternative of awarding the contracts and signing the agreements by the NEC normally results in long delays, taking perhaps six to nine months or longer. e) Limited number of bridge contractors meant that some proposed/planned bridge projects could not proceed either due to extremely high bid prices or no bids received. f) Insufficient counterpart funds were available at the start of the project. The situation improved markedly after PNGSDP provided the necessary counterpart funds for this component. g) Contractor s faced unreasonable compensation claims and/or damage to equipment from local villages for issues outside the contractor s control. Solution: Make the contract packages large enough to attract better resourced local contractors and foreign contractors. To achieve interest, the lower limit may be K50M. Ensure provincial governments and DoW work closely with villagers to assist the understanding of the importance of improved road infrastructure and avoid land disputes. 7.3 CONTRACT WORKS SUPERVISION a) DoW unable to provide sufficient experienced supervisory staff. This is being exacerbated by the recruitment of engineers into the oil and gas and mining industries. b) Lack of funding to support supervisory staff to carry out the work, eg for vehicle fuel and repairs. Solution: This was addressed by DoW in agreeing to employ national engineers in the EPM team. Otherwise, there is a need for DoW to consider the use of consultants. 7.4 DOW FINANCIAL MANAGEMENT a) Lack of experienced financial staff to undertake basic DoW financial processes. b) Lack of motivation or interest by financial staff to undertake basic DoW financial processes leading to very slow processing time (i.e. payments to contractors and consultants; bank reconciliations; making numerous errors; not fixing transaction errors etc). c) DoW not meeting legal taxation obligations by not raising withholding tax cheques or by not releasing them to the Internal Revenue Commission for many months. Solution: This was addressed by: EPM putting in place very strong internal controls and stringent monitoring and checking of all transactions; constant follow up and review of outstanding issues; and working closely with DoW senior management to raise issues for action. 7.4 ANNUAL EXTERNAL AUDIT Slowness of conducting the annual external audit managed by the Auditor General s Office resulting in the issue of the audit certificate constantly being late. Also, relationship issues between the Auditor General and external auditors. Solution: Advanced meetings with the Auditor General s Office to discuss the timeframe required under the Loan/Credit Agreement, and constant follow-up to expedite the issue of the certificate. Agreement reached for the AG s Office to tender and contract the external audit company. 52

67 7.6 FINANCIAL MANAGEMENT SYSTEM Complex nature of World Bank s loan categories, various reimbursement percentages, disbursement applications and reporting requirements in conjunction with multi funding sources (i.e. World Bank, GoPNG, AusAID, PNGSDP, provincial governments) necessitated strong financial management. This became even more important with the combination of both IBRD and IDA funding. Solution: The development of a stand-alone financial management system individually designed to cater for the specific RMRP requirements. 7.7 LAND DISPUTES Road corridors are in the customary land, except few sections which were purchased by the government. Land owners have agreed with the government to build the roads through their customary land. A consultative process was put in place by appointing provincial government land officers to be in-charge of consultation and communication with all stakeholders to ensure there was a common understanding and acceptance (ownership) of project goals and to the advantages to all parties. Nevertheless there were occasional issues with local communities which lead to delays to the contractors. These were resolved via the consultative process, with the intervention of DoW. Solution: A consultative process to be carried out at the planning stages and a formal agreement signed with all land owners. 53

68 APPENDIX A KEY STAFF INVOLVED IN THE IMPLEMENTATION DEPARTMENT OF WORKS Secretary Deputy Secretary (Technical) First Assistant Secretary (Operations) First Assistant Secretary (Contract Management) First Assistant Secretary (Design Services) Assistant Secretary (Maintenance Coordination Services) Project Director (World Bank Projects) Environmental Manager Regional Works Manager (Northern and Southern) Regional Works Manager (Islands) Provincial Works Manager (Central) Provincial Works Manager (East New Britain) Provincial Works Manager (Gulf) Provincial Works Manager (Manus) Provincial Works Manager (Morobe) Provincial Works Manager (Oro) Provincial Works Manager (West New Britain) CENTRAL SUPPLY AND TENDERS BOARD The Chairman Board Secretary Road Sector Officer DEPARTMENT OF FINANCE First Assistant Secretary DEPARTMENT OF TREASURY Deputy Director First Assistant Secretary Assistant Secretary Senior Financial Analyst (World Bank) DEPARTMENT OF NATIONAL PLANNING & MONITORING Deputy Secretary First Assistant Secretary Planning Officer (World Bank Desk) DEPARTMENT OF TRADE AND INDUSTRY Deputy Secretary Assistant Secretary (Training) Training Officers DEPARTMENT OF TRANSPORT First Assistant Secretary Assistant Secretary 54

69 AUDITOR GENERAL S OFFICE Auditor General Assistant Auditor General Assistant Auditor General Special Audits Auditor National Government Branch PROVINCIAL GOVERNMENTS Central Provincial Administration East New Britain Provincial Administration Gulf Provincial Administration Manus Provincial Administration Morobe Provincial Administration Oro Provincial Administration West New Britain Provincial Administration WORLD BANK Task Team Leader Civil Engineering Consultant Procurement Specialist Financial Management Specialist Social Development Specialist Environmental Specialist Senior Operations Officer Senior Program Assistant MANAGING CONSULTANT CARDNO ACIL PTY LTD (CARDNO EMERGING MARKETS) Project Director (part time) Full-time assignments Team Leader Project Financial Controller Senior Provincial Engineer (Central and Gulf) Senior Provincial Engineer (Oro and West New Britain) Provincial Engineer (East New Britain and Manus) Provincial Engineer (Morobe) Assistant Provincial Engineer (Central) Assistant Provincial Engineer (Gulf) Assistant Provincial Engineer (Oro) Short term assignments Procurement and Contracts Engineer Project Accountant Senior Bridge Design Engineer (checking of bridge designs) Bridge Engineer (Karama Bridge load capacity assessment) 55

70 Annex 6a: Borrower's Comments on Bank ICR DEPARTMENT OF WORKS HEADQUARTERS World Bank Project Co-ordination Branch P O Box 1108 Telephone: BOROKO, NCD Facsimile: Papua New Guinea gkgabi@works.gov.pg Our Ref : 128P-60MP-820/M34 Date : 19 th December 2012 Mr. Jim Reichert Task Team Leader, RMRP II Level 19, 14 Martin Place CML Building Sydney NSW 2000 Australia Dear Jim, RE: DoW COMMENTS ON IMPLEMENTATION COMPLETION REPORT FOR ROAD MAINTENANCE AND REHABILITATION PROJECT FUNDED UNDER IBRD LOAN NO PNG AND IDA CREDIT NO PNG The version of the Bank s ICR was in pdf so we were not able to mark the suggested changes in the text. Instead we have listed our comments below, and suggest that you amend the ICR accordingly. DoW Comments on World Bank Draft ICR, dated 5 December The Bank s ICR is considered comprehensive and accurate and DoW is in general agreement. A few detail comments are provided as follows. 2. DoW has produced a Summary Project Completion Report and proposes that this be included as Annex 7 Summary of Borrower s in the Bank s ICR, which was sent to you yesterday. 3. Paragraph 38. The figures are not correct or complete, if both National and Provincial Roads are included. Also, DoW suggests that the amounts be stated in PGK. The following corrections are proposed: At the end of the first sentence, add after. maintenance, the words of National and Provincial Roads. 56