IMPLEMENTATION COMPLETION AND RESULTS REPORT (SLIDF TF-13246) ON AN IDA GRANT FROM THE SIERRA LEONE INFRASTRUCTURE DEVELOPMENT FUND

Transcription

1 Public Disclosure Authorized Document of The World Bank Report No: ICR Public Disclosure Authorized IMPLEMENTATION COMPLETION AND RESULTS REPORT (SLIDF TF-13246) ON AN IDA GRANT FROM THE SIERRA LEONE INFRASTRUCTURE DEVELOPMENT FUND IN THE AMOUNT OF US$16 MILLION TO THE REPUBLIC OF SIERRA LEONE FOR AN ENERGY ACCESS PROJECT January 31, 2018 Public Disclosure Authorized Public Disclosure Authorized Energy and Extractives Global Practice Africa Region i

2 CURRENCY EQUIVALENTS (Exchange Rate Effective October 31, 2017) Currency Unit = SLL SLL 1.00 = US$ US$1.00 = SLL 7, FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS AfDB BIS DFID EDSA EGTC EIRR ESURP EVD FM ISR IRR M&E MoE MTR NPA NPV O&M PAD PDO PMU PRSP RAP SLIDF UPS African Development Bank Business Information System U.K Department for International Development Electricity Distribution and Supply Authority Electricity Generation and Transmission Company Economic Internal Rate of Return Energy Sector Utility Reform Project Ebola Virus disease Financial Management Implementation Status and Results Report Internal Rate of Return Monitoring and Evaluation Ministry of Energy Midterm Review National Power Authority Net Present Value Operation and Maintenance Project Appraisal Document Project Development Objective Project Management Unit Poverty Reduction Strategy Paper Resettlement Action Plan Sierra Leone Infrastructure Development Fund Uninterruptible Power Supply ii

3 Vice President: Country Director: Sector Manager: Project Team Leader: ICR Team Leader: ICR Primary Author: Makhtar Diop Henry Kerali Wendy E. Hughes Nash Fiifi Eyison Nash Fiifi Eyison Nestor Ntungwanayo iii

4 REPUBLIC OF SIERRA LEONE THE ENERGY ACCESS PROJECT CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context and Development Objectives 1 2. Assessment of Outcomes Key Factors that Affected Implementation and Outcome Bank Performance, Compliance Issues, And Risk to Development Outcome Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1. Project Costs and Financing Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Bank Lending and Implementation Support/Supervision Processes Annex 5. Beneficiary Survey Results Annex 6. Stakeholder Workshop Report and Results Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Annex 9. List of Supporting Documents MAP 51 iv

5 A. Basic Information Country: Sierra Leone Project Name: Project ID: P L/C/TF Number(s): TF ICR Date: 01/31/2018 ICR Type: Core ICR Lending Instrument: SIL Grantee: Original Total Commitment: Revised Amount: Environmental Category: B Sierra Leone Energy Access Project - Sierra Leone Infrastructure Development Fund REPUBLIC OF SIERRA LEONE US$ Million Disbursed Amount: US$ 15.67M US$ Million Implementing Agencies: Ministry of Energy Cofinanciers and Other External Partners: DFID B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 03/08/2012 Effectiveness: 09/20/ /18/2013 Appraisal: 09/11/2012 Restructuring(s): 10/29/ /13/2017 Approval: 01/23/2013 Mid-term Review: 06/30/2016 No MTR Closing: 10/31/ /31/2017 C. Ratings Summary C.1 Performance Rating by ICR Relevance of objectives Outcomes: Bank Performance: Monitoring and Evaluation: High Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory v

6 C.2 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: No Yes Satisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Original Actual Major Sector/Sector Energy and Extractives Other Energy and Extractives Major Theme/Theme/Sub Theme Private Sector Development Jobs Job Creation Urban and Rural Development Rural Development Rural Infrastructure and service delivery Urban Development Urban Infrastructure and Service Delivery E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Makhtar Diop Country Director: Henry Kerali Yusupha Crookes Practice Manager: Wendy E. Hughes Meike van Ginneken Project Team Leader: Nash Fiifi Eyison Elvira Morella ICR Team Leader: ICR Primary Author: Nash Fiifi Eyison Nestor Ntungwanayo vi

7 Measurement Unit Base Line F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) Original Project Development Objectives (PDOs) and Key Indicators (as approved) The PDOs are to: (i) reduce losses in electricity supply in Freetown Capital Western Area; (ii) improve commercial performance of the National Power Authority (NPA) 1 ; and (iii) increase access to electricity in selected rural areas. Project statement in the Project Appraisal Document (PAD) and in the Grant Agreement is identical. Revised Project Development Objectives The revised PDO is to reduce losses in electricity supply in Freetown Capital Western Area and to improve commercial performance of the Electricity Distribution and Supply Authority (EDSA). 2 (a) PDO Indicator(s) Performance against the Results Framework at Approval in January PROJECT RESULTS FRAMEWORK AT APPROVAL PDO Results Indicators Original Target Values (from approval documents) Actual Value Achieved at the project closure (July 31, 2017) PDO Core Results Indicators (1) Electricity losses per year in the project area Value (quantitative or Qualitative) % 38% 33% 34.5 % Date achieved 2/2/2009 1/23/2013 7/31/2017 Comments (incl. % achievement) Partially achieved: While the information is preliminary, actual percentage losses fell from 38% down to 34.5%, an achieved level of 70% of the target. Upgrade of the distribution network was key to this trend. The probability of achieving the target in a 12-month 1 NPA was unbundled and replaced by (a) EDSA and (b) the Electricity Generation and Transmission Company (EGTC) in January The PDO was revised during the October 2015 level one restructuring, following the cancellation of the component covering rural electrification activities. vii

8 period is high with the full deployment of the new substations and electrical lines. (2) Collection rate by EDSA Value (Quantitative or Qualitative) % 76% 86% 86% Date achieved 2/2/2009 1/23/2013 7/31/2017 Comments (including % achievement) Fully Achieved: The collection rate increased from 76% to 86%, a 10 percent increase. The surge is consecutive to (i) ) increase in ratio of consumers pre-paid metered consumers to postpaid consumers, and (ii) the launching of a strong revenue protection and arrears recovery program. (3) Public buildings in rural villages provided with access to electricity under the Project through photovoltaic systems Value (Quantitative or Qualitative) Number Date achieved 6/30/2012 1/23/2013 7/31/2017 Comments (incl. % achievement) The component supporting this activity was cancelled during the restructuring approved in October 2015, and the indicator was dropped at the same time. 4. Direct project beneficiaries in (number), of which are female (%) Value (Quantitative or Qualitative) Number % 0 205,400 51% of female 720, ,200 Date achieved 6/30/2012 1/23/2013 8/31/2017 Comments (including % achievement) Intermediate Results Indicators (1) Substations rehabilitated under the project Value (Quantitative or Qualitative) Target exceeded, as the target was more than tripled. Upgrade in the distribution network and improvement in the planning and commercial functions of EDSA are at the core of this performance. The number is derived from 120,000 households connected during the period of project implementation with an average of 6 persons per household. Number Date achieved 2/2/2009 1/23/2013 7/31/2017 Comments (including % achievement) Target exceeded: Five substations were rehabilitated at the Wilberforce, Kingtom, Blackhall Road, Ropoti, and Wellington locations. Ropoti substation was a brand-new substation, while new equipment for Blackhall Road substation, is scheduled to be installed in the context of an ongoing World Bank project (P120304). (2) Distributions lines rehabilitated under the project Value (Quantitative or Qualitative) Km Date achieved 2/2/2009 1/23/2013 7/31/2017 viii

9 Comments (including % achievement) Fully achieved: The 7.8 km power line from Blackhall Road substation to Wellington station via Ropoti substation, operating at 11kV was replaced and upgraded to 33kV and constructed on double circuit towers. (3) Pre-paid meters installed under the project Value (Quantitative or Qualitative) Number 8,000 20,000 20,000 Date achieved 6/30/2008 1/23/2013 7/31/2017 Comments (including % achievement) Fully achieved: 20,000 pre-paid meters, including 17,800 single phase and 2,200 three phases were installed in 20 neighborhoods of Freetown, with the target achieved at 100%. (4) Statistical meters installed under the project Value (Quantitative or Qualitative) Number Date achieved 6/30/2008 1/23/2013 7/31/2017 Comments (including % achievement) Partially achieved: 100 bulk meters were installed at 8 switching stations and substations, with the target achieved at 50%. (5) Photovoltaic systems installed under the project Value (Quantitative or Qualitative) Number Date achieved 12/31/2008 1/23/2013 7/31/2017 Comments (including % achievement) Component 2 devoted to funding this activity was canceled during the October 2015 restructuring and the indicator was also dropped (6) Staff recruited at the Project Management Unit (PMU) under the Project Value (Quantitative or Qualitative) Number Date achieved 12/31/2008 1/23/2013 7/31/2017 Comments (including % achievement) Partially achieved: The PMU hired 6 staff, or 85 percent of the project target. However, the staff technical capacity needed improvement ix

10 Measurement Unit Base Line Performance against the Results Framework at Restructuring in October 2015 PROJECT RESULTS FRAMEWORK AT THE OCTOBER 2015 RESTRUCTURING PDO Results Indicators PDO Core Results Indicators Original Target Values (from approval documents) Revised Target Values on October 28, 2015 Actual Value Achieved on July 31, 2017 the Closing Date of the Project Except Otherwise Specified. (1) Collection rate by EDSA Value (Quantitative or Percentage % Qualitative) Date achieved 02/02/09 01/23/ /28/2015 7/31/2017 Comments (including % achievement) (2) Direct project beneficiaries Value (Quantitative or Qualitative) Fully achieved: The surge is consecutive to (a)) increase in ratio of consumers pre-paid metered consumers to post-paid consumers, and (b) the launching of a strong revenue protection and arrears recovery program. Number 0 205, , ,000 Date achieved 06/30/12 01/23/ /28/2015 7/31/2017 Comments (including % achievement) (3) Female beneficiaries Value (Quantitative or Qualitative) Target exceeded, as the target was more than tripled. Upgrade in the distribution network and improvement in the planning and commercial functions of EDSA are at the core of this performance. Number 0 51% 51% 51% Date achieved 01/23/ /28/2015 7/31/2017 Comments (including % achievement) Fully achieved: Assuming that the female population represent 51 percent of the beneficiary households. (4) Electricity losses per year in the project area Value (Quantitative or Percentage 38 33% 33% 34.5 % Qualitative) Date achieved 01/23/ /28/ /16/2106 Comments (including % achievement) Intermediate Results Indicators (1) Substations rehabilitated under the project Partially achieved: While the information is preliminary, actual percentage losses fell from 38% down to 34.5%, an achieved level of 70% of the target. Upgrade of the distribution network was key to this trend. The probability of achieving the target in a 12-month period is high with the full deployment of the new substations and electrical lines. x

11 Value (Quantitative or Number Qualitative) Date achieved 02/02/09 01/23/ /28/2015 7/31/2017 Comments (including % achievement) Target exceeded: Five substations were rehabilitated at the Wilberforce, Kingtom, Blackhall Road, Ropoti, and Wellington locations. Ropoti substation was a brand-new substation, how Blackhall Road substation was only rehabilitated while waiting for the new equipment to be installed in the context of an ongoing World Bank project (P120304). (2) Pre-paid meters installed under the project Value (Quantitative or Qualitative) Number ,000 20,000 20,000 Date achieved 02/02/09 01/23/ /28/2015 7/31/2017 Comments (including % achievement) Fully achieved: 20,000 pre-paid meters, including 17,800 single phase and 2,200 three phases were installed in 20 neighborhoods of Freetown, with the target achieved at 100%. (3) Staff recruited at the Project Management Unit (PMU) under the Project Value (Quantitative or Number Qualitative) Date achieved 06/30/08 01/23/ /28/2015 7/31/2017 Comments (including % achievement) Exceeded: The PMU hired 6 staff, exceeding the project target. However, the staff quality needed technical improvement. (4) Statistical meters installed under the project Value (Quantitative or Number Qualitative) Date achieved 06/30/08 01/23/ /28/ /16/2106 Comments (including % achievement) Partially achieved: 100 Statistical meters were installed at 8 Switching stations and Substations, with the target achieved at 50%. (5) Distribution lines constructed or rehabilitated under the project Value (Quantitative or km Qualitative) Date achieved 12/31/08 01/23/ /28/ /31/2017 Comments (including % achievement) Fully achieved: The 7.8 km power line from Blackhall Road substation to Wellington station via Ropoti substation, operating at 11kV was replaced and upgraded to 33kV and constructed on double circuit towers. (6) Distribution lines rehabilitated under the project Value (Quantitative or km Qualitative) Date achieved 12/31/08 01/23/ /28/ /31/2017 Comments (including % achievement) Fully achieved: The 7.8 km power line from Blackhall Road substation to Wellington station via Ropoti substation, operating at 11kV was replaced and upgraded to 33 kv and constructed on double circuit towers. Note: Because the existing 11 kv line was replaced by a 33 kv line, the 7.8 km count both as lines constructed and rehabilitated Source: Implementation Status and Results Reports (ISRs) and other World Bank documents. xi

12 G. Ratings of Project Performance in ISRs No. Date ISR Actual Disbursements DO IP Archived (US$ millions) 1 09/30/2013 Satisfactory Satisfactory /07/2014 Satisfactory Moderately Satisfactory /28/2015 Satisfactory Moderately Unsatisfactory /01/2015 Satisfactory Moderately Satisfactory /15/2016 Satisfactory Moderately Satisfactory /14/2016 Satisfactory Moderately Satisfactory /07/2017 Satisfactory Moderately Satisfactory xii

13 H. Restructuring (if any) Restructuring Dates Board Approved PDO change ISR Ratings at Restructuring Amount Disbursed at Restructuring (US$ million) 3 Reason for Restructuring and Key Changes Made PDO IP (i) 10/28/2015 N S MS 6.26 To extend the closing date of the project from October 31, 2015 until April, , and revise the PDO and the Results Framework. The above changes were triggered by the cancellation of the Component 2 activities, impossible to implement due to the Ebola Virus Disease outbreak. (ii) 04/13/2017 N S MS To extend the closing date until July 31, 2017 to allow the completion of the unfinished activities, and to reallocate resources among categories. 3 The disbursement levels are estimates xiii

14 I. Disbursement Profile xiv

15 1. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. Context at Appraisal (i) Country Context. 1. The recent history of Sierra Leone is that of a successful transition from a decadelong civil war to a peaceful and stable country, with a democratic change of power: The country went through a devastating civil war during , but a peace and reconciliation process brought a closure to the conflict period, and allowed two democratic elections in 2007 and in However, the recent internal situation remained fragile, due to a complicated political economy marked by the alluvial diamond industry, youth unemployment, and regional instability. 2. While the economy grew and poverty fell in the aftermath of the civil war, the country continued to rank among the least advantaged countries. At project appraisal, Sierra Leone was ranked 180 th out of 187 countries in the United Nations Development Programme Human Development Index rankings, despite seven consecutive years of sustained economic growth. There were still high levels of unemployment; child and maternal death rates; as well as elevated gender disparities and vulnerability of women. There were also weaknesses in infrastructure (energy, water, ports, and roads) and in access to quality finance (notably rural finance), as well as vulnerabilities to external shocks. (ii) Sector Context: 3. The country s electricity access and consumption were among the lowest in Africa. Electricity access in the country was below 6 percent of the households in A large majority of the country s population relied on inefficient and polluting fuels such as kerosene for lighting and fuel-wood and charcoal for cooking, with their adverse impact on health and the environment. Public electricity services were limited to selected areas in the urban centers, and in rural areas, where the bulk of the population leaves, electricity access was practically non-existent (Project Appraisal Document [PAD] p.10). 4. Reduced power supply and degrading infrastructure were the key obstacles to expanding energy consumption. Power generation capacity remained disproportionate to the country s overall power demand. Hydropower volume from the newly-commissioned Bumbuna power plant was seasonal, and high costs of imported fuel for the thermal power plants, coupled with the transmission and distribution bottlenecks diminished the available power capacity. Moreover, fast growth in the mining sector implied an exponential increase in energy demand, and called for a major scale-up in generation capacity. There was a need for a thought-through sector strategy for exploiting synergies in electricity supply by facilitating joint public and private venture investments in the energy sector. 1

16 5. Outside the power supply deficit, transmission and distribution bottlenecks were the major impediments to expanding electricity supply. Both transmission and distribution capacity were severely constrained due to high losses, which reached over 38 percent. The low voltage levels in certain areas and the high level of fault occurrence contributed to poor quality of supply. Black outs and load shedding were common place. If not addressed, transmission and distribution bottlenecks were to block the expansion of generation capacity and to suppress electricity demand, which was expected to increase considerably in line with continued economic growth. 6. The structural and operational weaknesses of the then National Power Authority (NPA) was at the heart of country s power sector challenges. Shortcomings included (a) the lack of adequate technical, operational and financial management (FM) capacity, (b) inaccurate customer data base, and (c) dysfunctional metering, billing and revenue collection systems and poor accounting, which led to low levels of commercial and operational performance. The NPA s financial situation has been fragile as illustrated by its inability to fully honor payments for the purchase of Bumbuna's electricity, and its financial situation was heavily dependent on Government support. 7. Institutional and regulatory frameworks needed to be redesigned for a forwardlooking perspective for the energy sector: The sector was under the aegis of the Ministry of Energy and Water Resources (now the Ministry of Energy-[MoE]), which supervised a vertically integrated national utility, without a regulatory authority. Both the MoE and NPA were understaffed and lacked the skills and resources needed to efficiently run the sector. (iii) Rationale for World Bank Involvement 8. The World Bank s intent was to support the Government s broad power sector reform aimed at unbundling the sector. The Government approved a National Energy Policy and Strategic Plan in 2010 outlining sector priorities and objectives, and approved the objectives and principles for setting electricity tariffs. Unbundling the energy sector was at the core of the reform agenda, and aimed to separate the responsibilities for operating and maintaining existing government-owned assets into new entities, and associate the private sector much more. However, there were challenges in implementing the envisaged reforms, the key ones being (a) how to quantify the financial and economic impact of the unbundling exercise, (b) what will be the new organizational structure and the role of the regulator, and finally (c) how to integrate the institutional, regulatory and financing frameworks for rural electrification in the sector reform. 9. The project was a coordinated effort of the International Development Association (IDA) and the UK Department for International Development (DFID) to support an integrated energy sector reform program. The Government s reform agenda was a large program covering the increase of transfer capacity of the transmission and distribution system; the revamping of the NPA, and the scaling up of electricity access in rural areas. The scale of the challenges and the resources needed 2

17 called for an integrated program, well beyond the capacity of any one institution. The proposed project aimed to: (a) focus on the areas where results can be achieved in the short- to medium- term; and (b) help remove some of the critical barriers to sector development. The project was entirely funded by the Sierra Leone Infrastructure Development Fund (SLIDF), which was a multi-donor trust fund established by the World Bank in 2010 with the DFID as the anchor donor. The fund was a vehicle for pooling donor support to Sierra Leone aimed at: (a) facilitating expanded access to basic infrastructure services; (b) raising the efficiency and effectiveness of infrastructure development by improving sector governance and accountability; and (c) building Government s capacity to plan and manage development projects. 10. The World Bank had plans to complement the project by follow-on IDA operations in the context of a larger Energy Access Program supported by the donor community. The long-term World Bank agenda in Sierra Leone was to support the recovery of the electricity sector by increasing availability and quality of electricity supply, improving finances and operational capacity of the energy utilities and raising sector performance and financial viability. The World Bank had planned a follow-on operation to continue: (a) investments to upgrade the Freetown distribution network focusing on other critical segments of the network; and (b) the support to the implementation of a loss reduction program. During the period of project implementation, the World Bank was also implementing the Energy Sector Utility Reform Project (ESURP) and prepared the Western Area Power Generation Project (WAPGP). Other donors and institutions supporting the energy sector development included the ECOWAS, the Japan International Cooperation Agency (JICA), the African Development Bank, the Millennium Challenge Corporation and the Islamic Development bank. (iv) Project Theory of Change and Results Chain 11. The results chain set out in table 1 presents the theory of change linking the project activities, the expected outputs and outcomes, and the Project Development Objectives (PDOs) as identified at appraisal. Table 1: Project Results Chain for the Sierra Leone-Energy Access Project Activities Outputs PDOs/Outcomes Long term Outcomes Objective 1: To reduce losses in electricity supply in Freetown Capital Western Area 3

18 (i) Rehabilitation of primary distribution network, and loss reduction (ii) Restoration of the distribution of power at 33kV, to increase the evacuation capacity of the network and reduce associated technical losses (i) Primary substations rehabilitated under the project (ii) Existing line operating at 11kV replaced by the construction of a 7.8 km 33kV line (iii) Statistical meters installed under the project (i) Reduction of losses in electricity supply in Freetown Capital Western Area (ii) Improved reliability of the network and safety concerns of operators at the substations addressed (i) Improved efficiency in energy distribution (ii) Extension of energy access in the Freetown Western Area. Critical assumptions Financial and technical support from DFID The Government has technical capacity to warrant effective management per World Bank procedures and to oversee implementation. Objective 2: To increase access to electricity in selected rural areas Installation of photovoltaic systems in public buildings in 14 villages (i) Public buildings in rural villages provided with access to electricity under the project (i) Increased access to electricity through public lighting. (ii) Increased access to electricity by households Extension of energy access in the rural areas (ii) Photovoltaic systems installed under the project. Critical assumptions Financial and technical support from DFID The Government has technical capacity to warrant effective management per World Bank procedures and to oversee implementation Objective 3: To improve commercial performance of the National Power Authority (i) To install pre-paid meters network meters, and a revenue management system (ii) To provide technical assistance and conduct studies (iii) To hire staff and acquire equipment (i) Pre-paid meters and bulk meters installed under the project (ii) Revenue management system set up and operational (iii) Staff hired, studies completed, and equipment delivered. (i) Improved operational and commercial efficacy and efficiency of the National Power Authority (EDSA) (ii) Increased collection rate by EDSA (i) Improvement of EDSA commercial performance (ii) Improvement of EDSA financial health Critical assumptions Financial and technical support from DFID The Government has technical capacity to warrant effective management per World Bank procedures and to oversee implementation Source: PAD and Project Management Unit (PMU) Note: EDSA = Electricity Distribution and Supply Authority. 4

19 Project Development Objectives (PDO) 12. The original PDOs are to: (i) reduce losses in electricity supply in Freetown Capital Western Area; (ii) improve commercial performance of the National Power Authority; and (iii) increase access to electricity in selected rural areas. (v) Key Expected Outcomes and outcome Indicators 13. The key indicators for the PDO level results were the following: a. Electricity losses per year in the project area (percentage): the target was set at 33 percent from a baseline of 38 percent. b. Collection rate by EDSA: the target was set at 86 percent from a 2012 baseline of 76 percent c. Public buildings in rural villages provided with access to electricity under the project through photovoltaic systems (number); the target was set at 28 buildings from a baseline of 0. d. Direct project beneficiaries (number), of which female (percent): the project target was set at 205,400, of which 51 percent female, against a baseline of 0. (vi) Project Components 14. The original project had three components as delineated below: Component 1. Rehabilitation of Primary Distribution Network, Loss Reduction and Improvement of NPA s Operational and Commercial Performance (US$12.2 million) This component aimed to rehabilitate the primary distribution network (33/11kV) in Freetown area, including overhead lines and substations, the supply and installation of about 20,000 pre-paid meters, an appropriate vending and control system, and a statistical metering and data management system. The component also was to fund (a) the setup of an appropriate business information system (BIS) to improve the utility management, particularly in the financial, commercial and logistics areas, and (b) the provision of extensive technical assistance to ensure optimal application of the investment support, and on strengthening energy utility expertise in two key areas of operations regulation, and commercial management. Finally, the component intended to provide limited resources for compensation for new encroachments and temporary disturbances to households and businesses. 5

20 Component 2. Rural Electrification: (US$1.46 million) This component was devoted to support a pilot program for the installation of photovoltaic systems in public buildings in 14 rural villages to demonstrate applicability of the solar technology for larger deployment. Component 3. Project Implementation Management: (US$1.225 million) This component was specifically designed to strengthen project implementation and management capacity, and to finance external expertise needed to support key project management functions. B. Significant Changes During Implementation 15. The Ebola outbreak in 2014 made the rural area inaccessible, prompting (i) the cancelation of the component with rural electrification activities, (ii) the change of the PDO and the result framework, and (iii) the reconfiguration of the project components. The revised PDO is to: (i) reduce losses in electricity supply in Freetown Capital Western Area and (ii) improve commercial performance of the Electricity Distribution and Supply Authority (EDSA). 16. The key indicators for the revised PDO-level results are the following: (a) Collection rate by EDSA: from a baseline of 76 percent, the target of rate of collection at closure was maintained at 86 percent. (b) Direct project beneficiaries: from a baseline of 0, the target was set at 624,000. (c) Female beneficiaries: from a baseline of 0, the target a closure was set at 51 percent of the total beneficiaries. (d) Electricity losses per year in the project area: from a baseline of 38 percent, the target at closure was maintained at 33 percent. 17. There were two key changes in the Results Framework of the revised PDO, as follows: (a) the cancellation of the indicator related to rural electrification, and (b) the tripling of the target for the direct beneficiaries of the project. 18. Following the level a restructuring approved on October 28, 2015, the project s components were revised as described below, and the total project amount was reallocated toward the two remaining components (Components 1 and 3). Component 1: Rehabilitation of Primary Distribution Network, Loss Reduction and Improvement of EDSA's Operational and Commercial Performance (Revised amount: US$13.88 million) 19. While the rehabilitation of the distribution network remained the key task under this component, including the installation of 20,000 pre-paid meters, an appropriate 6

21 vending and control system and statistical meters, additional activities included (a) financial advisory services to the Government and EDSA for a proposed private sector thermal generation project and (b) increases in resettlement costs associated with the rehabilitation of a transmission line and substations along Blackhall Road. On the contrary, the BIS was eliminated from this component, as it did not attract bidders due to limited scope. Team decided to implement this as part of a larger utility management system under the Bank financed Energy Sector Utility Reforms Project. Component 2: Rural Electrification (Cancelled) 20. This component was dropped from the project scope as DFID preferred to exclude rural electrification activities and to pursue a more comprehensive approach to rural electrification under a separate new engagement. The funds initially earmarked for rural electrification were therefore reallocated to cover increased costs under Components 1 and 3. Component 3: Project Implementation Management (Revised amount: US$2.12 million) 21. There was a cost increase in Component 3 resulting from the implementation delays and the consequent closing date extension of the project for a total of 21 months until July 31, Other additional costs incurred to the operation cost was the provision of temporal accommodation for Project Management Unit (PMU) because of a fire accident that gutted their offices at the MoE-Electricity House. Reallocated resources were also used to acquire office equipment and vehicles, and software for the PMU. C. Rationale for Changes and their Implication for the Theory of Change 22. The Ebola Virus Disease (EVD) outbreak in 2014 triggered the October 2015 project restructuring. The Ebola crisis halted project implementation in 2014, especially because it made the rural area inaccessible. Consequently, the Government and DFID agreed to postpone the implementation of the second component (Rural Electrification). Project implementation was continued only after the approval of a level 1 restructuring in October 2015, which made the following changes to the project: (a) a revision of the PDO, (b) a reconfiguration of project activities and (c) few changes in the Results Framework. These changes are described in detail under sub-section B above. 23. The financial resources freed by the cancellation of Component 2 were reallocated to finance additional activities under Components 1 and 3. More specifically, the budget amount of US$1.5 million initially allocated for rural electrification was reallocated to the two remaining components to top up the funding for resettlement cost under Component 1 and cost increase under Component 3 as detailed under paragraph The impact of the October 2015 restructuring on the project s theory of change was clearly a dropping in outcomes expected from the objective of increasing 7

22 rural electrification, with a potential improvement of results in the two other objectives. With the cancellation of Component 2, outputs and outcomes expected from rural electrification activities could not occur, with some possibility of increase in the scope of the results expected from the two remaining objectives. Project achievements after restructuring were not expected to fundamentally differ from those accomplished before the restructuring. The key difference is that, while the absence of results in rural electrification downsized project efficacy before restructuring, performance of the revised project had to be assessed based on two objectives. While key indicators for the rural electrification objective were canceled, all other indicators were maintained with their targets, except that with the upgrade of the distribution infrastructure and the planning and commercial functions of EDSA, the energy access target indicator was tripled. II. ASSESSMENT OF OUTCOME A. Relevance of Project Development Objectives (PDO) 25. Original project. The PDO prior to restructuring was to (i) reduce losses in electricity supply in Freetown Capital Western Area; (ii) improve commercial performance of the National Power Authority; and (iii) increase access to electricity in selected rural areas. The project is consistent with the Second Poverty Reduction Strategy Paper (PSRP II) for Sierra Leone covering Expanding and improving electricity supply is one of the strategic priorities of the Government's National Development Program outlined in PRSP-II and continues to be a critical priority under the new Poverty Reduction Strategy for The 2013 letter of Government Policy in Utility Reforms in the Electricity Sector confirms the Government s commitment to complete the unbundling of the sector and mandated the MoE to lead the full implementation of the Electricity Act, including the resourcing of the EGTC and the EDSA. 26. The project is also consistent with Sierra Leone s Joint Country Assistance Strategy covering , which was adopted by the World Bank, the International Finance Corporation and the AfDB to support PRSP II with its two pillars: Human Development; and Inclusive Growth. A new Country Partnership Strategy in preparation reiterates the focus on energy to support growth in the extractive sector. Activities under the project are well aligned with previous World Bank operations and are supported by other development partners. 27. Revised project: The revised PDO is to: (i) reduce losses in electricity supply in Freetown Capital Western Area, and (ii) improve commercial performance of the Electricity Distribution and Supply Authority (EDSA). The relevance of the revised objectives remained high after the project restructuring, because the Government s programs and policies in the energy sector did not change, and the World Bank s strategies in Sierra Leone in the energy sector were not revised during the period of project implementation. What changed was only the scope of the project activities to be implemented during the project life, and not the relevance of project objectives. 8

23 B. Achievement of Project Development Objectives 4 Original project: January 2013-October percent of disbursed resources Rating: Substantial (a) Objective 1: To reduce losses in electricity supply in Freetown Capital Western Area: 28. Achievements exceeded project targets in the areas of upgrading primary substations, and distribution line. Key outputs generated under this objective were (a) the rehabilitation and upgrade of the energy distribution of the Kingtom, Wilberforce, and Wellington substations, (b) the creation of a new Ropoti substation, (c) the installation of original equipment for Blackhall Road substation and the supply of additional equipment (transformers, switchgears, and related materials and accessories), (d) the replacement of the 7.8 km power line from Blackhall Road substation to Wellington substation via the Ropoti substation, (e) the installation of 100 network statistical meters at 8 substations and locations, and (f) the compensation of 107 project affected persons. 29. Most outputs indicators were either achieved or exceeded as follows: (a) the number of substations created or rehabilitated under the project reached 5, exceeding the 4 substations targeted, and (b) the length of distributions lines rehabilitated under the project, was slightly exceeded, reaching 7.8 km, against a target of 7.5 km. On the contrary, the number of statistical meters installed under the project reached 100 units, or 50 percent of the target. 30. Key outcome achieved includes the reduction of electricity total losses per year, and improved reliability of the network, but precise assessment and validation of total and technical losses is still work in progress. EDSA reported that generation capacity utilization increased from 76 percent in December 2016 to reach 90 percent in March The reduction of total losses on its distribution network improved, reaching on average 34.5 percent (see figure 1) during January 2001-July 2017, against a baseline of 38 percent in 2009, and a target of 33 percent at project closure. However, a good measure of electricity losses requires better coordination between EDSA and EGTC which is being put in place. 4 A split assessment of the project outcome will be conducted for this project, due to the October 2015 restructuring, and the ensuing revision of the project objectives and the results framework. 9

24 Figure 1: Loss Reduction in Freetown Source: EDSA Monthly Progress Report August 2017 (b) Objective 2: To improve commercial performance of the electricity distribution Authority. 31. Key outputs consisted primarily in the installation of prep-paid meters, the setting up of a revenue management system, and the completion of studies that set the stage for better management of the commercial function in the energy sector. In summary, (a) 20,000 meters were installed in 20 neighborhoods of Freetown; (b) two EDSA appointed staff were trained, (c) a revenue management system with its backup, software management application, power banks (uninterruptible power supply [UPS]) for main and backup power, and Human Machine Interface (HMI) was delivered, (d) a transaction advisory and tariff study was completed, (e) a business plan for the electricity distribution utility was completed, including resources management, commercial enhancement, and technical and operations modules, as well as an electricity network investment plan setting out priorities for the Greater Freetown area and the rest of the country were completed, and finally (f) the bidding document for a Management Contractor was prepared. 32. Achievements toward performance indicators were as follows: (a) the target of 20,000 pre-paid meters was fully achieved, and (b) the target for staff recruited at the PMU was exceeded, reaching 6 against a target of 4 following 2015 restructuring. 33. Key outcome indicators achieved were the following: (a) the collection rate target by EDSA of 86 percent was achieved and, (b) the target of 620,000 direct project beneficiaries, of which 51 percent are female, was exceeded. These results were obtained because of (a) a decisive decision to increase consumers connected via pre-paid meters, and (b) the launch of a strong revenue protection and arrears recovery program, as well as sustained commercial audit. Figure 2 shows a net change in trend of the ratio between the volume of MWh consumed and the volume of MWh billed since October

25 Figure 2: Comparison of Levels of Electricity Consumption and Billing Source: EDSA Monthly Progress Report August It is worthy to mention that key studies funded by the project and detailed in para 31 have had a strong impact on the operational and commercial performance of EDSA. (c) Objective 3: To increase access to electricity in selected rural areas: 35. Due to the Ebola outbreak, the pursuit of this objective was cut short, and during the 2015 level 1 restructuring, a consensual decision was reached by the Government and DFID to postpone the implementation of the identified activities, and to fund them later in the context of a new financial instrument. Expected outputs and outcomes under this objective could not be pursued. (d) Assessment of performance of the original project: 36. Efficacy of the initial project is assessed as substantial, owing to good performance toward the two objectives and a situation of force majeure that annihilated all efforts to pursue the third objective. Based on what was developed under the first objective, performance toward the electricity reduction loss was positive. While precise quantification of performance in reducing losses in electricity is work in progress, figure 1 shows that the losses were on average lower during December June 2017, in comparison to the previous period of January 2016-November The performance is set to increase further when most of the project s effects (from new electrical lines and upgrading of substations) are used over a longer period. Toward the second objective, the unbundling of the energy sector is still recent, but the commercial outcome of EDSA has shown strong signals of performance. A higher level of revenue collection was spurred by increase in shift from post-paid to pre-paid billing and the launch of a strong revenue protection program and arrears recovery, as well as sustained commercial audit. The pursuit of the third objective was curtailed, owing to a situation of force majeure., and not to shortcomings related to project design, monitoring and evaluation (M&E) or 11

26 supervision. On balance, the project s efficacy toward stated objectives of the initial project is rated Substantial based on good progress toward the two first objectives, and an unfortunate event that cut short the pursuit of the third objective. Revised project (November 2015-July percent of disbursed resources): Rating: High 37. The substantive change brought in by the 2015 level 1 restructuring was the cancellation of the rural electrification component and the objective that was pursued, with light reallocation of resources and adjustments in target indicators. Project achievements after restructuring do not fundamentally differ from those accomplished before the restructuring. The key difference is that, while the absence of results in rural electrification downsized efficacy before restructuring, performance of the revised project is assessed based on two objectives. (a) Objective 1: To reduce losses in electricity supply in Freetown Capital Western Area: 38. New activities prompted by the restructuring and orientated toward this objective were limited, and consisted of technical assistance and advisory work towards the preparation of the Western Area Power Generation Project and overall improved sector performance, and increases in resettlement costs associated with the rehabilitation of the distribution network. Overall, key outputs and outcomes achieved under this objective are like those described under the same objective during the initial project. This was further illustrated by the maintaining key indicators with their targets for this objective before and after restructuring, with the exception of the energy access target. (b) Objective 2: To improve commercial performance of the National Power Authority: 39. New activities implemented toward this objective after the restructuring were also limited. Reallocated resources principally funded technical assistance work that strengthened the management, planning and commercial functions of EDSA, and the recruitment of the private Management Contractor. Overall, key outcomes achieved under this objective are comparable to those described under the same objective during the initial project. Key indicators were also maintained were maintained, except that with the upgrade of the distribution infrastructure and the planning and commercial functions of EDSA, the energy access target indicator was tripled and was exceeded at project closure. (c) Assessment of the performance of the revised project: 40. The achievements of the revised project under the two objectives were strong. Project efficacy is shown through the theory of change presented in table 2. The theory of change of the restructured project omits from this results chain the activities (Component 12

27 2) that were canceled during the level 1 restructuring in 2015, while those consequent to project resources reallocation were added, including their implicit outputs and outcomes. 41. The two-key notable additional outcomes reinforced under the revised project were (a) the increased level of energy access and (b) institutional strengthening of EDSA owing to technical assistance to boost its management, planning and commercial functions. This additional performance in these two areas is attributable to the reallocation of resources freed by the cancellation of the rural electrification component. Table 2: Results and Theory of Change of the Revised Project INPUTS/ ACTIVITIES OUTPUTS OUTCOMES (A) Objective 1: Reduce losses in electricity supply in Freetown Capital Western Area (i) Rehabilitation of critical components of Freetown distribution network, (ii) Supply and installation of a bulk metering (iii) Installation of power lines (i) The Freetown 161kV Kingtom substation was rehabilitated and upgraded. (ii) The Wilberforce, Blackhall Road and Wellington substations were rehabilitated and upgraded. (iii) A new Ropoti substation was created and upgraded; (v) Transformers, switchgears, and related materials and accessories for Blackhall Road substation were supplied; (v) Replacement of the 7.8km power line from Blackhall Road substation to Wellington substation via Ropoti substation; (i) Restoration of power at 33kV, thus increasing the evacuation capacity of the network and reducing associated technical losses; (ii) Existing line operating at 11kV replaced by the construction of a 7.8km 33kV line; (iii) Improved reliability of the network and reduction of technical losses. (vi) 100 bulk meters installed at 8 locations in Freetown. (B) Objective 2: Improve commercial performance of the National Power Utility (i) Supply and installation of prepaid meters (ii) Supply and installation of the revenue management system (iii) Preparation of the business plan for the electricity distribution utility (i) 20,000 meters were installed in 20 neighborhoods of Freetown. (ii) A Factory Acceptance Test was conducted by a consultant firm, and two EDSA appointed staff were trained. (iii) A revenue management system with its backup, software management application, power banks (UPS) for main and backup power, and HMI was set up and is operational. (iv) A business plan for the national electricity distribution utility was completed, including resources (i) Higher number of beneficiaries connected to the electricity grid, including females. (ii) Improved revenue collection rates and debt repayment, but the scope of the achieved results was affected by modest funding. (iii) Electricity utility unbundling completed; and (iv) Institutional strengthening of EDSA.. 13

28 (iv) Preparation of the Electricity Network Investment Plan (v) Preparation of the transaction advisory and tariff study (vi) Preparation of the bidding document for the Management Contractor for EDSA Performance toward high level objectives management, commercial enhancement, and technical and operations modules. (v) Completion of the Electricity Network Investment Plan drawing priorities for the Greater Freetown area and the rest of the country. (vi) Completion of a transaction advisory and tariff study. (vii) Completion of bidding document for the selection of an EDSA Management Contractor. (viii) Supply of office equipment, vehicles, and software to the PMU, which was moved to work under EDSA after project completion. On balance, project achievements toward high level objectives was high. Source: Information collected from the Implementation Status and Results Reports ISRs and the PMU C. Efficiency Rating: Modest 42. Economic and financial efficiency. An ex-post economic and financial analysis of the project was carried out to update the project s financial flows in comparison to the scenario presented in the PAD. The economic analysis looked at the costs and benefits accruing to the urban households benefiting from connections to rehabilitated electricity infrastructure, and the additional equipment funded by the project to improve efficiency in the use of electricity. The high economic returns of the project s investments in rehabilitated electricity plants, and the savings arising from the pre-paid meters and the loss reduction mechanisms brought in by the project allowed the generation of a net present value NPV of about US$ 2.46 million and an economic internal rate of return (EIRR) of 14 percent over the 20-year period during which the main acquired infrastructure will last. 43. Benefits. The quantifiable benefits obtained from the different investments include the following: (a) the benefits from improved distribution (P*Q) and pre-paid meters, and (b) savings from the technical and non-technical losses reduction consecutive to the project investments. 44. Costs. The main costs of the different infrastructures include the investment costs and the costs for operation and maintenance (O&M). Additional costs specific to the subprojects were included under the specific analyses of the sub-components. A detailed account of the calculations of costs and benefits of the diverse categories of infrastructures are provided in annex 3. 14

29 45. Results. Table 3 summarizes the results of the ex-post economic analysis of the project. The NPV of economic benefits during a 20-year life of the project is US$2.46 million. The EIRR is 14 percent. The economic returns are moderate, since the investment return from the loss reduction programs is relatively small. The financial analysis of the project reveals that the project is financially viable with an internal rate of return (IRR) of 33 percent in the base case and an NPV of US$23.43 million. There are additional direct and indirect benefits from connections to electricity which are difficult to estimate, such as improvements in education, health, communication and productivity. Table 3: Summary of Economic and Financial Analysis ICR Results Appraisal Results NPV (US$, EIRR NPV (US$ ERR millions) millions) Economic Analysis % % Financial Analysis % % Source: World Bank staff calculations. Note: ICR = Implementation Completions and Results Report. 46. Operational and administrative efficiency. Because of the fragile environment context, the project delivery suffered from the Ebola outbreak when its implementation was at its climax. Project implementation before the Ebola outbreak was delayed, because the PMU had limited capacity in completing procurement and safeguards requirements to disburse and acquire goods and services. Activities related to pre-paid meters, and energy loss reduction were launched, and some of them were swiftly completed. The project came almost to a halt when the Ebola crisis hit to the extent that supervision missions were at times relocated to Abidjan, making it impossible for the World Bank s team to have a realistic assessment of the shortcomings on the ground. 47. There were implementation delays stemming from the lack of project implementation readiness. Deterioration of health conditions brought the project almost to a halt, affecting more specifically Component 1 related to rehabilitation of substations, because the procurement related to Component 1 were delayed. The operational efficiency of the project is rated Modest, mainly because of the delays in launching the project, due to weaknesses in the PMU and the consequences of the health deterioration on the ground. Overall efficiency is also rated Modest. D. Outcome Attribution 48. The achieved project s outcome embodies a major contribution from other key stakeholders in the country s energy sector. The above-described project outcomes were achieved because of the activities funded by this project, but also because of parallel support and funding provided by ongoing Bank projects and other donors, including inputs by the unbundled energy utilities, especially EDSA. In all, table 4 shows that ongoing support from other stakeholders who have contributed to the level of achieved outcome. The performance reflected by the key indicators of loss reduction and collection rates could 15

30 not be achieved without the role of the unbundling reform that led to the creation of EDSA, or the support provided by JICA, and other external donors. Table 4: Stakeholders Active in The Sierra Leone Energy Sector Partner Purpose Amount World Bank /EAP Supply of pre-paid meters Upgrade and construction of 33 kv primary substations Delivery of distribution lines Delivery of revenue management system and network meters US$16 million World Bank/ ESURP Japanese International Corporation Agency ECOWAS/WAPP Islamic Development Bank MCC Energy Sector Utility Reform Project - Network expansion and rehabilitation Construction of a new 33 kv primary substation at Goderich and 33kV distribution line between Wilberforce and Goderich The distribution component includes upgrading of network and extension of electricity to unserved communities, and the supply of pre-paid meters (approximately 8,000 meters) Rehabilitation and strengthening of medium and low voltage network Electricity Sector Reform Project (ESRP) with two components: (a) Roadmap and coordination activity and (b) the institutional strengthening activity US$40 million US$14 million US$21.8 million US$10.4 million NA AfDB Support to the energy transmission and distribution network NA EDSA Provision of 500 AMR meters for an Advance Metering Infrastructure pilot, being implemented by PEC SL (South Africa) NA Source: NRECA International Ltd, July 2016 E. Justification of the overall outcome rating Original project: 49. The relevance of objectives is high for the original objective. Efficacy of the initial project is rated Substantial, while efficiency is Modest. The overall outcome rating for the original project is Moderately Satisfactory. Revised Project: 50. The relevance of objectives for the revised project is also high. Efficacy of the revised project is rated High, while efficiency is modest. The overall outcome rating for the revised project is Moderately Satisfactory. Assessment of Overall Performance 51. Applying the World Bank formula for restructured projects, (4 X 0.40) + (4 x.0.60) = 4.00, the overall outcome rating for the Energy Access Project is Moderately Satisfactory as shown in table 5. This rating summarizes the weighted performance of the project before and after restructuring, and goes beyond the achievement of expected of outputs and outcome indicators. It takes into account the contribution of this project to the 16

31 stated objectives, including also the contribution of EDSA, other World Bank projects, and other donors to the same objectives through their respective interventions. While project results contributed to the achieved outcome and objectives, other World Bank projects and other donors supported the energy sector and similar activities (see paragraph 48, table 4), and contributed to the achievement of the same objectives Table 5: Weighted Project Performance Against original PDOs Against Revised PDOs Overall Rating for MS 4 4 Weight (% disbursed before/% disbursed after 40% of US$16.0 million 60% of US$16.00 million Weighted value Final Rating - - Moderately Satisfactory Comments Cancellation of the rural electrification component affected the performance of the initial project, while reallocated resources enhanced the performance of the revised project that remained with two objectives. F. Other Outcomes and Impacts 52. Gender: The immediate gender impact of the project is that 51 percent of the new beneficiaries of energy are female, and an estimated 360,000 females have access to energy. Other gender impacts attributable to the project have not been assessed yet. 53. Institutional strengthening. The project has had positive institutional effects on the implementing agencies, namely the MoE and EDSA. The project was instrumental in supporting EDSA, by funding the advisory work and the technical assistance leading to the hiring of a Management Contractor under the Energy Sector Utility Reforms Project (ESURP) to oversee improvement of EDSA s operational and finance performance. The Network Investment Plan financed by the project has laid the foundation for reinforcing the network and further expansion which will also be implemented under the ESURP. Contribution to the institutional strengthening in the Government was minimum, because the PMU hired independent consultants to support project implementation, who are not bound to remain with the public sector after the project closure. 54. Other positive and unintended impacts. Following the completion of the project, there are expected direct or indirect effects involving the private sector business that will be spurred by the availability of electricity in the new neighborhoods, thus contributing to poverty reduction and shared prosperity. However, there have been, so far, no surveys to assess the scope of that impact. 17

32 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME 3.1 Key Factors during Preparation 55. The project design reflected the challenges in the country s energy sector, and the willingness of two donors (the World Bank and DFID) to support the Government s sector reform program. The sustainability of the country s entities involved in the generation and transmission of electricity was central to economic growth and diversification, and a condition for the Government to improve public services to people in the future. The Government did not have the resources and the expertise necessary to undertake the appropriate diagnostic of the energy sector, and to chart out a reform program that could improve its efficacy and efficiency. This is the context that underpinned the dialogue between the Government on one side, and the World Bank and DFID on the other side, and which resulted in the design of this project. This project was a consensual undertaking between the Government and the two donors, and it was a first phase of a long-term program to reimagine the development of the country s energy sector. 56. The project design incorporated the lessons learned from the World Bank s previous projects implemented in the country s power and water sectors, and the dialogue that surrounded their design and delivery. The project had well-perceived goals of improving the efficiency of the energy utility through the reduction of losses, and the launching of rural electrification. However, the second objective was postponed following the Ebola outbreak in 2015 and DFID s option to support identified activities through another financial instrument. Overall, the key goal of the project was concentrated around stepping up the technical and commercial performance of the energy utility. The project objectives were not ambitious and the envisioned outcomes were realistic, and drew from the dialogue that brought together key stakeholders supporting the energy sector, including the Government, its public entities, and the external donors. Identified activities were the Government s priorities in the sector, and their costing was sound, including earmarked contingencies to cater to variations in the quantities and prices of goods and services. 57. The design of implementation and M&E arrangements drew from the World Bank s interventions in the country s water and energy sectors. The project design drew from the World Bank s experience in implementing the Power and Water Project (P087203) and the Bumbuna Project (P086801), and the field experience accumulated during the implementation of these two closed projects. The choice of the implementing agency, and the selection of M&E indicators drew from the experience of implementing of the above projects. An already existing PMU in the MoE was maintained to take up the supervision of the project, and attention was stressed to make sure that procurement, FM, and safeguards functions are dealt with carefully, as they underperformed during implementation of previous projects. 58. Risks were assessed as high at appraisal, because of high country risks and the crippling weaknesses in the project implementing agency. The fragile country context, the weak sector regulatory institutions, and the systemic sector governance 18

33 challenges were all factors that could potentially jeopardize the project implementation. Because of weak capacity in the electricity utility and the MoE, project implementation was entrusted to the existing PMU established at the ministerial level, based on prior experience from power sector projects that showed that low implementation capacity had increased the risk of hiring unqualified firms for carrying out fiduciary management tasks. To mitigate implementation risks, the option adopted consisted of hiring a group of individual consultants for key PMU functions and to ensure that most of procurement will be conducted through international competitive means, and to enhanced supervision through site visits to allow for immediate mitigation of issues that might be identified during implementation. Table 6 summarizes developments related to identified and nonidentified risks and the extent to which they were mitigated. Table 6: Developments Related to Identified and Unforeseen Project Risks Identified Risks Country-level risks: Given the fragile context, emergencies might occur, shifting the Government s focus to other development priorities. Fiduciary and governance risks: Implementing agencies had weak FM and procurement capacity, including weak internal controls and audit systems that may affect implementation and oversight on the project and long-term outcomes. Mitigation Actions Adopted or Consequences of Absence of Remedy Both the Government and the electricity utility (NPA/EDSA) remained committed to the project objectives and helped to address implementation obstacles throughout the project life. The risks materialized and there were implementation delays due to procurement weaknesses and red tape in processing the procurement and delivery of goods and in hiring required expertise. Procurement of goods was slow, because procurement was not staffed properly in numbers and quality. Implementation capacity risks: Both the electricity utility and the MoE had weak institutional and project management capacity to effectively implement the project. Unforeseen risks: The Ebola outbreak in 2015 triggered the revision of project objectives, components and implementation. The risks materialized, and were mitigated by through the fielding of experienced international expertise to support the PMU ( including the supervising Engineer). The Ebola outbreak brought the project implementation to a halt, triggered the cancellation of the rural electrification component, and shifted the project focus to upgrading the substations, and strengthening the PMU. 3.2 Key Factors during Implementation 59. The reform to unbundle the energy utility and the sector support by external donors were positive factors for the project implementation. Among other changes in the sector, the unbundling reform led to the setup of EDSA and to putting it under private management. The reform has created a conducive environment for changes in consumer behavior, including the move from post-paid toward pre-paid billing. Moreover, the presence of other donors supporting the reforms in the energy sector has created a momentum of change in the energy sector, thus helping in the launching of measures and actions supported by the project, notably revenue protection programs to increase revenue collection. 19

34 60. There was a slow start to the project implementation, owing to weak capacity in the implementing agency with regard to key skills required to effectively run the project. More than a year after project approval, the key PMU positions of FM and procurement specialists were not filled yet, leading to difficulties and delays in the launching of project activities, including the procuring of pre-paid meters and the associated revenue management system. Despite some progress over time, the World Bank team remained concerned until late in the project implementation regarding protracted lags in procurement, weak knowledge of and compliance with World Bank fiduciary requirements and the lack of coordination and communication among the key project stakeholders. The PMU was demonstrating weak planning and oversight of deliverables and insufficient command of the requirements related to procurement and safeguards as well as stronger coordination with and support by the staff of the distribution utility. The observed weaknesses posed at the time a risk to timely achievement of the project s expected results. 61. The outbreak of the EVD in 2014 annihilated the thrust of implementation of the key project activities. The implementation of the network rehabilitation component that had started in the meantime, including the implementation of the Environmental and Social Management Plan and the relocation of Project Affected Persons suffered significant delay, because of the outbreak of EVD. Following the inability of the supervision mission to travel to the country because of the EVD outbreak, meetings among the World Bank s team and the PMU representatives involved in project implementation took place in Abidjan during to assess progress on the ground, and to agree on which steps could be taken despite the ground difficulties. 62. A change in the priorities and the support nature of a key donor (DFID) led to a level 1 restructuring that affected the project configuration and the expected outputs and outcomes. Following the delays and implementation disruptions that arose from the EVD, some activities under Component 1, and Component 2 had fallen behind their implementation schedule or were facing execution obstacles. While the contract for the rural electrification for prioritized rural villages had already been prepared, the bidding process was put on hold due to EVD as no disbursements for Component 2 had taken place. Similarly, the BIS activity under Component 1 failed to attract bidders. In the instance, DFID had indicated that it wished to exclude the pilot program for photovoltaic systems in rural villages under the project and pursue a more comprehensive approach to rural electrification under a separate engagement. All these developments put together led to a level 1 restructuring in October 2015 that comprised the following: (a) a change in the PDO, (b) the cancelation of the second component related to rural electrification, (c) the elimination of the BIS activity under Component 1, so that it can be part of a larger procurement package under a new project, (d) the reallocation of resources, and (e) the extension of the closing date of the project until April 30, PMU weaknesses in addressing procurement and environment safeguards issues and delays provoked by the EVD led to the extension of the project closing date. When the country was declared Ebola free in October 2015, the implementation of key components resumed, although the general weak capacity of the PMU combined with the 20

35 lack of incentives to process contracts was the key obstacle to project progress toward completion. By the end of 2016, only the civil works of the Freetown Upgrade contract, and its associated compensation and relocation of project affected people was still under way, while all other activities not cancelled during restructuring were completed in the meantime. To finalize the works related to the Freetown Upgrade network rehabilitation and the associated Resettlement Action Plan (RAP), stakeholders agreed to extend the project execution until July 31, IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. Quality of Monitoring and Evaluation Monitoring and Evaluation (M&E) Design, Implementation and Utilization Rating: Moderately Satisfactory 64. M&E Design. A Results and Monitoring Framework was negotiated and agreed upon among the PMU, the electricity utility (NPA) at the time, and the Ministry of Energy and Water Resources (annex 1 of the PAD). The framework identified result indicators for the project and for each of its components. The framework had annual target values for the results indicators and baseline data against which results were to be measured. Most data were to be provided by the NPA from the utility database and accounts, or collected through direct observation, or from technical reports. The PMU was responsible for collecting and verifying the data, and submitting progress reports to the World Bank, annually for PDO indicators and semi-annually for the intermediate indicators. 65. M&E Implementation: There were 7 filed ISRs, and all of them provided the progress achieved toward the results indicators. Following the level 1 restructuring of the project in October 2013, the Result Framework was revised to reflect the change in the project objective and components. The midterm review (MTR) exercise could not take place, because it coincided with the outbreak of the Ebola crisis. When the Ebola crisis subsided, the project team concentrated on resolving operational issues of the project to ensure that the retained components are completed. 66. M&E Utilization: The data and information provided by the ISRs were used to monitor progress in the project implementation, but the Ebola outbreak and the cancelation of the MTR were factors that reduced the possibility of collecting M&E information, and using it to spearhead the project implementation. 67. Overall, M&E was mostly rated Satisfactory in the ISRs from the outset until April 2016 when it was downgraded to Moderately Satisfactory to reflect delays in reporting from the PMU and incomplete progress reports. This review concurs with the Moderately Satisfactory rating in the last ISR. 21

36 B. Environmental, Social and Fiduciary Compliance Social and Safeguard Compliance: 68. The project was classified under the safeguards category B, and two safeguards policies were triggered by the project: (a) Environment Assessment OP/BP 4.01, and (b) Involuntary Resettlement OP/BP An environmental and Social Impact Assessment and an Environment and Social Management Plan were prepared to ensure that adequate mitigation measures are implemented. Safeguards preparation and compensation payment started of satisfactorily but deteriorated to Moderately Unsatisfactory in the last months to project closure. The project at start impacted several project-affected persons. The RAP was implemented satisfactorily with only two outstanding cash compensation (totaling about 1000 USD) largely attributed to procedural bottlenecks. One project-affected person was not available during asset enumeration and negotiation for compensation; the other involved court litigation among multiple claimants. The mechanism involved a four-tier system of resolving conflicts at the community level under the ward councilors (first-tier) through the two management committees, the RAP Implementation Committee and the Intermenstrual Sub-Committee (second and third-tier respectively), and the final stage where the complainant has the option to seek redress in a court of law (fourth-tier). The PMU at implementation further paid additional compensation to PAPs due to omission or requiring additional land as well as from inconveniencies to a project affected person (PAP) following prolonged implementation of the project. About 65 to-70 PAP participated in a Livelihood Restoration Program. Seedlings were provided to all participants for vegetable gardening. Training on health and sanitation was concluded and 68 hand washing basins, hand sanitizers and soaps were distributed to PAPs in the wake of Ebola crisis. 69. During implementation, the project suffered from safeguards challenges. Dismantling and stringing of towers resulted in additional damage to properties. The project by design in a heavily congested area was a factor. It is worth noting however that, the design followed the existing right of way from the plans under the previous Sierra Leone Power and Water Project. There was minimal buffer for work space and environmental and social safeguards staffing had weakened. The client s safeguards specialist resigned and it took a while to replace her. The Ebola crisis also affected supervision. Above all, there was lack of speedy responsiveness by British Engineering Services, the contractor on the ground, in adhering to very basic safety precautions to minimize the impacts. There was minimal progress by the contractor following several safeguards missions and recommendations to fix the issues. Upon escalation to the Sierra Leone Country Management Unit, Energy Minister and Minister of Finance, disbursement was subsequently suspended for a while to force compliance. 70. Compensation has been paid to PAP who was absent during enumeration. Regarding the other outstanding payment, the Ministry in charge of energy will, and expects to use interest earnings on project funds to, offset the compensation when the court case is resolved. Contractor has resolved all compensations to cover damages during construction. Any outstanding issues that is not contractual obligation to the contractor will be addressed under the ESURP. 22

37 71. Overall rating of Moderately Satisfactory is conferred as the project is in compliance with social and environment safeguards at the time of finalizing ICR due to efforts made by the Bank s team, PMU and Government to resolve issues during implementation and after project closure. The project s safeguards rating was always satisfactory except for 3 months in the later stages of the project when it deteriorated. A positive note is that any issues that may arise will be dealt with in the context of ESURP which is building on project s achieved outcome. Fiduciary Compliance: 72. FM. The risk assessment of the PMU FM was moderate at approval, as existing FM arrangements of the PMU satisfied the World Bank s minimum requirements. A set of measures and an action plan were identified to reduce the FM risk and ensure that FM systems provide accurate and timely information on the status of the project. FM was rated Satisfactory throughout the project implementation period, and no irregularities were reported. However, an external audit will be conducted after the project closure to conclude on the FM performance of the project, and this is due at end of January Procurement: The PMU had weak procurement capacity and benefitted from the contribution of an internationally hired procurement specialist, and some strengthening occurred with the recruitment of a procurement assistant. However, the contract of the international procurement specialist was not extended leading once again to a decrease in the overall procurement capacity of the PMU. Additional support was provided by the World Bank through the energy specialist deployed to Sierra Leone, but there was no international hiring as the Government was not favorable to replacing the procurement specialist. There was a near mis-procurement on one of the contracts, because of incomplete bid documents, and major procurement activities were put on hold until the situation was later resolved. Therefore, the procurement performance was rated Moderately Satisfactory in the ISRs throughout the period of project implementation to reflect the procurement shortcomings described above. This review concurs with the Moderately Satisfactory rating. C. Bank Performance Rating: Moderately Satisfactory (a) Bank Performance in Ensuring Quality at Entry 74. The World Bank tapped the findings from its previous interventions in fragile environments and designed a simple project, but the project had weak M&E and implementing arrangements. Some of the World Bank s findings and lessons with regards to emergency operations were factored in the project s design as follows: (a) the scale and complexity of emergency projects have to ensure that the borrower has the capacity to manage the projects and absorb the lessons arising from them and (b) World Bank supervision teams needed to have the necessary range of expertise to deal effectively 23

38 with the key project components. As such, the project s proposed scope was realistic and simple, but implementation arrangements involved entities that had weak experience in the type of activities that were proposed. 75. The World Bank used its convening and management power to implement a project funded by an external partner (DFID): The resources used to fund the project were provided by DFID, and showed the extent to which the World Bank enabled a synergy between its convening and management capabilities and that of other donors to support the energy sector. 76. While the overall risk was assessed as high, mitigating measures were correctly identified, but the Ebola outbreak annihilated all mitigating mechanisms at the country level. Some mitigating measures at operation level worked and rescued the project. At the country level, the main risks that could jeopardize the project as identified in the PAD, included the implementing agency risk and the delivery monitoring and sustainability. Unfortunately, all mitigating measures were blown away by the overwhelming nature of the Ebola crisis that hit the country during , leading to a slow-down and then a halt in the project implementation. (b) Quality of Supervision 77. There were delays in launching the project, because of low implementation readiness, and weak technical capacity of the PMU. The procurement of equipment to upgrade the substations was problematic in the beginning, because the PMU was not staffed to handle the operation, and the Ebola crisis worsened the context making the close of the operation even more difficult. Throughout the project implementation until the project closure, there were weaknesses in procurement, FM reporting, and supervision of social and safeguards aspects of the project. Implementation was accelerated only when a resident World Bank task team leader was brought in after the Ebola crisis subsided. 78. While the overall risk was assessed as high, mitigating measures were correctly identified, but the Ebola outbreak annihilated all mitigating mechanisms at the country level. At the country level, the main risks that could jeopardize the project as identified in the PAD, included the deterioration of the macro-economic, security and political environment. While the above risks were under control, unfortunately, all mitigating measures were blown away by the overwhelming nature of the Ebola crisis that brought the project implementation to a halt. 79. While the Ebola crisis brought the project implementation to a halt, World Bank supervision remained resilient. The Ebola pandemics that hit the country during the period brought the project implementation to a halt, and made supervision meetings bringing together the World Bank staff and the representatives of implementing agencies difficult. During the above period, project activities were either stopped or slowed down, because of dire health conditions on the ground. When the country was declared Ebola free in 2015, the World Bank, DFID, and the Government agreed for the 2015 project restructuring, to continue the project implementation with a less ambitious agenda and 24

39 update the Result Framework. The new operational adjustments supported the continuation of project implementation with improved efficacy. (c) Justification of Rating for Overall World Bank Performance 80. Implementation arrangements could have been improved. Supervision was affected by weak capacity in the PMU and the Ebola crisis, but the World Bank s mitigation measures through a project restructuring and fielding of country-based staff with the relevant experience came in with a delay. Overall, World Bank performance is rated Moderately Satisfactory. D. Risk to development outcome 81. The borrower committed to follow up on pending issues identified during the ICR mission. Three weak spots were identified during the ICR mission in September 2017 as follows: (a) there were concerns that installation of the equipment delivered at the Blackhall Road substation could be delayed or forgotten, (b) there were social and safeguards issues related to the electrical line linking Blackhall Road substation, and Wellington substation, which were not settled at project closure, and (c) capacity building in the borrower s institutions was not dealt with correctly during project implementation. The borrower committed to follow up and act on those issues satisfactorily in the context of government initiatives and cooperation with external donors. 82. The World Bank s ongoing and future interventions will build on the achieved results in the energy sector. The World Bank is active on the ground through the ESURP and the Western Area Power Generation Project. The ESURP will take care of urgent pending activities, particularly the installation of the transformers and switchgears at the Blackhall Road substation, while both projects will continue to build on achieved results to make the country s energy sector more efficient. 83. The hiring of a private Management Contractor to provide technical assistance in the oversight of EDSA s performance might contribute to improved outcomes in the country s energy sector. Bringing private management ethics in the management of energy distribution might improve commercial performance, and the energy sector as a whole. Similar approaches have worked under different circumstances. If the World Bank and other donors maintain the momentum of reforms and support to the energy sector, the outcome achieved under this project could be sustained. V. LESSONS LEARNED 84. Implementation readiness at project approval and adequate M&E are central to prompt project launching and effective supervision. There were delays in the launching of the project, because there was inadequate preparation of key documents such as bidding documents and the PMU was not properly staffed, staff rotation in key functions (coordination and procurement) was observed throughout the project life. Furthermore, 25

40 weaknesses in the PMU hampered the collection and dissemination of the information related to project implementation progress. Mitigation measures would include making sure that at project approval there is adequate technical capacity in the PMU to perform key functions of project management and M&E. In fragile environments, the World Bank team should assess more rigorously, existing technical capacity at the implementation agency during project preparation to facilitate efficient planning and provision of adequate technical assistance. 85. When strengthening technical capacity in the borrower s institutions becomes a challenge, one effective option is to bring in external expertise to move forward with project implementation. Two years after project effectiveness, the project was accumulating delays, because implementing agencies lacked the technical know-how to process procurement and disbursements. Fielding of a resident task team leader made a difference in the project implementation. At the project closure, weaknesses in the PMU and the borrower s institutions were still observable. Besides outsourcing expertise, it is important to devise better strategies such as foreign exchange program to enable staff from fragile countries train on similar Bank projects in other countries as part of ongoing reforms and that technical capacity is built in local institutions using the World Bank s resources before the project closes. 86. Progress in reforming the energy sector relies on balancing the necessary investments toward upgrading the energy infrastructure that can deliver good service to the client, and the scope of changes in the policies governing the energy sector. Some of the policy measures that drove the ongoing energy sector reform in Sierra Leone were the tariff increases, and the prepayment system in revenue collections. These difficult reforms were launched and conducted successfully and seem to generate good results and are generally supported by the public, because currently the unbundled electricity utilities (EDSA and EGTC) strive to make the service available in a sustained manner, owing to recent investments partly supported by the World Bank. 87. World Bank specialists need to get involved more in project design and implementation to improve compliance with social and environmental safeguards. Both the World Bank and the borrower put insufficient attention in ensuring that skilled staff oversee compliance with social and environmental safeguards from the outset of project preparation and during implementation such as interacting timely with contractors. Efforts to catch up the lost time yielded reduced results, with the poor bearing most of the burden of low-level compliance with social and environmental safeguards, which is contrary to the World Bank s motto of eradicating poverty. Involvement of safeguards experts in the preparation of the project would have provided advise on an alternative routing of transmission line and led to a simpler and practical design of infrastructure. 88. The World Bank management should consider increased support for countrybased staff if necessary: In fragile countries, the World Bank team is quick to secure financial resources, but has difficulties to disburse because of weaknesses in the PMU, particularly in handling procurement and disbursement. The World Bank s team can 26

41 achieve success if hands-on support from the World Bank management is provided, more so, if this is done in the early stages of the project. It is a fine line and can be very confusing for country based staff, who need encouragement and most importantly guidance from management to take on this calculated risk of intervening in a fragile environment. VI. COMMENTS ON ISSUES RAISED BY BORROWER/IMPLEMENTING AGENCIES/PARTNERS (a) Borrower/implementing agencies Annex 7 (b) Cofinanciers Annex 8 (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) Not Applicable 27

42 Annex 1. Project Costs and Financing (a) Project Cost by Component (US$ Millions Equivalent) P Sierra Leone Energy Access Project - Sierra Leone Infrastructure Development Fund- TF-13246/ TF Components Component 1: Rehabilitation of Primary Distribution Network, Loss Reduction and Improvement of NPA s Operational and Commercial Performance Component 2: Rural Electrification Component 3: Project Implementation Management Appraisal Estimate (US$ millions) Revised Estimate at Restructuring Actual/Latest Estimate (US$ millions) Percentage of Appraisal % % % Total Baseline Cost % Physical Contingencies Price Contingencies Total Project Costs % Total Financing Required % (b) Financing P Sierra Leone Energy Access Project - Sierra Leone Infrastructure Development Fund- TF-13246/ TF Source of Funds Type of Financin g Appraisal Estimate (US$ millions) Actual/Latest Estimate (US$ millions) Percentage of Appraisal Borrower SLIDF % SLIDF 0.75 Total Financing % 28

43 Annex 2. Outputs by Component Component 1: Rehabilitation of Primary Distribution Network, Loss Reduction and Improvement of EDSA's Operational and Commercial Performance: US$12.2 million at appraisal, actual cost of US$13.29 million (A) Investments (a) Rehabilitation of critical components of Freetown distribution network: Extension of the Freetown 161kV substation by inserting a new 40MVA 161/33kV transformer and creation of a four-ways outdoor 33kV switchgear including the control and protection system and substation data management system. Complete rehabilitation of the Wilberforce substation by providing a new 33 kv and 11 kv switchgear, a new 20MVA 33/11 kv transformer, and a station transformer and control and protection system; provision of AC/DC system and connection to the existing 33 kv and 11 kv panels. Complete Rehabilitation of Wellington Substation by providing a new 33 kv and 11 kv switchgear, a new 20 MVA 33/11 kv transformer, a new AC/DC system, and a station transformer and control and protection system; Creation of a new Ropoti substation with 33kV and 11kV switchgear, a new 20 MVA 33/11 kv transformer, a new AC/DC system, and a station transformer and control and protection system; Supply of a 20 MVA transformer, 33 kv and 11kV switchgear, station transformer, and all related materials and accessories for the Blackhall Road substation; Complete replacement of the 7.8km power line from Blackhall Road to Wellington via the Ropoti substation. The line was operated at 11kV and is now upgraded to 33 kv and constructed on double circuit towers. (b) Supply and Installation of network statistical meters Supply and installation of 100 number of bulk meters. These meters were installed at switching stations and Substations. The stations considered in this project are Freetown 161, Kingtom, Falcon Bridge, Brookfields, Congo Cross, Wilberforce, Regent, Blackhall Road and Wellington. The meters were also supplied with Data Management Software and training was carried out for EDSA engineers. (c) Supply and installation of pre-paid meters: Supply and installation of 17,800 single phase prepaid meters and 2,200 three phase meters. These meters were installed in the following neighborhoods of Freetown and 29

44 replaced the old meters: Kortright, Maeba Town, PWD, Ferry Junction, Cline Town, Foulah Town, Kissy Area in the East; Mountain Cut, New England, Freetown Central in Central; and Tengbeh Town, Congo Town, Signal Hill, Wilberforce, Hill Station, Spour Road, Babadorie, Lumley, Juba Hill and Goderich. To ensure the adequacy of operation of meters and compatibility of meters with the existing meter management system, the PMU hired the services of Factory Acceptance Test consultancy services to carry out all tests required, train two EDSA appointed staff and prepare a complete factory tests report. (d) Supply and Installation of the revenue management system: Supply and Installation of complete revenue management system to back up the existing one which was old and unable to host an additional number of customers. The new system consists of a Data Management System with its backup, software management application, power banks (UPS) for main and backup power, and HMI. The vending system consists of 20 vending machines equipped with their software and data management system. The system was installed at Electricity House and the backup system was installed at Blackhall Road. The contract also covered the training of EDSA staff to use and get familiar with the system. (e) Compensation under the RAP 107 affected persons identified during project preparation and pre-construction stages of the construction the electric line were compensated (B) Technical Assistance. (a) Preparation of the business plan for the national electricity distribution utility. The prepared plan has three segments as described below: (i) The Management of Corporate Resources Module. This portion of the plan describes the operational procedures for management of corporate resources, and provides for a new organizational structure and identification of procedures to achieve high efficiency in management of corporate resources. An action plan ensures a smooth transition within a three-year period from the existing practices to full implementation of the new structure and procedures. (ii) The Management of Commercial Functions Module. The plan describes an organizational structure and related operational procedures for the development of commercial functions, including the management of the commercial cycle, and service orders, focusing on sales to large customers, attention to customers at customer service centers or by phone, on-line management of the customers database, energy balance and other commercial processes and activities. It also describes the development of commercial functions, with the support of the state- 30

45 of-art Commercial Management System and revenue protection programs to be incorporated by the utility. (iii) The Management of Network Planning and Operations Module. This component of the plan describes the organizational structure and related operational procedures for network planning and O&M of the energy utility, with focus on procedures for management of incidents in electricity supply (customers complaints, failures in networks), in particular in restoring normal electricity supply after incidents and customers claims. It provides also for specification of information systems and other tools to support network planning and operations related to response to and resolution of customers claims. (b) Preparation of the Electricity Network Investment Plan: This Five-Year Plan defines (i) the emergency investments to be carried out in the immediate term to address critical network rehabilitation needs; (ii) prioritized investments aimed at improving network performance in the medium term; and (iii) a logical direction for undertaking future grid development activities for Sierra Leone during System Investment Plan for the Greater Freetown area: A consulting firm conducted extensive inspection of the electricity supply system of the Greater Freetown area, studied information, and proposed a network rehabilitation and expansion projects, and developed the Five-year System Investment Plan for the Greater Freetown area. The plan prepared (i) priced schedules of requirements for implementing the distribution-level works identified under activities above as well as the grid-level works, (ii) Single Line Diagrams and geographic layout drawings, in electronic format, and procurement packages under which the System Investment Plan for the Greater Freetown area would be implemented, and (iii) the whole set of bidding documents for the procurement of goods. Electricity Network Investment Plan for the country. The consulting firm prepared (i) a base case electricity demand forecast based on the expected evolution of consumption by existing and prospective users, and (ii) a model to analyze historic energy demand by major subsectors and tariff classes and to produce a range of load forecasts using an appropriate set of variables. The model developed (i) a forecast of base-load, peak and intermediate power demand, with associated load shapes, for , and (ii) a country-wide grid design considering, among other things, the proposed interconnection project under the West African Power Pool 225kV project and existing 161kV and 33kV infrastructure. Finally, the firm carried out technical feasibility analysis of the grid design, made lump-sum estimates by geographical area for the distribution-level investments required to meet predicted levels of demand, customer population, prepared a priced schedule of investment requirements, and prepared Single Line Diagram and geographic layout drawings for the design developed. 31

46 (c) Transaction advisory and tariff study. The study developed a Revenue Enhancement Program, as well as a Power Sector Cash Flow Analysis. The study set out a sector cash flow model and analyzed the impact of a series of variables on the cash flows. The main themes covered under this study included: (i) a Revenue Protection Program, (ii) Reducing Commercial Losses and Electricity Theft, (iii) Automated/Advanced Metering Infrastructure, (iv) Improving Collection Rates, (v) Metering and Energy Audit, (vi) Billing Record Enhancement, (vii) Improving Customer Services and Customer Satisfaction, and (viii) Reducing Technical Losses, Loss Reduction Forecast, and EDSA Cash Flow Analysis. (d) Preparation of the bidding document of Management Contractor. A private firm was hired to draft the bidding document to select the EDSA Management Contractor. The exercise led to the selection of the Lebanese firm Khatib & Alami as the EDSA Management Contractor. (e) Environment and social management i. Safeguards policies triggered by the project were Environment Assessment OP/BP 4.01, and Involuntary Assessment OP/BP ii. Implementation of the RAP under the Energy Access Project included the implementation of an Abbreviated Resettlement Action Plan to mitigate the impacts of the project on project affected persons. The PMU involved all stakeholders and used door-to-door sensitizations, meetings with individuals, groups, communities, relevant line ministries and departments, and the civil society. iii. The PMU addressed compensation issues that were directly within its mandate such as additional compensation due to omission or requiring additional land as well as from inconveniencies to project-affected persons due to prolonged implementation of the project. The mechanism involved a four-tier system of resolving conflicts at the community level under the ward councilors (first-tier) through the two management committees, that is, the RAP- Implementation Committee and the Intermenstrual Sub-Committee (second and third-tier respectively), and the final stage, where the complainant has the option to seek redress in a court of law (fourth-tier). iv. There were issues that were still unresolved at project closure: late signature of negotiations with project-affected person who was outside the country which was submitted to the project after 31st July, 2017 and ongoing court dispute among claimants of a property. In addition, 15 new complains have been received after the contractor had compensated 92 properties affected during installation of conductors. 32

47 Component 2: Rural Electrification: US$1.46 million at appraisal, nil actual cost Due to the Ebola epidemic outbreak, DFID and the Government of Sierra Leone agreed to cancel this component and reallocate the funds to Components 1 and 3. Resources added to Component 1 helped improve EDSA operations by adding several 11 kv and 33 kv panels at the Blackhall Road substation, and to undertake additional sector studies as detailed under Component 1. Resources added to the component 3 helped fund external expertise to support the PMU, and acquire office equipment, vehicles, and software packages for the PMU, which had to be moved to EDSA and continue to support the implementation of the World Bank s ESURP. Component 3: Project Implementation Management: US$2.12 million at appraisal, actual cost of US$2.4 million. The PMU closely coordinated with EDSA and the MoE, as the two implementing agencies for the project. The PMU Director ensured general oversight of the project and effective coordination between the PMU and the project agencies. The PMU Director delegated day-to-day oversight to the General Project Coordinator. The PMU hired external expertise, including (i) a supervising engineer, (ii) a project coordinator for rural electrification, (iii) a procurement specialist and procurement assistant; (iv) FM specialist and FM assistants; (v) an environmental and social development specialist; (vi) a public relations/communications specialist, and (vii) a project secretary The PMU Director/Permanent Secretary of the MoE held regular monthly meetings with the members of the Project Oversight Committee to review project progress and discuss any strategic issues that might need the support of the Oversight Committee. The PMU Director also participated in the meetings of the Steering Committee overseeing the SLIDF work program, to provide briefings on project progress. Project task team leader reported a deficit of performance in some of the experts (procurement and safeguards), explaining staff rotation and delays in project implementation. Moreover, capacity building targeting staff in the PMU and in the implementing agencies did not occur. 33

48 Annex 3. Economic and Financial Analysis (A) Project Economic Analysis at Appraisal: At appraisal, the economic analysis focused on assessing the benefits deriving from loss reduction because of the rehabilitation of the distribution system and collection improvements. Net economic benefits were measured by estimating the changes in consumer surplus. The NPV of economic benefits to a 20-year life was estimated at US$26.7 million, with an ERR of 39 percent. The economic returns were robust as the hurdle rate was reached already in year 5 of operational benefits, and in year 10 reaches 36.8 percent. An economic analysis of the rural electrification component was not carried out due to the lack of survey information from which to derive credible and reliable quantitative estimates of economic benefits. In the perspective of the energy utility, the financial analysis of the project indicated that the project was financially viable with an IRR of 31 percent in the base case and a NPV of US$10.1 million. Component I had an IRR of 36 percent and a NPV of US$11.3 million. Conversely, the rural electrification component had a negative IRR and a NPV of minus US$1.2 million, as customers will only pay for the O&M costs of the solar panels. (B) Project Economic and Financial Analysis at Project Completion At project completion, economic analysis refers to investments supported by the project aimed at technical and commercial loss reduction, and those aimed at better demand side management. There were no investments related to rural electrification, because the Ebola outbreak led to the cancellation of the component devoted to its financing. Key investments supported by the project (i) Investments toward technical loss reduction: The following investments targeting technical loss reduction were completed at the project closure: Extension of the Freetown 161kV substation and creation of a four-ways outdoor 33kV switchgear including the control and protection system and substation data management system; Rehabilitation of the Wilberforce substation by providing new 33 kv and 11 kv switchgear, a new 20MVA 33/11kV transformer, and a station transformer and control and protection system; and provision of AC/DC system and connection to the existing 33 kv and 11 kv panels; Rehabilitation of the Wellington substation by providing new 33 kv and 11 kv switchgear, a new 20 MVA 33/11 kv transformer, a new AC/DC system, and a station transformer and control and protection system; Creation of a new Ropoti substation with 33kV and 11kV switchgear, a new 20 MVA 33/11 kv transformer, an AC/DC system, and a station transformer and control and protection system; 34

49 Supply of a 20 MVA transformer, 33 kv and 11 kv switchgear, station transformer, and all related materials and accessories for the Blackhall Road substation; Replacement of the 7.8 km power line from Blackhall Road to Wellington via the Ropoti substation. The line was operated at 11 kv and is now upgraded to 33 kv and constructed on double circuit towers Supply and installation of 100 bulk meters installed at 9 switching stations and substations. (ii) (iii) Investments toward commercial loss reduction: The following investments targeting improvement of commercial performance were completed: Supply and installation of 17,800 single-phase prepaid meters and 2,200 threephase meters. These meters were installed in 20 neighborhoods of Freetown and replaced the old meters. Investments toward demand side management. The main investment consisted in supplying and installing a complete revenue management system to back up the existing one which was old and unable to host an additional number of customers. The new system consists of a data management system with its backup power, software management application, power banks (UPS) for main and backup, and HMI. The vending system consists of 20 vending machines equipped with their software and data management system. The system was installed at Electricity House and the backup system was installed at Blackhall Road. Economic and Financial Analysis (i) Methodology and assumptions: The economic analysis has focused on Component 1 of the project and assessed the benefits deriving from loss reduction as a result of distribution system rehabilitation and collection improvements. Net economic and financial benefits are measured by contrasting the benefits and savings gained from upgraded distribution and collection infrastructure, and the costs of investment and O&M. Assumptions. The following assumptions have been made for the calculation of economic and financial returns: (i) The loss reduction targets are assumed to be phased in with a three-year lag of benefits over investment outlays. Non-technical transmission and distribution losses are assumed to decline from 30 percent to 15 percent, and the collection ratio to improve from the 2011 rate of 76 percent to 86 percent. System-wide technical losses are assumed unchanged at 18.5 percent. (ii) The investment in the 11/33 kv primary distribution network is expected to improve reliability of supply, and will enable more power to be distributed. An additional 8MW is expected to be evacuated once these works are completed, about 31 GWh at the estimated load factor. (iii) Tariffs are expected to increase by 10 percent every 5 years. 35

50 (iv) Life of assets and of newly installed equipment is assumed as follows: 20-years for the 11/33kV distribution system; 7 years for the BIS; and 10 years for pre-paid meters. (v) The discount rate is set at 10 percent. No taxes and duties are included in the investment costs, which are taken at constant 2016 prices. (ii) Results of economic analysis The NPV of economic benefits over a 20-year life of the project is US$2.46 million. The ERR is 14 percent. The economic returns are moderate, because the loss reduction program achieved so far under the project was small. The economic returns are robust with respect to lifetime assumptions. (iii) Results of project financial analysis The financial analysis of the project reveals that the project is financially viable with an IRR of 33 percent in the base case and an NPV of US$23.43 million. 36

51 Table1 3.1: Energy Access Project- Project Investments Costs Project Investments Costs TECHNICAL LOSS REDUCTION Total Energy distribution infrastructure Statistical meters Engineering and supervision Total distribution equipment Total engineering & supervision Total technical loss reduction US$ millions Cumulative COMMERCIAL LOSS REDUCTION Meters and installation Other Software Total commercial loss reduction US$millions Cumulative DEMAND SIDE MANAGEMENT Equipment/revenue management system Total Demand-side Management US$millions Cumulative Total Cumulative

52 Table 3.2: Energy Access Project- Economic and Financial Analysis Economic Evaluation Total Benefits Technical loss reduction Commercial loss reduction Gross benefits Costs Investment cost Cumulative O&M Total costs Net cash flow EIRR/NPV 14% 2.46 NPV costs (million) $10.1 NPV Benefits (million) $12.5 Financial evaluation Total Benefits Technical loss reduction Commercial loss reduction Gross benefits Costs Investment cost O&M Total Costs Net cash flow EIRR/NPV 33% NPV costs (US$ million) $10.1 NPV Benefits (US$ million) $33.5 Discount rate 10% 38

53 39

54 40

55 41

56 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team Members Names Title Unit Responsibility/ Specialty Lending Elvira Morella Energy Specialist AFTEG Team Leader Mudassar Imran Sr. Energy Economist SEGEN Team Leader Nikolay Nikolov Sr. Energy Specialist AFTEG Technical Analysis/Policy Dialogue Brent G. Hampton Sr. Energy Specialist AFTEG Technical Analysis Sameh Mobarek Sr. Counsel LEGPS Legal Christine Makori Counsel LEGAM Legal Arnaud Braud Financial Analyst AFTEG Financial Analysis Manuel Berlengiero Energy Specialist AFTEG Technical/Economic Analysis Maria Concepcion J.Cruz Lead Social Development Specialist AFTCS Social Safeguard Moses Y. Duphey Environmental Safeguard Specialist AFTEN Environmental Safeguard Joyce Olubukola Agunbiade Financial Management Specialist AFTME Financial Management Frederick Yankey Sr. Financial Management Specialist AFTME Financial Management Viorel Velea Procurement Specialist AFTPE Procurement Adu-Gyamfi Abunyewa Sr. Procurement Specialist AFTPE Procurement Raima Naomi Oyeneyin Language Program Assistant AFTEG Project Team Support HQs Fatu Karim-Turay Team Assistant AFMSL Project Team Support Sierra Leone Peter Meier Lead Consultant (Economic Analysis) Supervision/ICR Chita Azuanuka Obinwa Senior Program Assistant GEE01 ACS Support Collins S. Umunnah Program Assistant GEE01 ACS Support Joseph Quayson Energy Specialist GEE08 Engineer Elvira Morella Energy Specialist AFTEG Team Leader Mudassar Imran Sr. Energy Economist SEGEN Team Leader Nikolay Nikolov Sr. Energy Specialist AFTEG Technical Analysis/Policy Dialogue Brent G. Hampton Sr. Energy Specialist AFTEG Technical Analysis Sameh Mobarek Sr. Counsel LEGPS Legal Christine Makori Counsel LEGAM Legal Arnaud Braud Financial Analyst AFTEG Financial Analysis Manuel Berlengiero Energy Specialist AFTEG Technical/Economic Analysis Maria Concepcion J.Cruz Lead Social Development Specialist AFTCS Social Safeguard Moses Y. Duphey Environmental Safeguard Specialist AFTN3 Environmental Safeguard Joyce Olubukola Agunbiade Financial Management Specialist AFTME Financial Management Frederick Yankey Sr. Financial Management Specialist AFTME Financial Management Viorel Velea Procurement Specialist AFTPE Procurement 42

57 Adu-Gyamfi Abunyewa Sr. Procurement Specialist AFTPE Procurement Raima Naomi Oyeneyin Language Program Assistant AFTEG Project Team Support HQs Fatu Karim-Turay Team Assistant AFMSL Project Team Support Sierra Leone Carol Litwin Senior Energy Specialist Task Team Leader Cephas Gakpo Senior Consultant N/A Engineer Jianping Zhao Senior Energy Specialist GEE08 Task Team Leader Richard Olowo Lead Procurement Specialist GCFKE Procurement Specialist Gloria Malia Mahama Social Protection Specialist GSU01 Safeguards Specialist Pedro Antmann Lead Energy Specialist GEE08 Sector Lead Rahmoune Essalhi Procurement Specialist GGO01 Procurement Specialist Robert Robelius Consultant GEN05 Environmental Specialist Innocent Kamugisha Procurement Specialist GGO01 Procurement Specialist Sydney Augustus Olorunfe Godwin Financial Management Specialist GGO31 Financial Management Specialist Felix Nii Tettey Oku Environmental Specialist GENDR Environmental Specialist Anita Bimunka Takura Tingbani Environmental Specialist GEN01 Environmental Specialist Nash Fiifi Eyison Senior Energy Specialist GEE07 Task Team Leader and ICR contributor Nestor Ntungwanayo Consultant IEGSD ICR Primary Author (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$ thousands (including No. of staff weeks travel and consultant costs) Lending FY , FY , FY , Total: , Supervision/ICR FY , FY , FY , FY , FY ,

58 FY , Total: ,

59 Annex 5. Beneficiary Survey Results (if any) Not Applicable 45

60 Annex 6. Stakeholder Workshop Report and Results (if any) Not Applicable 46

61 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR 47

62 Annex 8. Comments from Sierra Leone DFIF Office Dear Parminder, Thank you for sending Mary the Implementation Completion Results and Report for Energy Access Project (P126180). On her behalf (with thanks to Simon Kenny and Tim Hatton), here are DFID Sierra Leone s comments to enable the team to finalise and submit for Board approval. Please also note the request at the end, which is not for inclusion in comments section of the report of for publication. Overall the report provides a fair assessment of the project at completion. The project outcome was rated Substantial against the original development objective: (i) to reduce losses in electricity supply in the Freetown Western Area; (ii) improve commercial performance of the National Power Authority, and (iii) increase access to electricity in selected rural areas. The project was restructured in October 2015, when the third component (rural electrification) was dropped. The restructured project outcome has consequently been rated as High. Almost all the project outputs were either achieved or surpassed expectations. Notably the project rehabilitated 5 sub-stations (against a target of 4); installed 20,000 pre-paid meters; installed 100 statistical meters; and constructed 7.8 km of distribution line. The project also prepared a number of important sector related policy documents related to network planning and investment. The project s efficiency rating has been scored Modest. This was based on the ex-post economic and financial analysis being positive but marginally less than expected at the time of appraisal; and a recognition that the Project Management Unit had limited capacity in completing procurement and safeguards requirements. The Ebola outbreak is (rightly) cited as a justification for the operational efficiency being rated as Modest. Other reasons given include delays in launching the project and limitations in the Bank s supervision function. The Bank s overall performance was rated as Moderately Satisfactory. Quality at entry was relatively good; supervision was affected by weak capacity in the PMU and the Ebola crisis. The project was rated as High Risk at the time of appraisal. Government commitment to the project remained high. However, fiduciary and governance risks (mainly procurement procedures and safeguards), implementation capacity risks, and unforeseen (i.e. Ebola) risks all materialised. Procurement performance was rated Moderately Satisfactory, after mitigation measures were put in place through the deployment of an energy specialist. Financial management was rated Satisfactory. Financial Performance shows 98%, with a latest estimated utilisation of $15.67 million against an appraisal estimate of $

63 million. It is noted in paragraph 64, however, that an external audit of the project is due in January We are expecting a reimbursement of unspent funds from the Bank in due course. DFID took part in the ICR review mission and confirmed the review s findings. The draft mission report was used as the basis for our own Project Completion Report, and this is consistent with the results reported in this draft ICR report. Overall the DFID project scored an A: Outputs met expectations. The project outcome was also deemed to have met expectations and provided value for money. It should noted that the 3 rd component of the original project (rural electrification), which was dropped from the EAP, is now being implemented under a much larger DFID funded 34.8 million Rural Electrification in Sierra Leone (RESL) project, managed by UNOPS. We are in full agreement with the ICR report s assessment of future risk to the development outcome of the project, as highlighted on page 23 of the report. The transformer and switchgear provided at Blackhall Road substation had not been installed at the time of project completion. This issue was raised at the completion review mission wrap up meeting with the Minister of Energy in Freetown in October It was agreed that these works would be completed under the World Bank s current Energy Sector Utility Reforms Project. It is critical that the World bank follow through on this agreement. The Bank should also ensure that the Government of Sierra Leone (GoSL) makes full use of the two studies on Tariffs and on Integrated Resource Planning that were financed from the Energy Access Project. Bobby Stansfield Economic Growth Team Leader DFID Sierra Leone 49

64 Annex 9. List of Supporting Documents 1. Project Appraisal Document, 2. Restructuring paper of the project, October Aide-memoires of supervision missions, Implementation status and results reports, Borrower s implementation completion report, November EDSA Monthly Progress Report, August EDSA Business Plan and Management Contractor Strategic Plan- Comparative analysis and Progress. 50

65 MAP 51