Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report. An IDEV Country Strategy Evaluation

Transcription

1 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report An IDEV Country Strategy Evaluation October 2016

2 IDEV conducts different types of evaluations to achieve its strategic objectives Impact Evaluations Evaluation Syntheses Corporate Evaluations Sector Evaluations Project Performance Evaluations (Public Sector) Thematic Evaluations Project Cluster Evaluations Regional Integration Strategy Evaluations Country Strategy Evaluations Country Strategy Evaluation Project Performance Evaluations (Private Sector)

3 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report An IDEV Country Strategy Evaluation October 2016

4 ACKNOWLEDGMENTS Task manager Team members Consultant External peer reviewer Knowledge management officers Special thanks to Division manager Evaluator-General Madhusoodhanan Mampuzhasseril, Principal Evaluation Officer Erika Maclaughlin, Long Term Consultant, Foday Turay, Chief Evaluation Officer, Latefa Camara, Long Term Consultant, James Sackey, Consultant Agrer S.a. - N.v., Belgium. Paolo Liebl Von Schirach, Team Leader. Members: Baptiste Forquy, Habtom Asmelash, Bernd Drechsler, John Murphy, Charles Haanyika, Vikramdityasing Bissoonauthsing Bruce Murray, former Director General, Operations Evaluation Department, Asian Development Bank Jayne Musumba, Principal Knowledge Management Officer Jerry Lemogo, Junior Consultant, Communications and Knowledge Management Norad The Norwegian Agency for Development Cooperation Samer Hachem Rakesh Nangia 2016 African Development Bank Group All rights reserved Published October 2016 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report IDEV Country Strategy Evaluation, October 2016 Disclaimer Unless expressly stated otherwise, the findings, interpretations and conclusions expressed in this publication are those of the various authors of the publication and are not necessarily those of the Management of the African Development Bank (the Bank ) and the African Development Fund (the Fund ), Boards of Directors, Boards of Governors or the countries they represent. Use of this publication is at the reader s sole risk. The content of this publication is provided without warranty of any kind, either express or implied, including without limitation warranties of merchantability, fitness for a particular purpose, and non- infringement of third-party rights. The Bank specifically does not make any warranties or representations as to the accuracy, completeness, reliability or current validity of any information contained in the publication. Under no circumstances including, but not limited to, negligence, shall the Bank be liable for any loss, damage, liability or expense incurred or suffered which is claimed to result directly or indirectly from use of this publication or reliance on its content. This publication may contain advice, opinions, and statements of various information and content providers. The Bank does not represent or endorse the accuracy, completeness, reliability or current validity of any advice, opinion, statement or other information provided by any information or content provider or other person or entity. Reliance upon any such opinion, advice, statement, or other information shall also be at the reader s own risk. About the AfDB The overarching objective of the African Development Bank Group is to spur sustainable economic development and social progress in its regional member countries (RMCs), thus contributing to poverty reduction. The Bank Group achieves this objective by mobilizing and allocating resources for investment in RMCs and providing policy advice and technical assistance to support development efforts. About Independent Development Evaluation (IDEV) The mission of Independent Development Evaluation at the AfDB is to enhance the development effectiveness of the institution in its regional member countries through independent and instrumental evaluations and partnerships for sharing knowledge. Independent Development Evaluation (IDEV) African Development Bank Group AfDB Headquarters Avenue Joseph Anoma, 01 BP 1387, Abidjan 01, Côte d Ivoire Phone: idevhelpdesk@afdb.org idev.afdb.org Design & layout: CRÉON Original language: English Translation: AfDB Language Services Department

5 Contents Acknowledgments ii Abbreviations and Acronyms v Executive Summary 1 Management Response 9 Introduction 22 Background 23 Economic Context 23 Social Context 24 Development Challenges 24 Overview of the Bank's Country Strategies and Portfolio 26 Overview of Strategic Priorities 26 Project Portfolio 26 Evaluation Approach and Methodology 29 Evaluation Issues and Questions 29 Methodology and Lines of Evidence 29 Evaluation Findings 30 Relevance 30 Effectiveness 32 Efficiency 45 Sustainability 49 Crosscutting Themes 52 Quality at Entry 55 Supervision 58 Leveraging and Co-financing 59 Conclusions and Recommendations 61 Annexes 67

6 Contents List of Figures Figure 1 Number of operations and amount approved ( ) 27 Figure 2 Number of operations and amount for sectors ( ) 27 Figure 3 Zambia Corruption Perceptions Index scores ( ) 37 Figure 4 Zambia CPIA Scores - Accountability Transparency and Corruption ( ) 37 Figure 5 Ease of Doing Business Indicators (number of days) 38 Figure 6 No. of new businesses registered during Figure 7 Domestic credit to the private sector (percent GDP) Figure 8 Access to improved water and sanitation source 44 Figure 9 Disbursement Ratios Zambia and AfDB ( ) 48 List of Tables Table 1 Relevance rating 30 Table 2 Effectiveness rating 32 Table 3 PEFA Scores - Procurement and External Audit 36 Table 4 Progress on key PEFA indicators 39 Table 5 Planned and Delivered Knowledge Work 40 Table 6 Progress against selected macroeconomic indicators 43 Table 7 Progress on selected health and education indicators ( ) 45 Table 8 Relevance rating 46 Table 9 Sustainability rating 49 Table 10 Regional Tariff Levels 51 Table 11 Crosscutting Issue Rating 53 Table 12 Operational selectivity in Zambia CSPs 56

7 Abbreviations and Acronyms v Abbreviations and Acronyms ADB ADF CEDR African Development Bank African Development Fund Comprehensive Evaluation of the Bank's Development Results CETZAM Christian Enterprise Trust of Zambia CPIA CSP CSPCR CWMIP DAC DAPP DO EPC ERB ESMP FI FNDP GNP GDP GRZ HIPC HRH ICT IFMIS Country Policy and Institutional Assessment Country Strategy Paper CSP Completion Report Community Water Management Improvement Project Development Assistance Committee Development Assistance from Person to Person Development Objective Engineering, Procurement and Construction Energy Regulation Board Environmental and Social Management Plan Financial Intermediary Fifth National Development Plan Gross National Product Gross Domestic Product Government of Republic of Zambia Highly Indebted Poor Countries Human Resources for Health Information Communication Technology Integrated Financial Management Information System IMF ITPC JASZ JICA MoFNP MPSAs MSME International Monetary Fund Itezhi-Tezhi Power Corporation Joint Assistance Strategy for Zambia Japanese International Cooperation Agency Ministry of Finance and National Planning (Zambia) Ministry, Provinces and Spending Agencies Micro Small and Medium Enterprises NRWSSP National Rural Water Supply and Sanitation Project OAG OECD PAF PCGF PEFA PEMFA PPA PPP PRBS Office of the Auditor General Organization for Economic Cooperation and Development Performance Assessment Framework Partial Credit Guarantee Facility Public Expenditure and Financial Accountability Public Expenditure Management and Financial Accountability Power Purchasing Agreement Public-Private Partnership Poverty Reduction Budget Support PRODAP Project to Support Lake Tanganyika Integrated Regional Development Program RDA SIP SME SNDP Roads Development Agency Small Scale Irrigation Project Small and Medium Enterprises Sixth National Development Plan An IDEV Country Strategy Evaluation

8 vi Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report TA TSA UA USD WHO WSS XSR Technical Assistance Treasury Single Account Unit of Account (of African Development Bank) US Dollar World Health Organization Water Supply and Sanitation Extended Supervision Report ZANACO Zambia National Commercial Bank Plc. ZCCM ZESCO ZMFO ZMW ZPPA Zambia Consolidated Copper Mines Zambia Electricity Supply Corporation Zambia Field Office (of the African Development Bank) Zambian Kwacha Zambia Public Procurement Agency

9 Executive Summary 1 Executive Summary This report presents a summary of findings, conclusions and recommendations from an evaluation of the Bank's Country Strategies and Program in Zambia over the period The evaluation was conducted to support the preparation of the Zambia Country Strategy and contribute to IDEV's comprehensive evaluation of the Bank's development results (CEDR). The evaluation seeks to assess the relevance and performance of the Bank's strategic interventions in Zambia and identify findings, conclusions and recommendations to inform strategies and operations going forward. Zambia Country Strategies. The evaluation covers three Country Strategy Papers (CSPs) that is, the CSP, extended to 2006; the Joint Assistance Strategy for Zambia (JASZ); and the CSP. The strategy emphasized agricultural development, access to water supply and sanitation and the promotion of child welfare. Under the JASZ, strategic emphasis on infrastructure development to support the agricultural sector was retained in addition to the promotion of accountability and transparency in the management of public resources through general budget support. The CSP signaled a shift in the Bank's priorities, with infrastructure development now placing greater emphasis on regional integration, particularly with regard to transport and power infrastructure. In fact, the Bank's policy-based operations have evolved to target private sector regulatory reforms aimed at increasing access to finance. Portfolio. During the period, the Bank approved a total of 43 operations amounting to UA million net of cancellations. The portfolio of projects under review comprises 34 projects, four studies, three emergency operations and two technical assistance grants. In addition to these national operations, there were 19 multinational operations amounting to UA million where Zambia was involved in varying degrees. The evaluation focused on national projects and examined regional projects only where there was a tangible national component. Over the years, the portfolio has grown in terms of numbers and the amount committed. The majority of the Bank's support by value has been in the transport, power, WSS and agriculture sectors (31.7 percent, 19.3 percent, 14.3 percent and 10.9 percent respectively). Evaluation Findings Relevance Relevance was assessed as satisfactory. The Bank's CSPs were aligned with Zambia's national development plans. The Bank s operational priorities reflected national development priorities and the Bank's comparative advantages. This alignment is evident from the general budget support to address financial and budgetary transparency and accountability as identified in the CSP update. The 2007 JASZ reflected the GRZ's emphasis on good governance as per the Fifth National Development Plan (FNDP). The CSP's emphasis on the development of economic infrastructure, including domestic and regional transport and power infrastructure, reflected the objectives of the Sixth National Development Plan (SNDP). Interventions targeting improved economic and financial governance reflect the priorities of Zambia's Vision 2030 and the Revised SNDP. An IDEV Country Strategy Evaluation

10 2 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report The alignment of projects with Country Strategies was rated satisfactory overall. Alignment of the private sector operations, despite targeting job creation and economic development under the CSP and the JASZ, were constrained by the practice of identifying operations opportunistically. However, private sector operations were better aligned under the CSP. Social sector projects, on the other hand, did not always reflect the CSP priorities. Although the CSP targeted increased access to basic services this objective was expressed only in terms of water supply and sanitation infrastructure rather than health and education. Although support to vocational training and skills development was not explicitly mentioned as an operational priority in the CSP, the Support to Science and Technology Education Project was identified in the CSP Logframe as a means of addressing skills gaps and reducing barriers to private sector development. With a few exceptions, the alignment of projects with the needs of beneficiaries was rated satisfactory. While projects in the multi, power, social and transport sectors have addressed alleviation of development constraints at the national or regional level, projects in the agriculture and WSS sectors targeted the needs of specific beneficiary groups. Private sector projects were well-aligned with the needs of targeted companies and financial intermediaries but did not always fully address the needs of SMEs. Effectiveness Overall progress is rated as moderately unsatisfactory. The achievement of project outputs across sectors was moderately satisfactory, with 78 percent of outputs delivered successfully. However, achievement of project outcomes was moderately unsatisfactory, with progress demonstrated only toward 67 percent of outcomes. All the ongoing transport projects have faced considerable implementation delays for the delivery of outputs. About 50 percent of the physical works were completed for the Nacala Corridor project by the end of September 2015, but 95 percent of the original project timeframe had elapsed. The Kazungula Bridge project was delayed due to procurement issues. At the end of the evaluation period, just 0.98 percent of the project funds had been disbursed, 41 months after the approval of the project. Nonetheless, these projects will most likely achieve their intended outcomes. Overall, 80 percent of power sector projects achieved planned outputs. The planned Cost of Service Study, which would have informed the power sector reform, has not progressed due to an unsuccessful procurement process. Outcomes achieved include: a) additional electricity generation capacity of 120 MW; b) job creation for members of the local community although this tends to be temporary given the skill requirements required for more permanent jobs; and c) increased access to basic infrastructure, thereby promoting private sector development. Delivery of outputs of the WSS projects was rated satisfactory at 90 percent across projects. The outcome level achievements include: a) improved access to water sources for over 1,500,000 people; b) increased hours of service from five hours per day to 16 hours per day for the Nkana WSS project and 22 hours for 8 Centers WSS project; and c) reduction in waterborne diseases with the Nkana and 8 Centers project areas reporting a 43 percent and 48 percent decrease in diarrhea and dysentery respectively, while the interventions under the NRWSSP reduced the incidence of diarrhea from 50 percent to 35 percent across the Northern and Luapula provinces. The majority of the Bank's agricultural operations did not fully deliver the planned outputs and only moderate progress was made toward expected outcomes. However, some progress has been made towards increased incomes for farmers, access to markets and increased use of improved irrigation methods.

11 Executive Summary 3 The PRBS program has a large social component in addition to reforms targeting PFM and private sector regulatory reform. Delivery of outputs for the PRBS was satisfactory in terms of the implementation of identified triggers and reforms. However this was less the case for outputs related to PFM, which included the implementation of IFMIS and the Treasury Single Account (TSA). With respect to outcomes, progress was achieved towards strengthening accountability and transparency functions with the implementation of a National Anti-Corruption Plan and increase in coverage of external audit. The Zambia Public Procurement Agency (ZPPA) was empowered with oversight functions but there are still challenges because of the absence of an impartial complaints redressal system and irregularities resulting from inadequate understanding of guidelines. Private sector regulatory reforms considerably reduced the cost of doing business. There has been improvement in the access to health and education with an increase in supervised childbirths and improvements in teacher student ratios. Nonetheless, limited progress was achieved in terms of strengthening budget credibility and execution with reported irregularities in commitments and poor enforcement of regulations. Finally, about 87 percent of the planned private sector project outputs was delivered. The achievement of outcomes presents a mixed picture but is largely positive. Positive outcomes were achieved in the area of government revenues, job creation, and access to basic services and infrastructure. Achievements were less than expected in the areas of profitability of companies supported by the Bank (for example Lumwana Mine), and improvement in the terms of finance. over the period. The Bank is also a prominent dialogue partner for the Performance Assessment Framework (PAF) working group under the PRBS; it possesses a privileged relationship with the GRZ regarding policy dialogue. Delivery of knowledge work has intensified over the evaluation period, and ZMFO has used studies to inform and support policy dialogue, engaging in a high level policy dialogue on youth employment in 2012 and hosting a seminar on Jobs and Growth in Knowledge work in the form of studies has also been used to add value to the Bank's operations. At the broader country strategy level, the Bank's interventions made a tangible contribution towards developing an enabling business environment and increasing access to basic infrastructure and services. However, limited progress was achieved in strengthening public financial management and promoting agricultural productivity and diversity. The Bank's interventions in the WSS, power and social sectors have made a tangible contribution to increasing access to basic services. National trends for access to improved water and sanitation sources have been positive with the percentage of Zambians having access to an improved water source climbing from 55 percent in 2002 to 65 percent in In contrast, limited improvement has been seen in access to improved sanitation. The Bank's completed projects in the water sector have benefitted over 1.5 million people. With respect to access to electricity, the Bank has contributed 120 MW to Zambia's installed generation capacity through the implementation of the Itezhi-Tezhi power project. Progress has also been made towards health and education outcomes. An IDEV Country Strategy Evaluation After the establishment of ZMFO, the Bank has been participating effectively in policy dialogue with the GRZ. Since 2006, the Bank has been an active participant in the donor coordination efforts in Zambia, leading the Cooperating Partners Group troika in 2012, as well as leading the agriculture, transport and WSS working groups at various points In addition, the Bank has sought to promote economic growth and reduce poverty by increasing agricultural productivity and trade. Although overall agricultural exports have increased, little progress has been made in diversification. Agriculture as a share of total exports has decreased from percent in 2002 to 7.49 percent in

12 4 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report 2014 though there is an increase in absolute terms. Furthermore, there is evidence of increased land under cultivation and use of improved irrigation techniques. During , the total area cultivated increased by 16 percent, whereas total irrigated land in Zambia increased by seven percent during , suggesting that some gains have been achieved for agricultural productivity. However, there is limited evidence of agricultural diversification away from maize production, which the government has accorded highest priority. The Bank's interventions have aimed at improving the business environment in terms of ease of doing business, access to finance and business development. The ease of doing business index improved from 92 in 2006 to 83 in Access to finance as reflected in the domestic credit to the private sector as a percentage to the GDP increased from 10.2 in 2007 to 17.2 in 2013, though the Bank s contribution in this area was modest. The Bank missed the opportunity to support the corporate social responsibility initiatives adopted by Itezhi-Tezhi Power Corporation and Barrick, which would have enhanced the impact of the Bank s investments. Finally, during , the Bank's efforts to strengthen macroeconomic management through general budget support appear to have contributed to the reduced gross government debt and budget deficits. In 2014, however, the gross government debt has increased, reaching 35.1 percent of GDP. Similarly, annual deficits have returned to 2004 levels, reaching 10.6 percent. Furthermore, annual wage expenditure accounted for an increasing share of GDP and, in raw terms, has increased by more than 400 percent, from 2,968 billion ZMW in 2006 to 11,897 billion ZMW in These changes coincide with a decline in scores on financial governance indices, including CPIA scores for Fiscal and Debt Policy, from 4.5 in 2009 to 3.0 in Efficiency The efficiency of the Bank's portfolio is rated as unsatisfactory owing to considerable timeliness issues and portfolio disbursement ratios which have consistently fallen below the Bank-wide average. The Bank s completed projects have incurred on average eight-month delay for every planned year of project implementation. Several factors account for this delay, including project design issues, delays in meeting conditions, complexities with co-financed projects and procurement delays. Project design issues in the Bank s projects were characterized by an underestimation of costs leading to incomplete implementation of components or additional costs and time due to a re-engineering of the technical design. Indeed, delays in the fulfilment of conditions precedent to first disbursement have been a recurrent problem. The average delay for loan effectiveness was 13 months, while the average delay for first disbursement was five months. However, delays between approval and first disbursement have improved recently, falling from 16 months in 2011 to 12 months in Such delays have serious implications for the private sector operations where the Bank begins supervision only after the first disbursement. The financial viability of the borrower can deteriorate yet the Bank would be unaware of the risks involved. The complexities of co-financed projects were reflected in the incompatibility between the private and public sector approval processes of the Bank. These were exacerbated by delays in reconciling lending terms among multiple DPs. Finally, procurement delays resulted from weak procurement capacity within line ministries and heavy internal government approval processes. Related to this issue is the requirement that all contracts must be approved by the Office of the Attorney General (OAG), which is not able to clear all contracts in a timely manner due to resource constraints.

13 Executive Summary 5 The overall disbursement ratio has improved during the evaluation period, from 83 percent for the CSP to 96 percent for the JASZ and 93 percent for the CSP. As of 2008, the disbursement ratio for Zambia has fallen slightly below the Bank-wide average, with some recent improvements. As per the calculations available in PCRs, on average, the ratio between ex-post and ex-ante EIRR was 0.96, indicating that the projects came close to meeting their expected rates of return. Sustainability Sustainability of the Bank project outcomes was rated moderately unsatisfactory with variability across sectors, as noted below: Transport sector projects face financial sustainability and institutional capacity risks in ensuring the sustainability of outcomes. Institutional risks arise from management challenges of over-procurement and contract management irregularities. The primary sustainability concern for the Itezhi- Tezhi power project has been the financial sustainability of the project given that Zambia's energy tariffs continue to fall far below cost recovery levels. Finally, sustainability of private sector projects was compromised by regulatory changes as well as variable profitability among the targeted institutions. Crosscutting Themes The coverage of gender across the CSP periods was rated moderately unsatisfactory with uneven identification of gender-specific development constraints and few targeted interventions. However, gender has been mainstreamed into project design for the majority of projects by identifying employment and income generating opportunities for women. In some cases, efforts to promote income-generating activities for women were limited by project design omissions whereas other projects missed opportunities in this regard. The coverage of inclusive growth and environmental sustainability was rated moderately unsatisfactory, and was most explicit under the CSP but decreased across subsequent CSPs. The themes of inclusive and green growth were integrated across the portfolio through the promotion of socially and environmentally responsible investment and a reduction of disparities between urban and rural areas. Overall, integration of inclusive and green growth within the project portfolio was rated moderately satisfactory. An IDEV Country Strategy Evaluation WSS projects have benefitted from the selection of simplified and uniform technical solutions to reduce the cost and complexity of maintenance. But in some instances they have struggled to ensure financial sustainability of operations and reduce unaccounted for water. The primary risks to the sustainability of agricultural projects stem from institutional sustainability, ownership by the beneficiaries, and exogenous shocks. Poor sustainability of outcomes from the Bank's PRBS program is attributed to institutional capacity risks and insufficient government ownership. Leveraging and Co-financing Approximately 55 percent of projects approved have cofinancing from partners. Furthermore, the 2015 CSPCR notes that, over the strategy period, the Bank managed to secure co-financing of 220 percent of the original ADF allocation. With respect to the leveraging of funds, the Bank has served as the lead arranger for three projects, that is, the Itezhi-Tezhi Power Generation and Transmission Project, the Nacala Corridor Road Project and Kazungula Bridge. Through these projects the Bank was able to leverage its own contribution by factors of 16, 3.68 and 1.82, respectively. The Bank has also secured co-financing from emerging donors, including

14 6 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report India Exim Bank, the Development Bank of South Africa and the OPEC Fund for International Development. Managing for results All CSPs were rooted in a robust analysis of the political context, macroeconomic context, dimensions of poverty, national development strategies and constraints to growth surrounding the Bank s projects. Understanding of the country context has been underpinned by consultations to inform CSP design. However the clarity and realism of the intervention logic for the Bank's CSPs remained relatively weak until the CSP. Despite implementing activities in a broader range of sectors, the Bank's strategy has managed to become more coherent in that projects across a range of sectors now support a more limited number of strategic outcomes. Project design weaknesses were identified with regard to: a) realism of the intended outcomes given the scope and design of projects; and b) clarity of the intervention logic, including confusion between project outputs and outcomes issues on the realism of project design arose from the quality of feasibility studies underpinning the project designs. This resulted in an underestimation of project costs, time and quality of engineering designs. The frequency of supervision has increased over the evaluation period. A portfolio review indicated that, during , projects were supervised, on average, once every two years. The feedback from the government on the Bank s supervisions shows that the quality of interaction with the Bank has improved over the years with the establishment of the Country Office. Overall Assessment In the background of the above findings, the overall performance of the Zambia Country Strategy and Program has been rated moderately unsatisfactory, as shown below: Criteria Relevance Alignment of CSPs with National Development Plans Alignment of the Project Portfolio with CSPs Alignment of Projects with the Needs of Beneficiaries Rating for Relevance Effectiveness Delivery of Project Outputs Achievement of Outcomes Rating for Effectiveness Efficiency Timeliness of Project Implementation Financial and Economic Performance Rating for Efficiency Rating Satisfactory Satisfactory Satisfactory Satisfactory Moderately Satisfactory Moderately Unsatisfactory Moderately Unsatisfactory Unsatisfactory Moderately Unsatisfactory Unsatisfactory

15 Executive Summary 7 Criteria Sustainability Private Sector Power Sector Agriculture Sector Multi-Sector Social Sector WSS Sector Transport Sector Rating for Sustainability Crosscutting themes Coverage of Gender within CSPs Gender mainstreaming across the portfolio Inclusive and Green Growth within CSPs Inclusive and Green Growth across the portfolio Rating for Crosscutting Themes Overall Rating Rating Moderately Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory Unsatisfactory Moderately Unsatisfactory Moderately Satisfactory Moderately Satisfactory Moderately Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory Moderately Satisfactory Moderately Unsatisfactory Moderately Unsatisfactory Conclusions and Recommendations The foregoing analysis leads to the following main conclusions: a. The Bank s Country Strategies and Programs have been well aligned with both national development plans and the Bank's comparative advantage. b. The Bank's portfolio has become more coherent, adopting an integrated approach to development challenges. c. The Bank has largely delivered planned outputs but the achievement of outcomes has been limited by project design weaknesses and delayed implementation. d. The Bank has contributed to increasing access to basic services and improving the business environment. However, opportunities for upscaling have not been leveraged. e. Political and governance risks are becoming an increasingly important factor in the sustainability of projects. f. Whereas the Bank's activities have attempted to promote inclusive and green growth, opportunities to mainstream gender have not been fully leveraged. Based on the above findings and conclusions, the evaluation proposes the following nine recommendations. 1. Continue to support private sector regulatory reform and build capacity among the accountability functions of government, particularly audit and procurement. 2. Strengthen the Bank s role in donor coordination, analytical work and policy dialogue. The policy dialogue needs to be supported by rigorous analytical work in several areas of public policy including private sector development, public private partnerships, and regulatory systems with policy predictability. Coordinated efforts are required to address deficiencies in the monitoring and evaluation system at the country level and, in this regard, the Bank should contribute to the ongoing initiative. Policy dialogue should also be implemented to An IDEV Country Strategy Evaluation

16 8 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report address political and governance risks to the sustainability of projects. 3. Address constraints to private sector involvement in service delivery and address infrastructure bottlenecks. Inadequate tariff structures and macroeconomic pressures create disincentives to private sector participation. The Bank should identify, through analytical work and policy dialogue in particular, opportunities to mitigate the impact of these constraints through a range of instruments, including lending, TA for project selection and guarantees for service delivery and purchasing agreements. 4. Identify opportunities to upscale development outcomes from private sector investments. The corporate social responsibility initiatives of the Itezhi-Tezhi Power Project and the Lumwana Mine provide opportunities for complementary investments which could scale up the development impact of those projects. 5. Deepen the integration of gender in Bank s operations and engage in policy dialogue on gender with the government in collaboration with cooperating partners. The Bank, with other cooperating partners, should assist in the implementation of the National Gender Policy 2014 with a focus on gender auditing, gender responsive national planning and budgeting, and the establishment of a system for monitoring gender outcomes at country level. 6. Promote synergies within the Bank program and in coordination with other donors between projects which improve the productivity of small businesses and increase access to finance. Increasing the pool of SMEs that generate sufficient revenue to afford formal financial products remains an obstacle to improving access to finance. Opportunities for creating linkage between interventions which increase production of high value crops, incomes and access to markets among underserved groups and LOC or TA for SME finance should be explored. 7. Identify means of harmonizing project implementation approaches with other co-financing partners as well as within projects that possess both private and public sector components. The Bank needs to work closely with partners to reduce delays by identifying harmonized terms, conditions, engineering and procurement approaches for co-financed projects to avoid implementation delays, cost-overruns and output quality inconsistencies. 8. Develop the capacity of the GRZ for project selection, design and engineering, particularly for infrastructure projects and Public Private Partnerships. 9. Identify and mitigate operational issues surrounding multinational projects aimed at regional integration. Based on the identification of constraints, the Bank should establish linkage between the CSP and the broader regional integration strategy to make sure bottlenecks are addressed at the right level either through dialogue at country level or coordinated approach at regional level.

17 Management Response 9 Management Response Management welcomes the Independent Development Evaluation (IDEV) report: Evaluation of the African Development Bank's Country Strategy and Programme for Zambia ( ). The purpose of the Evaluation was to assess, among others, the relevance, effectiveness, efficiency, and sustainability of the strategic interventions and to inform the preparation of the next Zambia Country Strategy. The evaluation revealed that the Bank s Country Strategies were relevant in terms of strategic focus and alignment in addressing Zambia s Development challenges. The projects supported by the Bank delivered most of the planned outputs while project outcomes did not always meet the targets. Project efficiency was rated moderately unsatisfactory largely due to the lags in preparing projects and the delays between loan approval and effectiveness for first disbursement. Sustainability was rated moderately unsatisfactory. This was due to low profitability of some private sector projects while issues of cost reflective pricing was lacking in water and energy supported projects. Introduction The IDEV evaluation covers the period and three Country Strategy Papers: the Country Strategy, which was extended to 2006; the Joint Assistance Strategy for Zambia formulated as a joint program with support from multilateral and bilateral development agencies operating in Zambia; and the Country Strategy. CSP Strategic Focus The key strategic focus of the three Country Strategies was the support to infrastructure development in support of agriculture, water and sanitation, energy and transport. In the past five years, a higher degree of support to economic and financial governance was pursued to target regulatory reform to facilitate private sector development as well as public financial management reforms. Evaluation Design The evaluation team developed an evaluation matrix and theory of change to guide data collection and analysis. This included key indicators and judgment criteria based on a six point scale that was used to create a transparent account of how IDEV examined the achievement of results across each sector. Although Management received an overall rating for each judgement criteria, management has not seen how each individual project was assessed. This would have increased transparency on the part of the evaluation and allowed management to better understand the findings in terms of outputs and outcomes. As the evaluation period is quite long (more than 10 years), the report could also provide some information on how the performance changed over time, to make the assessment a bit more dynamic and indicate some trends. An IDEV Country Strategy Evaluation

18 10 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report The evaluation team notes various valid limitations of the evaluation methodology. Furthermore flaws in the intervention logic and the lack of baseline data made it difficult to evaluate progress on some indicators. Management agrees with the finding. The results based framework will be strengthened in the context of a clear intervention logic as well as indicators will be selected where there is baseline data available when drafting the next Country Strategy Paper. Program Implementation The evaluation uses four key criteria: relevance, effectiveness, efficiency, and sustainability to evaluate the country strategies. In addition, crosscutting issues such as gender, quality-at-entry, and leveraging and co-financing are also assessed. Each of these elements are commented on below. Project alignment to the needs of beneficiaries was rated overall satisfactory. Management agrees with the finding that the needs of Small and Medium Sized Enterprises have not always been sufficiently catered for. In the new Country Strategy it is proposed that the Bank will provide more targeted support to Small and Medium Sized Enterprises through collaboration with local financial institutions. Lines of Credit will be better targeted for the needs of SME s such as providing business development services which are often in high demand. Projects designed to support and expand SME s will also be developed. In fact, the Bank is currently financing the Cashew Project that links farmers with local processors and helps expand the market for Cashew. Entrepreneurs in the cassava value chain are also being supported through improved product development, produc tivity increase, and commercialising the crop. These support is being provided through capacity building and business development services. Relevance Relevance is measured as the degree of alignment of Country Strategies to National Development plans, and of projects to Country Strategies and beneficiaries. The alignment of Country Strategies to National Development Plans was rated satisfactory. The Bank has used its comparative advantage in the support of infrastructure development while addressing the specific needs of the Zambian Government as outlined in the National Development Plans (NDPs), consistent with Bank s Ten Year Strategy ( ) and the Hig-5s. Given the large investment needs that remain in transport, energy, and water and sanitation, Management will, consistent with the priorities in the NDPs, continue to support this strategic focus in the new Zambia Country Strategy. Effectiveness Project effectiveness is measured based on achievements of outputs and outcomes targets. Achievements of outputs are rated moderately satisfactory, while achievements of outcomes are rated moderately unsatisfactory. Project Output: Most prioritized sectors have demonstrated good progress in achieving planned outputs including private sector, energy, agriculture, water and sanitation, transport, and governance. Agriculture has been a challenging area with limited progress due to a difficult implementation environment and complex procurement processes that contributed to implementation delays. Project Outcome: Achievement of outcomes is unsatisfactory as a result of start-up delays, implementation and procurement delays, and delayed completion dates for projects. This in turn

19 Management Response 11 affected project impact and achievement of results. The evaluation report states that achievements were less than expected in private sector projects due to profitability and lack of improvement in the terms of finance. It should be noted that for most of the period under evaluation, Zambia was an ADF only country with most of the financing targeting public sector projects. The price of copper between 2011 and 2015 declined by more than 50 percent with prices falling to under USD 4800 per tonne in 2015 affecting Governments revenue generation. The added pressure to Government s fiscal position led to higher domestic lending pushing up the price of funding thus crowding out the private sector. Average interest rates are high and currently at above 28 percent. Although the Bank has provided Lines of Credit to strengthen SME financing, it has no influence on the cost of treasury bills which are used as the benchmark for commercial lending. Policy dialogue and knowledge work: Management notes the positive development of the decentralisation process in terms of policy dialogue and donor coordination. The Bank s physical presence has made it possible to service its key clients, build a close relationship and enhance dialogue with the Government, use the local networks, build the portfolio while leveraging finance from development partners, provide capacity building to project implementation units and respond efficiently to arising needs. Strategic Outcomes: At the strategic outcome level the Evaluation Report concludes that the Bank has made a positive impact on improving the business environment, and increasing access to basic services and infrastructure. Limited progress has been achieved to strengthen Public Financial Management and promote diversification by promoting the agriculture sector. The Bank intends to continue supporting agriculture and agribusiness as it is critical for employment and rural incomes and as it is an important source of foreign earnings given its large market potential within the region. Diversification of the sector has been hindered by the Governments continued support to the Farmer Input Support Program and the purchase programs by the Food Reserve Agency. Both these programs are heavily focussed on supporting the production of maize. From 2015 an e-voucher system provides some promise in diversifying the sector as farmers can now choose inputs of their choice. In the new Country Strategy, Management is proposing to provide a holistic approach to agriculture by supporting value chains and developing markets in line with the Feed Africa-Strategy for Agricultural Transformation in Africa ( ). Improving public financial management has been a challenge. A Multi-donor Trust Fund, led by the World Bank is currently supporting this area, in which the Bank is not providing direct support. In the new Country Strategy, Management is proposing to strengthen Public Financial Management capacity directly through the implementing agencies. This will help build a critical mass of qualified accountants, internal auditors, and procurement officers and build the needed institutional capacity in the executing agencies that we support. It is felt that institutionalized capacity building of professionals would be more sustainable than project based training or orientation which targets only project staff whose turnover is high as they move on after the projects are completed leaving the executing agencies without the required capacity to undertake other new projects. Agencies will be screened for Public Financial Management skills and needs during project design. Any support to agencies will be based on this assessment. Activities proposed will be coordinated with the activities supported by the Multi-donor Trust Fund to avoid overlaps. Efficiency The efficient delivery of projects is evaluated based on the timeliness of projects and on the financial and economic performance that are demonstrated. An IDEV Country Strategy Evaluation

20 12 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Timeliness: The dimension of timeliness is further evaluated using four sub-dimensions: inadequate project design; loan effectiveness and disbursement; co-financing complexities; and, procurement delays. Management notes with concern that despite recent improvements, projects are significantly delayed on the various sub-dimensions mentioned above. Management would like to point out that in the past 18 months the government introduced new approval procedures, including the need to obtain parliamentary ratification of loan agreements with the new constitution, and this caused significant project effectiveness delays. Management has already taken steps to strengthen project design and preparation to reduce the risk of delays. To reduce delays caused by loan effectiveness and effectiveness to first disbursement any preconditions or other conditions that risk affecting a project will be addressed upfront. This implies that projects will only be elevated to the Board once conditions have been addressed, especially conditions that borders on project readiness and quality at-entry. Key portfolio issues, including weak capacity of project implementation units in procurement, financial management and contract administration are being addressed through joint quarterly portfolio review meetings and annual fiduciary clinics organised with the Ministry of Finance and attendance of project staff. The Bank has been very successful in leveraging projects through co-financing arrangements. However, in terms of implementation, as pointed out in the Report, significant delays have been experienced, particularly when bilateral donors are involved. Management has strengthened efforts to harmonise internal procedures to avoid similar types of process related delays in the future. Clear agreements on procedures will be made before Board approval and in accordance with PD 02/2015. Lessons learnt from prolonged negotiations of common terms agreements for cofinancing partners would guide the formulation of future projects. Procurement capacity of Government counterparts is often weak with a high turnover of staff. Despite Bank support to capacity building and training, procurement delays often persist. Management will continue to prioritise and provide training to help build a critical mass of procurement officers in the public sector. As part of new projects public procurement training will be strengthened and institutionalized in the implementing agencies where relevant. In addition longer term support could be provided through the Zambian Public Procurement Authority that is mandated to carry out capacity building of procurement entities. Financial and economic performance: Between 2002 and 2014, the disbursement ratios for Zambia portfolio had fallen below the Bank wide average. However, in 2015 improvements in disbursement ratios were recorded. The Zambia portfolio is young (2.4 years on average) with limited disbursements in the early period of project implementation. Improvements are therefore expected during 2016 and 2017 as the new projects start to increase their disbursement levels. This is consistent with implementation of most of the high value large infrastructure projects which usually have slow disbursements at the beginning. In addition, by improving quality at entry and project readiness, the disbursement ratio is expected to continue to increase in the coming years. Management notes that agricultural interventions are rated moderately satisfactory, yet the agricultural projects addressed the needs of specific groups of beneficiaries in terms of increasing productivity, improving food security and reducing poverty. Thus this assessment appears to be conflicting, in terms of performance of agricultural projects.

21 Management Response 13 Sustainability The sustainability rating was evaluated below satisfactory. For utilities, this was related to cost reflective pricing of water and energy. Management agrees that usage of public resources need to be cost reflective. The Bank is currently supporting an industry wide cost of service study that will inform the Zambian energy market on the pricing of electricity at different consumer and firm levels. The water utilities have struggled with improving the none-revenue water in order to reach operational efficiency. The Bank continues to work with utilities to improve infrastructure and metering to reduce leakages while continuing to provide capacity building. Management will take into consideration the social aspect of utilities in terms of providing services and protecting the poor. A carefully designed cost reflective approach and tariff pricing should therefore be pursued when supporting such projects. The Evaluation Report concludes that some private firms were affected by regulatory changes and variable profitability that adversely affected the sustainability rating, e.g. the support to Lumwana mine and CETZAM financial institution. However, as mentioned earlier, profitability was affected by a significant decline in copper prices between 2011 and 2015 which also affected public finances and raised commercial lending rates. Notwithstanding the above and with a few exceptions, most AfDBsupported firms have managed to adjust to the more difficult business environment and are operating as going concerns. Cross-Cutting Two dimensions, gender and inclusive growth, were used to assess the level of integration of crosscutting issues into Country Strategies and the project portfolio. Gender: The Bank is implementing the Bank s Gender Strategy while gender mainstreaming in all its operations. However, according to the Evaluation Report gender received limited attention across the evaluation period. Management acknowledges the need to pay more attention to gender and recognises a strengthened approach to addressing gender as the new Country Strategy is being prepared. Initially, projects lacked effective tools for addressing gender issues in Bank operations. A comprehensive country gender assessment is about to be commissioned to identify gender gaps and challenges, of which the outcome will inform gender specific interventions in Bank operations under the new CSP. Inclusive Growth: Focus on inclusive and green growth and environmental sustainability was most prominent between 2002 and Although poverty levels have fallen in urban areas, poverty in rural areas remains stubbornly resistant. Limited access to markets is cited as a key cause. In the new Country Strategy, Management plans to support a stronger linkage between rural production areas and urban markets. This will be done by supporting agriculture value chains and the development of markets. In the current portfolio, climate change is being addressed in several ways. One project, Strengthening Climate Resilience in the Kafue Basin, aims to mitigate against drought and flooding through improved planning and climate resilient infrastructure and production systems. Other projects such as Lake Tanganyika Development and the Agriculture Productivity and Market Enhancement Projects implement climate change as a cross cutting issue being mainstreamed into the main activities. Some older agriculture projects that were designed and implemented before 2008 may have had some adverse environmental impacts. Currently all projects are classified in terms of their potential level of social and environmental impacts as guided by Bank Policy. Category 1 and 2 projects are subject to Environmental and Social Management Plans. This practice will continue. An IDEV Country Strategy Evaluation

22 14 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Quality at Entry Two dimensions of quality-at-entry have been analysed. The conclusion is moderately satisfactory for the Quality-at-Entry of Country Strategies while quality and realism of project design was moderately unsatisfactory. Management notes with satisfaction that the intervention logic improved with the Country Strategy. The framework still needs to be more practical while higher attention needs to be placed on the realism of indicators, especially at the outcome level. Management will ensure that the results framework in the new Country Strategy demonstrates a clear and coherent theory of change and is more practical. Management notes that in some instances project design has been inadequate and that this led to under costing of project outputs and a lower number of outputs delivered. Management acknowledges that the use of Project Finance in a Zambian context was challenging due to limited understanding of the approach and weak capacity in the area. In order to build capacity, the Bank provided training to government officials in the region in the use of and preparation of Project Finance. Leveraging and Co-Financing Management efforts to promote leveraging and co-financing has been successful in Zambia as indicated in the Evaluation Report. This was particularly relevant following the financial crisis and when Zambia could only access ADF resources. With the reclassification to Blend country, Zambia has increased its access to ADB resources while the need for leveraging may diminish in some types of lower risk projects. However, management will continue to pursue co-financing to forge partnerships with development partners and utilise synergies. A recent example of such a project is the Kariba Dam rehabilitation project which the Bank is financing with two multilateral partners and a bilateral partner. Conclusions and Key Lessons Management appreciates the following conclusions and key lessons emanating from the Evaluation Report: The Country Strategies and programmes have been well-aligned with the national vision, national development plans and national strategies. This confirms the Banks focus on addressing the large infrastructure gaps that Zambia still faces. Furthermore the Management s plan to strengthen support to private sector in the new Country Strategy, while addressing regulatory constraints and bottlenecks, is the appropriate strategy going forward. The integrated approach that the Bank followed in the Country Strategy by focusing on the strategic outcomes has meant a broader selection of sectors that have received support. Additionally, the Report confirms that the integrated approach has more coherently addressed Zambia s development challenges. Management plans to continue this approach in the new Country Strategy. Although outputs have largely been delivered as planned, the level of achievement of outcomes is below satisfactory largely due to implementation delays. There is need for improved quality at entry to ensure project readiness. For example, for infrastructure projects that require feasibility studies, such studies should be completed before presentation of the project to the Board. Bank support has contributed to increased basic services and improvements in the business environment. Under private sector, more could be

23 Management Response 15 done in utilising channels for upscaling projects. One example the Report mentions is scaling up activities that the private sector supports through their Corporate Social Responsibility programs. Political, governance and regulatory risks have increased during the past 5-7 years with regulations that are passed without sufficient consultation with the private sector. This has affected project implementation and profitability of some private sector operators that have reduced their investments. For example, the most profitable mine in the country has not made new investments in the Country since 2008 due to regulatory changes that are not discussed with the private sector. Enhanced policy dialogue, knowledge work and addressing risk will be important activities in the implementation of the new Country Strategy. Mainstreaming gender in operations has not been fully leveraged. In the preparation of new projects enhanced gender screening and screening for smart climate approaches, skills needs and employability of direct beneficiaries and a conducive regulatory environment will receive more attention when designing projects in the implementation of the new Country Strategy. Recommendation MANAGEMENT ACTION RECORD Management response Recommendation 1: Continue to support private sector regulatory reform and build capacity among the accountability functions of government. Budget support and TA interventions have been successful in promoting private sector regulatory reform and strengthening of accountability functions, partly due to adequate ownership among implementing bodies. These issues continue to be relevant, particularly the need to strengthen audit and procurement capacity. Furthermore, strengthening external accountability functions can complement the implementation of financial control systems to reduce irregularities and improve public financial management. AGREED. Support to private sector reforms and capacity building have yielded positive results. These interventions need to be continued and extended to areas identified in the recommendation. Action: As proposed in the new CSP, Bank operations will continue to include support to Economic and Governance Reforms. Reforms can be supported through sector budget support or will be included as components in relevant projects and programs, and aligned with the High-5s. Additionally, institutional support program(s) including support to strengthening audit and procurement capacities in the public sector will be considered. Task Managers will be responsible for ensuring that projects are screened for relevant sector reforms during the project preparation stage and included in project design where it makes good sense. Furthermore, task managers will be responsible for assessing needs in relation to accountability functions in the project institutions that we support (ZMFO, 2017) An IDEV Country Strategy Evaluation

24 16 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Recommendation MANAGEMENT ACTION RECORD Management response Recommendation 2: Strengthen the Bank s role in donor coordination, analytical work and policy dialogue. The Bank, with its unique positioning in Africa and among the development partners, is expected to play a key role in donor coordination, analytical work and policy dialogue. The policy dialogue needs to be supported by rigorous analytical work, which is also demanded by the government in several areas of public policy including private sector development, public private partnerships, and regulatory systems with policy predictability. The donor coordination efforts are currently subdued due partly to the withdrawal of several cooperating partners from key areas of assistance including budget support. Coordinated efforts are required to address deficiencies in the monitoring and evaluation system at the country level to enable systematic reporting of results with active participation of all line ministries, and the Bank should contribute to the ongoing initiative in this regard. Finally, policy dialogue should also be implemented as a means of addressing increasing political and governance risks to the sustainability of the Bank's projects, particularly with regard to ensuring adequate consultation with the private sector to inform regulatory changes. AGREED. Management agrees with the key roles that the Bank is already playing in Zambia as well as the need to continue to strengthen them. Although the Budget Support group is no longer active, a Cooperating Partners Group continues to carry out policy dialogue at the highest levels. The Bank remains very active in this group and chaired it in 2012 while also chairing other working groups (transport, water and sanitation, agriculture and monitoring and statistics group) that carry out policy dialogue at the sector level. Action: Knowledge work has intensified during the evaluation period as stated in the report (section 5.2.7). The Bank will continue to strengthen analytical work to improve the underpinnings of our policy dialogue. As proposed in the new CSP, non-lending activities will support policy analysis and other knowledge work while policy operations will be pursued at the sector level to facilitate key reforms (e.g. energy reforms). The proposed CSP was based on a diagnostic study prepared by the Bank, relevant Bank studies and policies as well as sector studies from other agencies. Management will continue to identify, coordinate and drive relevant studies. Furthermore, during project preparation and design, relevant studies may be identified and included in the non-lending operations. (ZMFO, Ongoing) Monitoring and Evaluation at the country level will be strengthened. The Bank is working with the Ministry of Finance to create a simple tool for tracking projects at the formulation stage with the aim of reducing start-up delays. Expanding the tool to also incorporate results monitoring will be investigated and implemented, while taking into account a monitoring system that Government is already developing with the help of the EU, United Kingdom and Germany (Ongoing). The Country Programs Officer will be responsible to coordinate the development of the tracking tool during the first half of (ZMFO, 2017) Leveraging additional financial resources, both to finance public operations as well as private ones, would be carefully determined and pursued with various partners, especially for large infrastructure operations in energy, water and sanitation, road transport and agri-business development. (ZMFO, Ongoing).

25 Management Response 17 MANAGEMENT ACTION RECORD Recommendation Management response Recommendation 3: Address constraints to private sector involvement in service delivery. Private sector participation in service delivery, particularly in the energy sector, will play an increasing role in addressing Zambia's infrastructure bottlenecks. However, inadequate tariff structures and macroeconomic pressures create disincentives to private sector participation. The Bank should identify, through analytical and policy dialogue in particular, opportunities to mitigate the impact of these constraints through a range of instruments, including lending, TA for project selection and guarantees for service delivery and purchasing agreements. AGREED. The private sector will continue to be a key sector requiring Bank support. The Bank is currently supporting the undertaking of a cost of service study, whose finding and recommendation should help inform the setting of future tariffs as well as point out inefficiencies in the electricity system as a whole. The study will therefore inform Bank Policy dialogue in discussions of moving toward cost reflective tariffs. In the water sector, tariff and affordability studies are carried out for each utility and presented to the regulator. The tariff progressions are then implemented in parallel with efficiency gains from the utility based on key performance indicators that rewards good progress. However, improving efficiency is a process that can take several years to implement. Thus institutional strengthening and capacity building are important elements during project implementation. Action: In the new CSP, as part of the interventions in the energy sector, the Bank will target support to institutional strengthening and reforms as a means of attracting private investors. Furthermore, diversifying the energy mix will be important to reduce risk of relying only on hydro. Support to the sector will also entail providing Technical Assistance in the areas highlighted in the recommendation. The Bank will also support and/or use relevant studies to guide policy dialogue pertaining to the private sector. The Country Office together with the sector departments will be responsible for suggesting which instruments best address inadequate policy or constraints. The Government of Zambia has already requested for energy sector budget support which will provide an opportunity for the Bank to assist in reforming the energy sector and to enhance private sector participation. (ZMFO, 2017) Recommendation 4: Identify opportunities to upscale development outcomes from private sector investments. Although the Bank has helped improved access to services and infrastructure through public sector projects, private sector investments have yielded similar results through infrastructure works and corporate social responsibility initiatives. As demonstrated by the experience of the Itezhi-Tezhi Power Project and the Lumwana Mine, additional development impacts and private sector development could be realised through complementary infrastructure investments surrounding these projects. Integrating lessons from past experience into the CSP, the Bank should more systematically identify opportunities to upscale and complement such initiatives, thereby further contributing to service delivery, infrastructure improvements and business development. AGREED. Private sector operations have in many instances helped improve access to services and infrastructure. The challenge remains bringing on board similar operations to scale up such initiatives. Action: The Private Sector and the Agriculture Sector Specialists in the Country Office will be responsible for ensuring the following action points (ZMFO, OPSD, OSAN, 2017): In the new CSP, the Bank will focus on providing technical assistance such as business development services with the aim of helping identify and build capacity of the private sector. The Bank will also focus on support to the private sector through financing selected agri-value chains (production, processing and market linkages) and finance infrastructure needs (irrigation, collection, storage). It is expected that through these interventions, additional development impact will be realized. Management will seek more innovative interface in utilising public and private sector financing windows for agro-processing, irrigation and farm block infrastructure projects. An IDEV Country Strategy Evaluation

26 18 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Recommendation MANAGEMENT ACTION RECORD Management response Recommendation 5: Deepen the integration of gender in Bank s operations and engage in policy dialogue on gender with the government in collaboration with cooperating partners. Even though the Bank s analytical work on gender in Zambia had recommended specific areas for intervention, those were not implemented in earnest. In the increasingly relevant context of feminisation of poverty and HIV/AIDS pandemic, gender violence, women s unequal access to education, health services, and resources including land and credit, it is imperative that the Bank take concrete steps to integrate gender in its operations with respect to design, implementation and the achievement of results. The Bank, with other cooperating partners, should engage in policy dialogue with the government and assist in the implementation of the National Gender Policy 2014 with focus on gender auditing, gender responsive national planning and budgeting, and establishment of a system for monitoring gender outcomes at country level. AGREED. The Bank is implementing the Bank s Gender Strategy while mainstreaming gender in all its operations. The Bank will also continue to assist Government in the implementation of the Zambia National Gender Policy Action: Management will continue to ensure policy dialogue, collaborative analytical work and capacity building with relevant partners including the Ministry of Gender, United National Population Fund, and the Zambian Non-Governmental Organisation Coordinating Committee, as guided by the Zambia National Gender Policy. The Bank has already approved USD 30 million towards the cost of implementing the Zambia Skills Development and Entrepreneurship Project Supporting Women and Youth (SDEP-SWY). This operation aims to promote job creation, gender equality and poverty reduction. The project seeks to improve the livelihood opportunities of Zambia s working poor, especially in rural areas and will benefit particularly, women and youth by providing enabling infrastructure and entrepreneurship skills for MSME business development. This operation has commenced implementation, focusing on women and youth groups and is expected to close in (ZMFO, Ongoing) The Bank will continue to ensure that all operational staff obtain skills in gender analysis, especially for project preparation and appraisal. The Bank will further mainstream gender in all its operations by developing gender indicators in Results Based Frameworks. (ZMFO, Ongoing) Recommendation 6: Promote synergies within the Bank program and in coordination with other donors between projects which improve the productivity of small businesses and increase access to finance. Increasing the pool of SMEs that generate sufficient revenue to afford formal financial products remains an obstacle to improving access to finance. Opportunities for creating linkages between interventions which increase production of high value crops, incomes and access to markets among underserved groups with LOCs or TA for SME finance should be explored. Such linkages may yield synergies in terms of access to finance, business development and economic growth. Outgrower and farmers club schemes provide a proven means of improving agricultural productivity and diversity while linking farmers to markets and increasing both incomes and access to finance. Attention should be paid to ensuring that targeted project mechanisms are included to ensure that the benefits are gender inclusive. AGREED. Macro-economic instability has an impact on private sector s ability to borrow as they often compete with Government s domestic borrowing, impacting the cost of borrowing. Continued support to Government, through policy dialogue will create a conducive environment for private sector operations including lowering interest rates. Access to finance by private sector, especially SMEs forms a critical aspect of proposed pillars of the new CSP. Making resources available to SMEs continues to be a key ingredient in Bank s operations, especially the development of rural enterprises for job creation and inclusive development. In line with the Feed Africa Strategy, the Risk Sharing Facility will be promoted for agricultural value chain operations with the objective of catalysing the private investments and commercial bank lending. Action: The Private Sector Specialist in the Country Office will be responsible for ensuring the following action points (ZMFO, OPSD, 2017) The Bank s operations and interventions using lines of credit as an instrument will be accompanied by measures to improve capacity for business management and expansion. The Bank will intensify support to financial institutions to develop financial products for SMEs, especially those involved in agro-processing and value-chain related enterprises. This will include the incorporation of incubation programmes for youth and women SMEs. The Bank will ensure that financial and technical assistance to SMEs will clearly distinguish and address the peculiarities of enterprises that women usually engage in. Financial products, including insurance and bridge financing, would be applied on exceptional cases to women enterprises. The Bank will provide support towards agriculture rural enterprise development, promotion of outgrower schemes promotion of agricultural productivity and diversity interventions.

27 Management Response 19 Recommendation MANAGEMENT ACTION RECORD Management response Recommendation 7: Identify means of harmonising project implementation approaches with other co-financing partners as well as within projects that possess both private and public sector components. Addressing regional and domestic infrastructure constraints will likely necessitate continued cooperation and co-financing. The experience of the Kazungula Bridge and Itezhi-Tezhi Power Project demonstrates the need for the Bank to work closely with other partners by identifying harmonised terms, conditions, engineering and procurement approaches for co-financed projects in order to avoid implementation delays, cost-overruns and output quality inconsistencies. Furthermore, the design and implementation of cofinanced projects with both public and private components should be mindful of differences in the approval processes for these two sources of funds in order to avoid both unnecessary delays and additional costs. AGREED. Mixed results have been observed with operations that are co-financed with other partners. This is, in part, due to the significant differences in policy, regulatory and fiduciary requirements of each lender. Where project financing is sequenced from one lender to another, delays from one partner significantly affects the operations (procurement and disbursement) of other partners. Harmonization in this case only refers to co-financing arrangements where it is feasible. We have experience that demonstrates that even where rules and procedures are harmonized, for example with the WB, no arrangements are put in place to reduce dual No Objections from the financiers. However, we agree with the observation that it is not possible to apply one set of rules in all circumstances and in this case we have suggested full financing of specific contracts, i.e. parallel financing, to which specific rules would apply as opposed to co-financing a contract and subjecting it to several reviews from financiers. Action: The Bank will be responsible for ensuring the following action items (ZMFO, 2017): The Bank will ensure that project readiness mechanisms have been well articulated and agreed amongst partners. This would include availability of designs, preparatory studies, setting up of project implementation teams and harmonisation of procurement procedures irrespective of source of finance. The Bank will ensure that common terms agreements for complex transactions, especially PPP and project finance, are well negotiated in advance, ahead of Board approval. This will minimise potential delays and conflicts usually associated with such negotiation after Board approval. An IDEV Country Strategy Evaluation

28 20 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Recommendation MANAGEMENT ACTION RECORD Management response Recommendation 8: Increase the capacity of the GRZ for project selection, design and engineering, particularly for infrastructure projects and Public Private Partnerships. Inadequate project design and engineering has frustrated the implementation of projects across several sectors, including the transport sector, for which inadequate project design and engineering can result in considerable project delays as well as procurement disputes. Assistance in implementing high quality feasibility studies which identify reasonable project costs would also be beneficial for the Bank's ongoing support to the WSS sector, for which underestimation of the project complexity and costs have limited the successful delivery of improved sanitation facilities for past interventions. As additional PPPs are implemented in Zambia, technical assistance should be provided to improve project selection and ensure that PPP arrangements are providing added value. Finally, greater attention should be devoted to identifying and assessing sensitivity to political and governance risks, including tariff structures, regulatory changes and ownership. AGREED. As part of project readiness mechanisms that have been put in place, designs for infrastructure-related operations are usually in place ahead of Board approval, with the increased use of MIC TA grants, PPF and other relevant funding facilities. However, there are issues with the quality of some designs that require a review after Board approval to ensure quality during implementation. Action: The Country Programs Officer will be responsible for ensuring the following action items are addressed during project preparation (ZMFO, 2017): The Bank will intensify the use of advance contracting where appropriate for review of engineering designs and associated bills of quantities before Board approval. This will also help project teams to validate project detailed costs before Board consideration. The Bank will continue to provide capacity building and technical assistance to staff of key implementation units on various aspects of project formulation and implementation including PPP transactions, project formulation, and fiduciary issues such as financial management and procurement.

29 Management Response 21 MANAGEMENT ACTION RECORD Recommendation Management response Recommendation 9: Identify and mitigate operational issues surrounding multinational projects aimed at regional integration. The implementation of regional projects has been subject to excessive delays caused by coordination issues at the regional level, as seen in the case of PRODAP, which was discontinued in course of implementation. Delays in the full implementation of one-stop border post constrain the movement goods, services and people, affecting the competitiveness and economic growth of the region in general and the Zambia in particular. Based on the identification of constraints, the Bank should establish linkages between the CSP and the broader regional integration strategy to make sure bottlenecks are addressed at the right level either through dialogue at country level or coordinated approach at regional level. AGREED. Regional operations have continued to face significant challenges in coordination and harmonisation of processes. Soft issues of regional integration like the harmonisation of custom procedures, free movement of goods and services, have not received as much attention as infrastructure. Action: Task Managers with the support of the Country Programs Officer will ensure that a joint project implementation unit is established for multinational operations with representation from participating countries, as was the case of Kazungula Bridge Project. The use of joint project steering committee would ensure that most of the project issues are resolved at the highest level for speedy implementation of planned activities. (ZMFO, Sector Departments, Ongoing) In situations where joint financing of projects are required for Multinational operations, the Procurement Officer will ensure that procurement modalities are harmonised during project preparation. This is to avoid multiplication of reviews or approvals by different lenders and participating countries. As much as possible contract packaging will be aligned to total contract financing where adoption of one common procurement rule is not feasible. (ZMFO, ORPF, Ongoing) An IDEV Country Strategy Evaluation

30 22 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Introduction This report presents a summary of findings, conclusions and recommendations from an evaluation of the Bank's Country Strategies and Program in Zambia over the period of This evaluation was conducted for two purposes: (i) to inform the development of the Zambia Country Strategy for ; and (ii) to support IDEV's Comprehensive Evaluation of the Bank's Development Results (CEDR). This evaluation seeks to: (i) provide an evidence-based assessment of the relevance and performance of the Bank's strategic interventions in Zambia; and (ii) identify findings, conclusions and recommendations to inform operations going forward. The evaluation covers all lending and non-lending operations approved during

31 Background 23 Background Economic Context During , Zambia's economy was stable its mineral resources were well developed and the world market was generally favorable. In 1973, oil prices rose sharply followed by a drop in copper prices. At that time, Zambia relied on copper for more than 90 percent of its export revenue. 1 The result was continuous budget deficits and numerous external financing arrangements leading to serious economic distress. Throughout the 1980s, macroeconomic management was characterized by increasing state control of all industries, including the mining sector through the creation of Zambia Consolidated Copper Mines (ZCCM). Under state control, copper production decreased considerably because of poor investment in the sector, falling from over 700,000 tons in the early 1970s to just 256,884 tons in Zambia's economic woes were compounded in 1987 when slow progress in implementing an IMF reform agenda led to ineligibility for further IMF support and a discontinuation of finance from bilateral donors. By 1990, as a result of heavy non-concessional borrowing, Zambia had the highest debt to GNP ratio in the world at 225 percent. 3 Liquidity problems resulted in very high levels of arrears (USD 2.3 billion), including substantial arrears to multilateral finance institutions. Following a political regime change in 1991, Zambia continued to implement IMF reforms, leading to a resumption of large aid inflows. However, the economy remained undiversified and continued to perform poorly, registering growth as low as 0.3 percent in the 1990s with inflation rising to 70 percent. 4 As a result of rapid population growth, Zambia experienced a reduction in real per capita income over the period. In the 2000s, Zambia's economic situation improved with increased investment in mining, manufacturing, tourism and agriculture. With the sale of ZCCM completed, Zambia became eligible for debt relief under the Highly Indebted Poor Countries (HIPC) initiative. After reaching the HIPC completion point in 2005, Zambia's debt was reduced from USD 7.1 billion to USD 4.5 billion. Under the Multilateral Debt Relief Initiative, the debt stock was further reduced to USD 0.5 billion by the end of Zambia has demonstrated strong economic growth over the evaluation period, with average GDP growth of six percent between 2004 and 2014, reaching a high of 7.6 percent of GDP in The economy has been driven by the service sector, which accounts for 60 percent of GDP, followed by industry at 31.3 percent and agriculture at 8.6 percent. 7 Whereas the agriculture sector accounts for 85 percent of total employment, the mining sector accounts for 77 percent of Zambia's export revenue. 8 Between 2004 and 2010, Zambia benefitted from both a steady rise in copper prices and a reduction of gross government debt. The HIPC Program and Multilateral Debt Relief Initiative saw Zambia's gross government debt fall from 78 percent of GDP in 2004 to 27 percent in Economic growth has since slowed, with GDP growth just 3.6 percent in 2015 and projected to remain close to three percent in After reaching a record high in 2011, copper prices fell more than 34 percent by This fall in copper prices coincided with a loosening of fiscal management as the annual deficit returned to 10.6 percent of GDP in External borrowing has filled the gap, with gross government debt returning to 35 percent in As a result, An IDEV Country Strategy Evaluation

32 24 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report the Kwacha depreciated sharply, losing 61 percent of its value in the first 11 months of To add to these difficulties, Zambia has been experiencing a power crisis because of a considerable reduction in water levels in hydro-electric power stations caused by poor rains. The resulting power outages have taken a particular toll on the mining and manufacturing sectors. 13 Mining companies are looking to scale back high-cost investments. In Zambia, this has translated into the delay of new investments and job losses (estimated at 7,700 in 2015). 14 As such, Zambia's medium-term economic prospects are subject to a degree of uncertainty due to some key domestic risks, that is: (i) the expectation that copper prices will remain weak until 2018; (ii) a potential worsening of the power shortage due to delayed investment; (iii) failure of the government to control the growing public debt and deficit; and (iv) the potential for poor harvests to increase food prices which would place pressure on rural households. 15 Social Context In 2014, the estimated population of Zambia was 15,721,343, with an annual growth rate of 3.1 percent and an average life expectancy of 60 years. The population is evenly split between men and women but is predominantly young with roughly half of the Zambian population under the age of Overall, less than half of Zambians have completed primary and secondary education with levels of educational attainment among women lagging behind that of men. Furthermore, whereas 83 percent of Zambian men are literate, just 67 percent of Zambian women are literate. 17 Some progress has been made in recent years against key health indicators such as maternal, child and infant mortality but progress has been insufficient to meet the Millennium Development Goal indicators, presented in Annex G. Infant mortality has decreased from 107 to 45 deaths per 1,000 live births during Similarly, under-five mortality has decreased from 191 to 75 deaths per 1,000 live births over the same period. 18 Although progress has been made in reducing the prevalence of HIV/AIDS, it remains a serious development challenge: an estimated 13.3 percent of Zambians aged are HIV positive. 19 The majority (67 percent) of Zambians live in rural areas. Geographical inclusion is a serious issue in the country with most social indicators demonstrating an urban/rural divide. Whereas 90 percent of urban households have access to an improved source of water, the same is true for only 47 percent of rural households (national average of 65 percent). Similarly, 56 percent of rural households and 36 percent of urban households lack access to improved sanitation facilities. 20 Development Challenges Poverty Reduction and Inclusive Growth Despite Zambia's transition to lower-middle income country status in 2011, overall poverty levels have remained high at an estimated 60.5 percent. During , limited progress was made in reducing overall, rural and urban poverty. 21 Zambia is a relatively unequal society, with a Gini Coefficient of 57.5, ranked at 145 of 187 countries. 22 Furthermore, poverty in Zambia is overwhelmingly a rural phenomenon. Whereas urban poverty levels are estimated at 35 percent, rural poverty levels are estimated to be as high as 74 percent. 23 Part of the reason economic growth has not translated into poverty reduction is the fact that job creation has been concentrated among a small segment of highly skilled workers in the industrial and mining sector. Whereas just six percent of the labor force works within the industrial sector, the vast majority 85 percent works in the agricultural sector. Rural incomes in agricultural sector have been relatively stagnant since

33 Background 25 Regional Integration As a land-locked country, promoting regional integration and trade is particularly important to Zambia's development. Although many gains have been achieved, including the introduction of a onestop border post with Zimbabwe in 2009, the quality of infrastructure and inefficient border administration continue to increase trade costs. The International Growth Centre U.K. found that on average, it requires 44 days and USD 2,678 for Zambian firms to export a standard container of goods by ocean transport. 25 These high costs limit global competitiveness and discourage the entry of new firms, particularly in the manufacturing sector. Access to Finance The Zambian private sector is divided between large enterprises and MSMEs, with each group facing very different operating challenges. Industry surveys indicate that just a few thousand large businesses produce the majority of Zambia's industrial output and contribute the majority of tax revenues. The vast majority of businesses, however, are tiny, informal owner-operated MSMEs involved in rural agricultural production. 26 Of the 4.1 million Zambians who are employed, 88 percent are estimated to work for these small enterprises. 27 There is also a gap between larger enterprises and MSMEs in terms of access to formal finance. Only 11 percent of MSMEs use transactional products, such as bank accounts and money transfers, compared to 97 percent of large enterprises. Furthermore, only 2.3 percent of MSMEs were found to use formal credit products compared to 45 percent of large businesses. 28 Increasing access to finance for small enterprises faces numerous barriers. First of all, few banks possess branches in rural areas. Furthermore, it is estimated that 95 percent of MSMEs are not registered with any government authority, restricting access to formal finance. 29 In addition, most SME clients do not possess adequate collateral or financial documentation to qualify for a loan. 30 However, low productivity and the high cost of financial services are perhaps the most serious barriers it is estimated that only 7-8 percent of MSMEs generate enough revenue to afford a loan from a formal financial institution, which can carry interest rates as high as 20 percent and typically require collateral of up to 200 percent of the loan value. 31 Vulnerability to Exogenous Shocks Zambia's economic dependence on copper poses a serious economic challenge. During , copper exports represented an increasing share of Zambia's export revenue, accounting for 62 percent in 2000 and 78 percent in This increase is attributed to a rise in copper prices and a three-fold increase in copper production. Lack of economic diversification makes Zambia's economy vulnerable to fluctuations in commodity prices, as demonstrated by the most recent economic downturn. As copper prices fall, annual revenue and the value of the Kwacha follow suit, frustrating development planning and macroeconomic management. An IDEV Country Strategy Evaluation

34 26 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Overview of the Bank's Country Strategies and Portfolio Overview of Strategic Priorities This evaluation covers three Country Strategy Papers (CSPs): (i) the CSP, extended to 2006; (ii) the Joint Assistance Strategy for Zambia (JASZ); and (iii) the CSP. The Bank's strategic priorities for Zambia have evolved considerably over the evaluation period. The strategy emphasized agricultural development, access to water supply and sanitation and the promotion of child welfare. Under the JASZ, strategic emphasis on infrastructure development to support the agricultural sector was retained in addition to the promotion of accountability and transparency in the management of public resources through the provision of general budget support. The CSP signals a shift in the Bank's priorities, with infrastructure development now placing greater emphasis on regional integration, particularly with regard to transport and power infrastructure. The Bank's policy-based operations evolved to target private sector regulatory reform. These activities were complemented by private sector interventions aimed at increasing access to finance. Project Portfolio During period, the Bank approved a total of 43 operations amounting to UA million, excluding cancelations. The portfolio of projects under review comprises 34 projects, four studies, three emergency operations and two technical assistance grant. 33 In addition to these national operations, there were also 19 multinational operations amounting to UA million which involved Zambia to varying degrees. The evaluation focused on national projects, examining regional projects only where there was a tangible national component. Over the years, the portfolio has grown in terms of number and amount committed (see Figure 1). Nearly 45 percent of the Bank financing over the evaluation period has been done through the ADF window, closely followed by the ADB window at 43 percent. The remaining financing has been provided through NTF (0.9 percent) and other sources (11.6 percent). In total, loans represent 93 percent of net commitments with the remaining 7 percent in the form of grants. The majority of the Bank's support by value has been in the transport, power, WSS and agriculture sectors (31.7 percent, 19.3 percent, 14.3 percent and 10.9 percent respectively). However, agriculture, water supply and sanitation and transport are prominent in terms of numbers, followed by finance, power and multisector operations (see Figure 2).

35 Overview of the Bank's Country Strategies and Portfolio 27 Figure 1: Number of operations and amount approved ( ) ,3 1,3 0,4 66,8 0,6 60,6 7,7 102,0 66,0 94,0 165,7 96,7 264, Amount (UA million) Figure 2: Number of operations and amount for sectors ( ) Environment 27, Number of operations An IDEV Country Strategy Evaluation Ind / Mini / Qua 30,37 1 Finance 43,76 6 Social 43,81 2 Multi-Sector 81,80 4 Agriculture 97,82 11 WSS 135,95 7 Power 184,49 4 Transport 302, Number of operations Amount (UA million)

36

37 Evaluation Approach and Methodology 29 Evaluation Approach and Methodology Evaluation Issues and Questions The evaluation primarily addresses four evaluation issues: relevance, effectiveness, efficiency, and sustainability. However, other strategic issues relevant to the Bank's operations are also examined including: gender, inclusiveness and green growth; quality at entry; supervision; leveraging and innovation. These evaluation issues were used to identify a total of 29 evaluation questions (Annex A). Methodology and Lines of Evidence An evaluation matrix and theory of change were developed to guide data collection and analysis, including key indicators and judgment criteria, to provide a transparent account of how IDEV has examined the achievement of results across each sector. See Annex B for the evaluation matrix and intervention logic. Evaluation issues and questions were addressed through triangulation of evidence from four different sources: (i) a review of project documents; (ii) a literature review; (iii) stakeholder interviews; and (iv) site visits. In total, the evaluation reviewed over 200 project documents and other literature, collected feedback from over 175 stakeholders and beneficiaries and conducted in-depth site visits of 13 projects. A full listing of supporting documents, interviews and site visits conducted is provided in Annex C. For reporting purposes, each evaluation issue is assessed on a six-point scale ranging from Highly Unsatisfactory to Highly Satisfactory (Annex D). There are some notable limitations to the evaluation methodology. First, it was often not possible to attribute and quantify the Bank's contribution to sector level outcomes. Furthermore, the evaluation faced challenges regarding the quality and availability of supervision data. Project Logframes were often found to lack baseline data or possess flaws in the intervention logic, such that supervision data did not credibly report on outcomes. Finally, due to time and logistical constraints, it was not possible to conduct detailed interviews with all beneficiary groups. To account for these challenges, the evaluation team sought to triangulate supervision data with evidence from other sources, including interviews, site visits, external literature and evaluative evidence from other DPs. An IDEV Country Strategy Evaluation

38 30 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Evaluation Findings Relevance The Bank's CSPs have been well-aligned with the priorities set in GRZ's national development plans over the evaluation period and have responded to changes in these priorities over time. The Bank's project portfolio has generally been aligned with Country Strategies over the period as well as with the needs of targeted beneficiaries. The CSP's emphasis on the development of economic infrastructure, including domestic and regional transport and power infrastructure, is reflective of objectives identified in the Sixth National Development Plan (SNDP), particularly with respect to increasing economic growth through removing infrastructure-related bottlenecks. Interventions targeting improved economic and financial governance reflect the priorities of Zambia's Vision 2030 and Revised SNDP with respect to increased Table 1: Relevance rating Criteria Alignment of CSPs with National Development Plans Alignment of the Project Portfolio with CSPs Alignment of Projects with the Needs of Beneficiaries Overall Rating for Relevance Rating Satisfactory Satisfactory Satisfactory Satisfactory Alignment with National Development Plans Over the course of the evaluation period, the Bank's CSPs have been well-aligned with Zambia's national development plans. Operational priorities are reflective of both national development priorities and the Bank's areas of comparative advantage. Furthermore, the Bank's CSPs have been responsive to changes in national development priorities over time. Provision of general budget support to address financial and budgetary transparency and accountability identified in the CSP Update and the 2007 JASZ reflected the GRZ's increased emphasis on good governance expressed in the Fifth National Development Plan (FNDP). access to affordable finance, streamlined business registration and licensing processes and facilitation of regional trade. Alignment of the Bank's CSPs with National Development Plans is further illustrated in Annex F. Alignment of Approved Projects with Country Strategies Barring small exceptions, the alignment of Bank projects to CSPs has been satisfactory across sectors. Despite targeting job creation and economic development, the alignment of the Bank's private sector operations has been limited by the practice of identifying private sector operations opportunistically

39 Evaluation Findings 31 under the CSP and the JASZ. Under the CSP private sector operations are better integrated into the Bank's strategic priorities. For example, "supporting access to finance for SMEs through financial intermediation" is meant to contribute to the strategy outcome of improved economic and financial governance. Social sector projects have not always reflected the strategic priorities of the CSPs. Although the CSP targeted increased access to basic services, this objective was expressed in terms of water supply and sanitation infrastructure. Similarly, the health and education components of the first Poverty Reduction Budget Support (PRBS) project also aligned indirectly to the CSP by promoting decentralization of service delivery. Two interventions in the social sector were approved under the CSP, although support to vocational training and skills development is not explicitly mentioned as an operational priority in the strategy itself. The Support to Science and Technology Education Project is identified in the CSP Logframe to address skills gaps and reduce barriers to private sector development. The health and education components of the Fourth PRBS project, however, are not clearly aligned with the Bank's CSP over this period. Alignment of Projects with the Needs of Beneficiaries Except for a few exceptions, the alignment of the Bank's projects with the needs of beneficiaries has been satisfactory. Projects in the transport, multi-, social and power sectors have addressed the alleviation of development constraints at the national or regional level. In contrast, projects in the agriculture, WSS and private sector targeted the needs of specific beneficiary groups. Alleviating National and Regional Development Constraints. The poor quality of the transport network is a serious constraint for private sector development with transport challenges adding 40 percent to the cost of production for exports. 34 The Bank's projects in the Transport Sector have sought to increase access to markets and economic growth by providing reliable, seamless transport infrastructure along regional trade routes. For example, the Nacala Corridor Road Project will connect Zambia's major productive centers to the Port of Nacala in Mozambique while the Kazungula Bridge and Chinsali-Nakonde Corridor will connect the Copperbelt to the Port of Dar es Salaam. The Itezhi-Tezhi Power Generation project addresses the ongoing power crisis in Zambia by increasing installed generating capacity. However, the fact that the Bank's investments have targeted hydropower brings up more nuanced issues. Hydropower is the cheapest means of power generation and is better suited to Zambia's current tariff structure. However, the current energy crisis is partly due to an overreliance on hydropower, exacerbated by declining water levels in dam reservoirs due to reduced rainfall. The Bank's complementary Cost of Service Study is particularly relevant to the GRZ's need to achieve cost-reflective tariffs and promote private investment in other sources of clean energy generation, including solar. For the PRBS Projects, beneficiary needs were expressed in terms of financing requirements for the implementation of the Fifth and Sixth National Development Plans. Each individual project identifies expected budget shortfalls over the period and the extent to which the shortfall would be addressed through general budget support. With respect to skills development, the Support for Science and Technology Education Project (SSTEP), responds to a request for Bank Group support from the GRZ to support a nation-wide program to create one million jobs for youth and enhance skill development in agriculture, mining, tourism and energy. An IDEV Country Strategy Evaluation

40 32 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Targeting Specific Beneficiary Groups. Each of the Bank's agricultural projects have addressed the needs of specific groups of beneficiaries in terms of increasing productivity, improving food security and reducing poverty. For example, the SIP addressed development needs arising from the resettlement of the Tonga Community during the construction of the Kariba Dam in the 1950s. Displaced farmers were engaged in rain-fed subsistence farming on marginal lands. The CWMIP selected beneficiary communities on the basis of agricultural productivity below the national average despite being endowed with favorable climate conditions, fertile soil and water resources. Similarly, beneficiary districts for Bank's WSS projects were selected based on particularly low levels of access to improved drinking water sources and sanitation facilities. For example, communities covered under the NRWSS Project were found to have average rates of 21percent and 4percent for drinking water sources and sanitation facilities, respectively. In the case of the Nkana WSS project, the intervention was meant to increase the ability of the Nkana Water and Sewerage Company to provide services to fast-growing urban centers within its area of operations. Private Sector projects were well-aligned to the needs of targeted companies and financial intermediaries (FIs), but did not always fully address the needs of SMEs. The Lumwana Mine addressed the needs of Equinox Minerals, the project sponsor, by addressing the burden of major infrastructure constraints which raise development costs for mining operations. The Bank's participation in the project also secured the participation of other lenders, allowing the investment to proceed. Meanwhile, Lines of Credit (LOC) Projects were framed in terms of the business goals of the recipient FIs, including business expansion, increased lending to SMEs in particular sectors and introduction of new SMEtargeted commercial lending products. LOC Projects and the Partial Credit Guarantee Facility (PCGF) have been less successful in targeting the needs of beneficiaries due to the absence of means to improve the terms of finance. Cost of finance remains one of the fundamental challenges for small businesses. 35 Furthermore, it was never determined whether intended beneficiaries such as microfinance institutions would meet the due diligence requirements of the targeted financial intermediaries (FIs), such that some intended beneficiaries did not access to the project funds. Effectiveness Achievement of project outputs for completed projects across each sector has been moderately satisfactory, with 78 percent of outputs delivered successfully. However, achievement of project outcomes has been moderately unsatisfactory overall, with progress demonstrated toward 67 percent of outcomes. 36 At the strategy level, the Bank's interventions have made a tangible contribution toward the development of an enabling business environment and increasing access to basic infrastructure and services. Yet limited progress has been achieved in strengthening public financial management and promoting agricultural productivity and diversity. Overall progress is rated as moderately unsatisfactory. Table 2: Effectiveness rating Criteria Delivery of Project Outputs Achievement of Outcomes Overall Rating for Effectiveness Rating Moderately Satisfactory Moderately Unsatisfactory Moderately Unsatisfactory

41 Evaluation Findings 33 Private Sector Across the Bank's private sector interventions, 87 percent of outputs were delivered as planned. Information from the project XSR, confirmed by site visits, demonstrated that all expected outputs were delivered for the Lumwana Copper Mine, including development of the mine, infrastructure improvements, and environmental and social activities. However, just 84 percent of outputs were delivered for financial sector projects. Meanwhile, for two of the LOCs, funds were not disbursed to the targeted number of SMEs so consequentially some target beneficiaries, such as microfinance institutions, did not access the funds. In the case of CETZAM Financial Services, 46 percent of the funds were used to cover operating costs outside the scope of the loan agreement. 37 Delivery of technical assistance (TA) to FIs as part of the LOC projects was more successful in terms of adding new skills and systems. This included training of business development service providers FI staff in credit assessment techniques for SMEs. These successes are worth replicating. However, the installation of LOANCOM and provision of training in environmental impact assessment were less successful due to lack of demand among targeted FIs. Expected outcomes among private sector projects included: (i) increased government revenues; (ii) job creation; and (iii) profitability of the companies supported. The Lumwana Copper Mine was expected to improve access to basic services and infrastructure in the Lumwana and Solwezi districts whereas outcomes specific to LOC projects include: (i) increased SME lending; and (ii) improved terms of finance. With respect to Government Revenues, EITI Reconciliation reports indicate that the Lumwana Mine has generated 3.5 billion ZMW in government revenues since 2011, 38 whereas subproject data suggest that the SMEs which benefited from the LOC to ZANACO and Investrust have contributed at least 1.7 billion ZMW. 39 In terms of job creation, Private Sector interventions have been linked to the creation of at least 5,700 direct jobs. Supervision documents indicate that LOC projects contributed to job creation among the targeted FIs (425 jobs) and businesses which accessed the LOC (1,007 jobs). 40 With respect to indirect job creation, Barrick has concluded local service delivery contracts in excess of USD 500 million with businesses in the Lumwana area as part of its Local SME Development Program an initiative undertaken independently as part of Barrick's Corporate Social Responsibility activities. 41 The International Committee for Mines and Minerals (ICMM) estimates that for every direct mining job created in the Solwezi area, four additional jobs have been created elsewhere in the economy. 42 Profitability of the companies supported by the Bank has been uneven. The Lumwana Mine went through a period of heavy losses which threatened continued operations in The mine returned to modest profitability in 2013 after the implementation of costcutting measures. 43 For LOC projects, subproject data demonstrated that the majority of enterprises which received loans have been profitable and, in the case of ZANACO, companies registered an average increase in profitability of 197 percent. Works implemented as part of the Lumwana Copper Mine Project have increased access to basic services and infrastructure, including roads, electricity, ICT, and health services. Since the development of the mine, access to electricity in the Solwezi area has increased fivefold. 44 For education, the teacher to student ratio has improved from 1:63 to 1:53 and secondary school attendance has increased to 45 percent, meeting the national average. 45 An IDEV Country Strategy Evaluation

42 34 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Progress achieved in Increasing SME Lending was limited by the failure of CETZAM Financial Services, for which operations were taken over by the Central Bank of Zambia in May, However, supervision reports indicate that both ZANACO and Investrust saw increases in SME lending as a proportion of the overall lending portfolio from 25 percent to 30 percent of total loans for ZANACO and from two percent to 18 percent of total loans for Investrust. LOC projects were not successful in improving the terms of finance as average loan tenor and interest rates did not differ significantly from the rest of the FIs' portfolio. Furthermore, average collateral secured under the Partial Credit Guarantee Facility (PCGF) provided to ZANACO remained high at 183 percent. Project stakeholders confirmed that it was not possible to provide financing to SMEs on terms that are different from the rest of their portfolio. Power Sector Overall achievement of outputs for the power sector is rated as 80 percent. The Itezhi-Tezhi power plant was fully commissioned in January 2016, with accompanying transmission line packages commissioned in November and December However, the implementation of the Mumbwa and Lusaka West substations were still ongoing at the end of the evaluation. The Mumbwa substation was commissioned in November 2015 through the installation of a decommissioned transformer, enabling ZESCO to fulfil its power supply agreement with First Quantum Minerals. The planned Cost of Service Study, however, has not progressed due to an unsuccessful procurement process. 47 With respect to outcomes, the Itezhi-Tezhi power project was expected to contribute to: (i) increased power generation; (ii) job creation; and (iii) increased access to basic infrastructure. With respect to power generation, the Itezhi-Tezhi Power Corporation (ITPC) reports that, subsequent to the commissioning of the plant in January 2016, initial generation levels were approximately 50 percent of the expected level due to inadequate water level in the Kafue Gorge. However, as of May 2016, favorable weather conditions have since allowed the plant to reach its full generation capacity. Implementation of the project has contributed to job creation, with stakeholders from ZESCO, ITPC and district governments confirming that members of the local community were hired for initial infrastructure works, brush clearing and erection of transmission towers. However, ensuring permanent local employment has been challenging given the skill requirements for operation and maintenance activities. Works undertaken by ITPC have contributed to increasing access to basic infrastructure in Itezhi-Tezhi district, thereby promoting private sector development. A water treatment plant and transformers have been installed for use by the community and will eventually serve as a source of revenue for the district. Local government officials in Itezhi-Tezhi noted that the power plant has attracted potential investment from large companies, including Parmalat, but this investment has been delayed so far due to the inadequacy of transport infrastructure linking the district to major commercial centers. Further infrastructure upgrades would be useful in attracting additional investment to the area. Agriculture Sector The effectiveness of the Bank's agricultural interventions are rated moderately unsatisfactory. The majority of projects did not fully deliver the planned outputs and only moderate progress was made toward expected outcomes. Completion of PRODAP outputs was rated at 34 percent, with ten percent of outputs considered to be fully implemented. For the SIP, delivery of outputs was rated to be 70 percent, with just four of six planned irrigation and outgrower schemes

43 Evaluation Findings 35 considered to be fully implemented (Nega-Nega (595ha), Sinazongwe (100 ha), Nzenga (95ha) and Buleya Malima (45ha)). To date, 200 ha of sugar at Nega Nega have been harvested in 2016 while planting of crops at Nzenga is on-going. However, improved institutional management entities were installed at the three sites and training in farming techniques, business planning, record keeping and HIV/AIDS awareness exceeded targets. All expected CWMIP outputs were delivered including: (i) value chain analyses; (ii) training of farmers in horticulture, irrigation, soil and water management; (iii) drilling of wells; (iv) establishment of demonstration gardens; and (v) training in business planning and finance. Achievement of outcomes was uneven across the agriculture projects, with limited progress toward outcomes of PRODAP. The remaining projects, however, have made progress toward expected outcomes, including increased incomes, access to markets and increased use of improved irrigation methods. In the case of PRODAP, no changes were observed for health and social outcomes, including incidence of waterborne diseases or literacy rates among the project beneficiaries. Fish production within the project area has actually decreased by 58 percent since 2004 and the rate of post-harvest losses has remained unchanged. Although a 25 percent increase in the incomes of fisherman in the area was reported, it was not clearly attributable to the project. 48 On account of these challenges the project was cancelled prematurely and a new successor project for the development of Lake Tanganyika has been launched. With respect to access to markets, the Manyonyo irrigation scheme was able to secure a partnership agreement for sugarcane with Zambia Sugar while the Nzenga irrigation scheme was able to secure a similar market arrangement with a South African outgrower company to grow chilies and potatoes. Beneficiaries of CWMIP were able to access finance to grow their businesses with 1,193 farmers obtaining loans through a partnering FI. 49 The CWMIP contributed to the increased use of improved irrigation methods among beneficiaries. Cultivated land in the project area increased by 80 percent and fewer beneficiaries relied solely on bucket and drip irrigation. In total, irrigated land in the project area increased by an average of 123 percent per household. 50 Beneficiaries of both the SIP and CWMIP demonstrated increased incomes. Annual incomes for farmers within the Buleya Malima irrigation scheme increased from 479 ZMW in 2004 to 11,363 ZMW in 2014 (2,372 percent), whereas in Nzenga, annual incomes increased to an estimated 20,000 ZMW. For the CWMIP, farmers in Mkushi, Masaiti, Kapiri Mposhi and Chingola reported that their income had, on average, increased by a factor of eight. 51 Multi-Sector and Social Sector The Bank s Poverty Reduction Budget Support (PRBS) program has a large social component in addition to reforms targeting improved Public Financial Management and private sector regulatory reform. Delivery of outputs for the PRBS was satisfactory in terms of the implementation of identified triggers and reforms. 52 Performance was particularly strong for program components which sought to strengthen transparency functions (100 percent), improve access to health and education services (95 percent) and create a business enabling environment (95 percent). However, progress was far less satisfactory for outputs related to PFM (37.5 percent), including the implementation of IFMIS and the Treasury Single Account (TSA). With respect to governance components of the PRBS Program, key achievements include: (i) new procurement policy and procurement guidelines; (ii) an e-registry of business licenses; and (iii) approval An IDEV Country Strategy Evaluation

44 36 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report of a national anti-corruption plan. The GRZ also delivered a number of outputs in the health and education sectors, including: (i) hiring of 5,000 frontline health workers; (ii) the construction of eight hospitals and 231 health posts; and (iii) the construction of 1,000 basic schools and 11,000 classrooms and the hiring of 13,000 teachers. Furthermore, releases from the Ministry of Health to District Boards increased from 13 percent to 16 percent of annual budgets. 53 Overall, the achievement of outcomes for the Bank's PRBS operations in the multi- and social sectors is rated moderately satisfactory. Progress was achieved toward strengthening accountability and transparency functions, improving the business and regulatory environment and increasing access to health and education services. However, limited progress was achieved in terms of strengthening budget credibility and execution. Progress has been achieved in strengthening the GRZ's accountability and transparency functions including anti-corruption, procurement and external audit. Implementation of the National Anti-Corruption Plan has contributed to improved scores on international indices of accountability and corruption, including Transparency International's Corruption Perception Index and the Transparency, Accountability and Corruption index of the CPIA (see Figures 3 and 4). With respect to external audit, annual audit coverage has increased to 75 percent of public expenditure and 50 percent of public entities. 54 Detailed follow-up on audit recommendations has also improved, reaching 99 percent of recommendations in However, concerns persist regarding the independence of the OAG, particularly with regard to the inadequacy and unpredictability of financial resources and insufficient staffing. 55 Progress in strengthening the national procurement system was more modest. With the implementation of the new Public Procurement Policy, the Zambia Public Procurement Agency (ZPPA) was empowered to seek written justification for the use of less competitive procurement methods and conduct inspections of the procurement activities of MPSAs. However, there continues to be a lack of data on the use of less competitive procurement methods and Zambia continues to lack an impartial complaints procedure. The Auditor General has noted that irregularities continue to be found for the majority of procurement processes due to a lack of understanding of procurement guidelines. 56 The extent of the progress achieved is demonstrated by a progression in PEFA scores on indicators PI-19 "Competition and value for money in procurement" and PI-26 "Scope, nature and follow-up of external audit (see Table 3 below). Promotion of private sector regulatory reform has been more successful. Since 2006, notable reductions have been observed in the number of days to register a business, register a property and import goods, all of which have been targeted under the PRBS Program. The number of days to export, however, has remained relatively stable (see Figure 5). Some reforms have contributed to a genuine decrease in the cost of doing business. For example, the Business Compliance License List was reduced from 517 licenses to just 150, resulting in an annual reduction in the cost of business licensing by 32 percent. 57 Table 3: PEFA Scores - Procurement and External Audit Progress on Key PEFA Indicators Indicator PI-19 Competition and value for money in procurement D+ C+ D+ PI-26 Scope, nature and follow-up of external audit B+ B B Source: PEFA Reports

45 Evaluation Findings 37 Figure 3: Zambia Corruption Perceptions Index scores ( ) Source: Transparency International Figure 4: Zambia CPIA Scores - Accountability Transparency and Corruption ( ) 6 An IDEV Country Strategy Evaluation 4 3,50 4,00 4,00 4,00 4,17 4,17 4,17 3, Source: CPIA, World Bank Group

46 38 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Figure 5: Ease of Doing Business Indicators (number of days) ,5 Source: World Development Indicators Register a property Register a business Exports goods Imports goods Progress has also been achieved in terms of increasing access to health and education. Targets were achieved for increased proportion of supervised childbirths, increasing from 46 percent in 2010 to 51.2 percent in Furthermore, the number of districts with pupil to teacher ratios over 80:1 fell from 47 in 2006 to 14 in Finally, limited progress has been made in strengthening Budget Credibility and Internal Controls. Although the implementation of internal control systems, including IFMIS and the Treasury Single Account have recently made some progress, PEFA Reports indicate that there continues to be "widespread" irregularities in commitments with poor enforcement of regulations. 60 PEFA scores for the key indicators (see Table 4) suggest that budget credibility and execution remain problematic. Water Supply and Sanitation Sector Delivery of outputs among the Bank's completed WSS projects has been satisfactory at 90 percent across projects. Outputs delivered include the construction and rehabilitation of water points, boreholes, water treatment plants, sewage infrastructure and latrines. Achievement of outcomes has been similarly satisfactory.

47 Evaluation Findings 39 Table 4: Progress on key PEFA indicators Progress on Key PEFA Indicators Indicator PI-1 (i) Budget Credibility - Aggregate expenditure out-turn compared to C B D original approved budget PI-16 Predictability in the availability of funds for the commitment of D+ D+ C+ expenditures PI-20 Effectiveness of internal controls for non-salary expenditure C C+ C+ Source: PEFA The Bank's activities have contributed to improved water infrastructure in over 25 different urban and rural districts, including the construction and rehabilitation of: (i) over 8,000 boreholes and water points; (ii) over 3,000 latrines and sanitation facilities; (iii) 13 treatment plants; and (iv) multiple elements of sewage infrastructure. Furthermore, the Bank has contributed to the establishment of a commercial utility serving multiple districts. 61 Over 1,500,000 people benefitted from improved water sources implemented through the Nkana and 8 Centers and RWSSP projects, with over 95 percent of individuals in targeted areas having access to an improved water source. Furthermore, hours of service increased from an average of five hours per day to 16 hours for the Nkana project and 22 hours for 8 Centers project. With respect to sanitation, over 800,000 people gained access to improved sanitation facilities, including 9,924 individuals from unplanned settlements as well as public institutions such as health centers and bus stops. However, achievements in the sanitation components have fallen short of the targets, especially in the construction of on-site sanitation facilities. Progress in increasing collections was uneven with collection ratios declining for the Nkana project and failing to meet targets for the 8 Centers project. All of the completed projects noted a decrease in waterborne diseases with the Nkana and 8 Centers project documents reporting a 43 percent and 48 percent decrease in diarrhea and dysentery among project beneficiaries respectively. As a result of the interventions implemented under the NRWSSP, incidence of diarrhea fell from 50 percent to 35 percent across the Northern and Luapula provinces. Transport Sector All the ongoing transport projects have faced considerable implementation delays for the delivery of outputs. About 50 percent of the physical works were completed for the Nacala Corridor project by the end of September 2015, but 95 percent of the original project timeframe had elapsed. The civil works are now expected to be completed in However, implementation of the ESMP has been moderately satisfactory with compensation provided to all affected kiosk owners, construction of improved market infrastructure, drilling of nine boreholes and provision of supplies to local health centers to support ongoing HIV/AIDS sensitization. However, three of the nine boreholes provided were no longer operational. 62 The Kazungula Bridge project has delayed due to a procurement issue. A dispute over the contract award resulted in the withdrawal of funds by JICA for the bridge works. The Bank is continuing to fund other aspects of the project, excluding the bridge works, due to the issues arising with procurement. The bridge works are now underway and progress has been made in the resettlement of project affected persons. At the end of the evaluation period, however, just 0.98 percent of the project funds were An IDEV Country Strategy Evaluation

48 40 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report disbursed, 41 months subsequent to the approval of the projects. As the projects are still ongoing due to the delays a credible assessment of outcomes is not possible. Nevertheless, these project are regarded as likely to achieve their intended outcomes. Policy Dialogue and Knowledge Work Prior to the opening of ZMFO, the Bank was not able to participate effectively in regular policy dialogue with the GRZ. However, since 2006, the Bank became an active participant in the donor coordination efforts in Zambia, leading the Cooperating Partners Group troika in 2012, as well as leading the agriculture, transport and WSS working groups at various points over the CSP period. The Bank is also noted by DPs as a prominent dialogue partner for the PAF working group under the PRBS and regard it as possessing a privileged relationship with the GRZ regarding policy dialogue. Delivery of knowledge work has intensified over the evaluation period, with a total of nine works implemented or ongoing over the course of the evaluation period. There is some evidence that ZMFO has used knowledge work to inform and support policy dialogue, hosting a seminar on "Jobs and Growth" in 2014 and engaging in a High Level Policy Dialogue on youth employment in Knowledge work has also been used to add value to the Bank's operations. The Nacala Corridor and North-South Corridor Studies are informing the implementation of the Nacala Corridor Road and the Kazungula Bridge projects. The Cost of Service Study for the energy sector will help address risks posed to private sector power investments such as the Itezhi-Tezhi power project by the lack of costreflective electricity tariffs. Delivery of knowledge work, however, has often not corresponded with items planned in the CSP (see Table 5 below). Achievement of Strategy Outcomes The strategy-level outcomes reflective of the CSPs for Zambia include: a) creation of an enabling business environment; b) transparent and accountable Table 5: Planned and Delivered Knowledge Work Proposed Delivered Public Expenditure Review None Country Financial Accountability Assessment Study on Financial Management of HIPC Resources Proposed Delivered Study for the Creation of a Free Trade Area (COMESA) Public Expenditure Review of the Agriculture Sector Nacala Corridor Studies (2010) SADC North-South Corridor Study (2010) Proposed Delivered Support for the Preparation of the Zambia Transport Sector Master Plan Domestic Resource Mobilization for Poverty Reduction Zambia Private Sector Profile Constraints Facing Woman Entrepreneurs and Cross-border Traders Gender-responsive Audit of the Land Resources and Tenure System Source: 2005, 2010, 2015 CSP Completion Reports 2013 Update of Labor Force Survey (2013) Study on Zambia's Domestic Resource Mobilization Potential (2014) Zambia Private Sector Profile (2015) Zambia Manufacturing Study (2015) Preparation of a Multi-modal Transport Sector Master Plan (ongoing) Energy Cost of Service Study (ongoing) Study of Zambia-DRC Trade (ongoing)

49 Evaluation Findings 41 public financial management; c) increased access to basic services; and d) increased agricultural productivity and diversification. The available evidence suggests that the Bank has made a tangible contribution to improving the business environment and increasing access to basic infrastructure and services. However, limited progress has been achieved with respect to strengthening PFM and promoting agricultural diversification. Improved Business Environment and ZANACO and total disbursements of just 13.5 million USD. It was unclear that these projects were successful in reaching underserved groups. Although the PCGF was successful in expanding ZANACO's agriculture portfolio, subprojects under the PCGF and the LOC were dominated by Lusaka-based businesses (65 percent and 70 percent respectively). Feedback from ZANACO confirmed that loan recipients were all identified from its existing SME clientele, such that only one PCGF recipient was identified as a first-time borrower and only two businesses were identified as women-owned. Contribution to improving the business environment was assessed in terms of: (i) ease of doing business; (ii) access to finance; and (iii) business development. These outcomes were supported directly by the Bank's interventions in the private sector as well as through the PRBS. Zambia's Ease of Doing Business ranking has improved from 92 in 2006 to 83 in Similar improvement has been observed for the CPIA Business Regulatory Environment Index, which improved from 3.0 in 2004 to 4.17 in Improvement in these scores is partially attributable to reductions in the time necessary to register a business, register a property, import and export, all of which were addressed under the PRBS program. As a reflection of the improved business environment, formal business registration has increased and intensified, with 5,318 new businesses having registered in 2007 and 10,199 in 2015 (see Figure 6). Regarding access to finance, domestic credit to the private sector (percent GDP) has increased during the evaluation period (see Figure 7). Furthermore, the proportion of private firms using banks to finance investment increased from 10.2 in 2007 to 17.2 in However, the extent of the Bank's contribution to these changes was modest, given that fewer than 150 businesses are confirmed to have accessed finance through the Bank's LOCs to Investrust Another missed opportunity is the TA provided to Business Development Service Providers and Business Associations. Whereas 15 percent of the initial 105 businesses which received the training reported to have received new financing with 11 having received loans from Investrust, 65 there was no long-term monitoring of training recipients which would identify whether the training has indeed enabled them to eventually access finance. Opportunities were also missed in increasing the impact of the Bank's large private sector investments in Zambia, namely the Lumwana mine and the Itezhi-Tezhi power plant. Although these two investments have contributed to local business development by increasing access to services and implementing local supplier development programs, additional large business investments were being prevented by the poor state of transport infrastructure connecting the project sites to major trade centers. Both ITPC and Barrick have implemented comprehensive Corporate Social Responsibility activities which are addressing development constraints in the surrounding communities. Complementary support from the Bank to upscale these activities and further address constraints to private sector development may increase the catalytic potential of the Bank's projects in terms of business development. However, such opportunities are also limited by the small number of private sector investments in An IDEV Country Strategy Evaluation

50 42 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Figure 6: No. of new businesses registered during Source: World Development Indicators Figure 7: Domestic credit to the private sector (percent GDP) ,2 16, ,2 13,5 10 8,2 9,7 10 9, Domestic Credit to the Private Sector (% GDP)

51 Evaluation Findings 43 Zambia which meet the Bank's minimum threshold for project value. Transparent and Accountable PFM Contribution to transparent and accountable PFM has been assessed in terms of: (i) gross government debt; (ii) budget deficits; and (iii) public sector wage expenditure. Between 2006 and 2011, the Bank's efforts to strengthen macroeconomic management through general budget support projects appears to have contributed to reductions in gross government debt and reduced annual deficits. However, gross government debt has since increased, reaching 35.1 percent of GDP in Similarly, annual deficits have returned to 2004 levels, reaching 10.6 percent (see Table 6 below). Furthermore, annual wage expenditure has accounted for an increasing share of GDP and, in raw terms, has increased by more than 400 percent, from 2,968 billion ZMW in 2006 to 11,897 billion ZMW in These changes coincide with a decline in scores on financial governance indices, including CPIA scores for Fiscal and Debt Policy, which each reduced from 4.5 in 2009 to 3.0 in Increased Access to Basic Services Bank's interventions in the WSS, power and social sectors have made a tangible contribution to increasing access to these basic services. National trends for access to improved water and sanitation sources have been positive, with the percentage of Zambians with access to an improved water source rising from 55 percent in 2002 to 65 percent in These gains were largely due to increased access in rural areas, minimizing the disparity between urban and rural access to water. In contrast, limited improvement has been seen in access to improved sanitation (see Figure 8). The Bank's contribution to these changes has been sizable. Based on urban and rural population data for 2002 and 2014, at the national level, 2.05 million people in rural areas and 2.15 million in urban areas have gained access to an improved water source over this period. Meanwhile, the Bank's completed projects in the water sector have benefitted over 1.5 million people. With respect to access to electricity, the Bank has contributed 120 MW to Zambia's installed generation capacity through the implementation of the Itezhi-Tezhi power project. This contribution is moderate considering total installed capacity of approx. 2,300 MW. 68 Although these levels currently outstrip demand, power is being produced far below capacity due to low water levels in dam reservoirs. The size of the current power deficit is estimated to be 500 MW. 69 There is a clear need to expedite investment in the sector. The Ministry An IDEV Country Strategy Evaluation Table 6: Progress against selected macroeconomic indicators Indicator Real GDP growth (non-oil) Gross Government Debt (percent GDP) Overall Budget Balance (percent GDP) Annual Arrears (percent of annual expenditure) Wages and salaries (percent public expenditures) Sources: IMF World Economic Outlook, World Development Indicators

52 44 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Figure 8: Access to improved water and sanitation source Water Supply Sanitation Urban Rural National average Source: World Development Indicators of Energy estimates that demand is increasing at the rate of MW per year due to growth in economic activities. Although access to electricity has increased by 11 percent since 2000, 70 the timing of the Bank's investments precludes any direct contribution. Based on feedback from ITPC and ZESCO it is expected that the project will contribute to increased access to electricity for districts surrounding the power plant, the Mumbwa substation and the Karumbila mine resulting from the construction of new public infrastructure works. However, the power generated by the plant will primarily supply the Karumbila mine, for which energy requirements are expected to outstrip the maximum generation capacity of the plant at 200 MW. National health and education outcomes identified as part of the Bank's PRBS activities include: (i) increased childbirths assisted by a skilled attendant; (ii) infant immunization rates; and (iii) primary school completion rates. Progress has been made against all of these indicators. Furthermore, the gap in primary school completion rates between boys and girls has nearly vanished (see Table 7). However, other indices for access to health services suggest that additional efforts are necessary. Zambia continues to possess less than half of the WHO recommended Human Resources for Health (HRH) at 0.06 physicians per 1,000 people and 0.77 nurses and midwives per 1,000 people. 71 Furthermore national pupil to teacher ratios were stable at 48:1 during the evaluation period. 72

53 Evaluation Findings 45 Table 7: Progress on selected health and education indicators ( ) Indicator Births assisted by a skilled attendant (percent) Children months fully immunised Primary school completion - male (percent) Primary school completion - female (percent) Source: World Development Indicators Agricultural Productivity and Diversification The Bank has sought to promote economic growth and reduce poverty by increasing agricultural productivity and trade. National statistics suggest that although overall agricultural exports have increased, little progress has been made in agricultural diversification. Agriculture as a share of total exports has decreased from percent in 2002 to 7.49 percent in However, overall exports have increased by a factor of 10, such that the gross value of agricultural exports has actually increased by over USD 600 million. 73 There is also evidence of increased land cultivation and use of improved irrigation techniques. Between 2008 and 2012, total hectares cultivated increased by 16 percent, whereas total irrigated land in Zambia increased by seven percent between 2001 and 2011, suggesting that some gains have been achieved for agricultural productivity. However, there is limited evidence of agricultural diversification away from the production of maize. The Indaba Agricultural Policy Research Institute reports that production of maize among smallholder farmers remained relatively stable between 2008 and 2012 at 60.6 percent and 61 percent of cultivated land respectively. 74 Values from the Simpson Diversity Index over this period confirm these data with Zambia assessed at 0.42 in 2004 and 0.4 in 2012, suggesting limited progress over the period. 75 The DPs reported that promoting agricultural diversification was stymied by the Farmer Input Support Program (FISP), a national program which provides subsidized inputs of fertilizer and maize seeds to smallholder farmers with crops subsequently purchased by the Food Reserve Agency. Together, these two programs are estimated to account for nearly 80 percent of expenditures by the Ministry of Agriculture and Livestock. The program has also demonstrated history of inefficiencies, including overspending, poor targeting of beneficiaries, leakages and delays which limit overall productivity and income gains. 76 Furthermore, DPs reported that the program serves as a disincentive for diversifying agricultural production away from maize. In 2015, however, the program has shifted to a voucher system, allowing Zambian farmers to select the inputs of their choice. Efficiency Overall, the efficiency of the Bank's portfolio is rated as unsatisfactory owing to considerable project implementation delays and portfolio disbursement ratios which have consistently fallen below the Bank-wide average over the evaluation period. Economic Rate of Return for the Bank's projects was not rated due to unavailability of adequate data (available for only seven projects). Timeliness The average project implementation ratio for the Bank's completed projects stands at 1.68, suggesting an eight month delay for every planned year of project implementation. Across the portfolio, implementation delays have been caused by four key challenges: (a) inadequate project design; (b) delays in achieving loan effectiveness; (c) procurement delays; and (d) complexities associated with cofinancing. An IDEV Country Strategy Evaluation

54 46 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report Table 8: Relevance rating Criteria Timeliness of Project Implementation Financial and Economic Performance Overall Rating for Relevance Rating Unsatisfactory Moderately Unsatisfactory Unsatisfactory Inadequate Project Design Inadequate project design has been particularly problematic in the agricultural and transport sector. Project design deficiencies have resulted in underestimates of project cost and required implementation time as well as time spent on "re-engineering." The SIP was designed without detailed feasibility studies. As such, the project was considerably underfunded. Original cost estimates has identified implementation costs of USD 4,000/ha but the actual cost was USD 21,000/ha. 77 Initial funds were sufficient to implement just one of six planned irrigation schemes. The project was eventually implemented over ten years with an initial extension of Bank funds to 2009 as well as additional trust funds. Board approval of agricultural projects should be preceded by detailed feasibility and design studies if project implementation delays are to be avoided. In the case of the Nacala Corridor Road Project, the original project design had considered construction with limited diversions, resulting in both delays and additional costs as the project had to be re-engineered to include necessary diversion works. At the time of the evaluation, just 50 percent of project funds had been disbursed, whereas 95 percent of the original project timeframe had elapsed. Loan Effectiveness and Disbursement Delays in achieving loan effectiveness and first disbursement were noted across the portfolio. The average delay between project approval and loan effectiveness over the evaluation period was found to be 415 days (13 months), whereas the average time from loan effectiveness to first disbursement was 175 days (five months). Overall, this timeframe exceeds standards set in the Bank's Operations Manual, which identifies a timeframe of 14 months between approval and first disbursement. However, the 2015 CSP Completion Report (CSPCR) suggests that delays between project approval and first disbursement have improved for projects approved over the strategy period with delays falling from 16 months in 2011 to 12 months in The 2013 Development Effectiveness Review for Zambia confirmed that time between approval and first disbursement had fallen to 12 months. This timeframe is in line with the average across the ADF countries but falls short of the target of 11 months identified in the report. 79 Two sectors of particular concern are the agriculture sector and the private sector. In the case of the agriculture sector, PRODAP did not reach effectiveness until a full 4.4 years subsequent to approval. Among LOC Projects, average time between project appraisal and loan signing was nearly 20 months, and 25 months between appraisal and first disbursement. For the LOC to ZANACO, it took 20 months from the time of appraisal to loan signature and 26 more months for disbursement. These delays affected the achievement of project objectives. For example, the financial position of CETZAM deteriorated considerably between the time of appraisal in 2012 and the time of loan signature in September The deterioration was not monitored over this period based on the Bank's practice of supervision only after disbursement. Had the Bank been aware of this deterioration, it may

55 Evaluation Findings 47 have been possible to mitigate the risk. Despite the fact that CETZAM was already in default with the Development Bank of Zambia, Bank staff indicated that financial records were not sufficiently scrutinized, and CETZAM was cleared for first disbursement. The Bank became aware of the extent of CETZAM's difficulties only through feedback from the firm hired to provide TA. In the case of Investrust, disbursement delays for TA resulted in the delay in improving credit assessment systems which meant that the majority of the LOCs had already been disbursed. As such, it is unlikely that sub-projects had been assessed using improved systems. Complexities of Co-financed Projects Closely related to the issue of delayed loan effectiveness and disbursement is the issue of harmonizing practices with other donors for cofinanced projects. This issue was of particular concern for two of the Bank's projects: (i) the Itezhi- Tezhi Power Generation and Transmission Project; and (ii) the Kazungula Bridge Project. Itezhi-Tezhi involves innovative financial design in which the AfDB cooperates with ZESCO and six other donors to create a special purpose vehicle for power generation. The project involved public and private sector loans from the ADF and ADB windows and the Nigeria Trust Fund. The difficulties experienced in this case arose partly from the incompatibility between the Bank's public and private sector project approval processes. Since the Itezhi-Tezhi project involved both private and public sector loans, effectiveness of the public sector loan was delayed until reconciliation of terms could be completed for the private sector loan, which occurs after approval. Project stakeholders reported that an unwillingness of all parties to compromise on their terms and conditions for the private sector loan resulted in a 29-month delay for loan effectiveness and a 36-month delay to first disbursement. Through the goodwill demonstrated by ZESCO and TATA Construction, works were expedited, limiting the overall project delay to just five months. However, these delays, partly attributable to the Bank, have come at a cost to the project beneficiaries. Expedited works have resulted in additional costs of over USD 7 million in variation orders as well as USD 1 million in interest for bridging funds. Similar issues were evident in the implementation of the Kazungula Bridge project, which highlighted potential challenges of working with bilateral partners. As an exception, the Bank chose to adopt JICA's procurement rules and differences arose between the parties with respect to the appropriate technical requirements for the bid. JICA disputed the award of the contract and withdrew their support for the bridgeworks. The resulting delay has been significant, with less than one percent of funds disbursed 39 months after project approval. Procurement Delays Procurement delays were noted for multiple projects across each sector. These delays have resulted from: (i) weak procurement capacity within line ministries; and (ii) heavy internal government approval processes. Procurement capacity within line ministries is weak owing to lack of professionalization of the procurement function, high staff turnover and an unfamiliarity with the Bank requirements. Within sector ministries staff are sent for procurement training at the Institute of Purchasing and Supply. However, this training is better suited to private sector procurement. Although ZMFO has been proactive in providing workshops for GRZ counterparts on the Bank's procurement system, these efforts are limited by high staff turnover. Project delays subsequently result from improperly prepared documents, which must be returned for revision. A second source An IDEV Country Strategy Evaluation

56 48 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report of delay is the requirement that all contracts be approved by the Office of the Attorney General (OAG). This requirement, along with the resource constraints within the OAG, were noted to have delayed project procurement by a matter of months. Financial and Economic Performance Financial and economic performance of the portfolio was assessed in terms of: (i) the disbursement ratio over the evaluation period relative to the average disbursement ratio across all Bank projects; and (ii) Economic Rates of Return at completion relative to appraisal. The overall disbursement ratio has improved over the evaluation period, from percent for the CSP to 96 percent for the JASZ and 93 percent for the CSP. As of 2008, the annual disbursement ratios for Zambia has fallen slightly below the bankwide average albeit there has been some recent improvement (See Figure 9 below). With respect to the economic return, ex-post Economic Internal Rate of Return (EIRR) figures were available for only seven projects across the portfolio. As per the calculations available in PCRs, on average, the ratio between ex-post and ex-ante EIRR was 0.96, indicating that projects came close to meeting their expected rates of return. However, specific outliers can be noted, including the PRODAP Project, for which the ratio fell at just 0.06 percent and the NRWSSP, for which the EIRR far exceeded expectations at appraisal with a ratio of Figure 9: Disbursement Ratios Zambia and AfDB ( ) 60 55, ,8 24,4 30,4 31, , Disbursement Ratio Bank-wide Disbursement Ratio Zambia Source: World Development Indicators

57 Evaluation Findings 49 Sustainability The sustainability of the Bank's interventions was assessed in terms of the extent to which risks to the sustainability of project outcomes were addressed in project design and implementation, as well as the observed sustainability of outcomes. Sustainability was found to be variable across sectors, but moderately unsatisfactory overall, suggesting that the sustainability of the Bank's operational outcomes in Zambia is not assured. noted that this shutdown reflects the continued sensitivity of the mine to changes in the operating context. Medium-term profitability of FIs targeted through LOC projects was threatened by regulatory changes imposed by the Bank of Zambia. The project design for the three LOCs did not assess the sensitivity of operating profit margins to such regulatory changes. For example, CETZAM experienced a marked decline in profitability subsequent to the Bank of Zambia's Table 9: Sustainability rating Sector Private Sector Power Sector Agriculture Sector Multi-Sector Social Sector WSS Sector Transport Sector Overall Rating Private Sector. The sustainability of private sector projects has been compromised by regulatory changes, as well as variable profitability among the targeted institutions, but it was found to be moderately unsatisfactory overall. In 2012, the Lumwana Mine incurred USD 3.8 billion in impairment charges, but returned to modest profitability in 2013 as a result of cost-cutting measures, earning USD 87 million. 80 The mine then temporarily suspended operations in December 2014 as a result of the GRZ's decision to raise royalty fees to 20 percent. It was stated that this would make the mine no longer profitable. Stakeholders at Barrick Sustainability Rating Moderately Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory Unsatisfactory Moderately Unsatisfactory Moderately Satisfactory Moderately Satisfactory Moderately Unsatisfactory introduction of an interest rate and fee cap for microfinance institutions. CETZAM previously charged an annual interest rate of 104 percent. With the cap placed at 42 percent, CETZAM could no longer retain sufficient income to sustain its business model. In addition, the quality of CETZAM's loan portfolio deteriorated, with Non- Performing Loans (NPLs) reaching 49 percent of loans. An effort to restructure CETZAM failed and by the end of 2014, CETZAM reported a Return on Assets of (-28 percent), Return on Equity of (-433 percent) and a Cost to Income Ratio of 376 percent. 81 CETZAM has been in default with the Bank since July 31, An IDEV Country Strategy Evaluation

58 50 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report ZANACO and Investrust faced similar challenges when the Bank of Zambia decided to increase statutory reserve requirements to 18 percent. 82 ZANACO has remained profitable with NPLs below eight percent, absorbing the increase by liquidating a portion of its bond portfolio and incurring a slight decrease in profits. Investrust, however, has faced difficulties. Since disbursement, Investrust's financial position gradually deteriorated, with NPLs reaching 40 percent in During , Investrust's operating income grew at 84 percent, but operating expenses more than doubled as a result of an aggressive expansion strategy. In this context, the raised capital requirement added pressure to an already difficult operating context. Furthermore, the project XSR notes that the Bank failed to identify weaknesses in Investrust's credit underwriting processes, which could have been addressed through TA. Although the LOC facility has already been repaid to the Bank, reported losses in 2013 and 2014 call into question whether Investrust's SME financing activities will be sustainable. Agriculture. With respect to agriculture sector projects, the primary risks to sustainability stem from: (i) institutional sustainability; (ii) ownership by the beneficiaries; and (iii) exogenous shocks. Overall, sustainability was found to be moderately unsatisfactory. The sustainability of the PRODAP project, notwithstanding poor performance with respect to outcomes, was limited by a failure at the outset of the project to: (i) assess the capacity of stakeholders in procurement, financial management and project management; (ii) determine the availability and capacity of local bidders and contractors; and (iii) ensure buy-in through engagement with local suppliers. For SIP, the Ministry of Agriculture and Livestock has promoted institutional sustainability by establishing companies responsible for all aspects of growing and marketing of crops as well as operation and maintenance. The African Management Services Company has been hired to provide organizational capacity development to further ensure institutional sustainability. Similarly, as part of the CWMIP, all farmers clubs were trained in group management, crop management, record keeping, water management and loan management. The project also facilitated the registration of participating farmers clubs as legal entities, expanding the range of business transactions these clubs can participate in to grow their businesses. In the case of the SIP, ownership has been promoted through consultation with beneficiary farmers throughout project design and implementation. For example, the beneficiaries were involved in selecting the sites for new irrigation schemes. However, ownership of the schemes is threatened by the uneven distribution of revenues inherent in the irrigation scheme management structure. Furthermore, potential water issues resulting from climate change have not been satisfactorily addressed for SIP. Two schemes were found to be facing operational challenges due to low water levels in Lake Kariba and, as a result, one of the schemes has gone through periods of non-functionality. Power Sector. The primary sustainability concern for the Itezhi-Tezhi power project has been the financial sustainability issues given that Zambia's energy tariffs continue to fall far below cost recovery levels. Although the operation of the Itezhi-Tezhi Power Plant is covered by a Power Purchasing Agreement (PPA) with ZESCO and supported by a guarantee from the GRZ, the lack of cost-reflective tariffs pose a serious risk to ZESCO's financial sustainability. The Energy Regulation Board (ERB) notes that current PPAs range from USD seven cents/kwh to cents/kwh whereas the average tariff is USD six cents/kwh. 83 These costs do not take into account the ongoing maintenance and investment in the transmission network, nor do they address

59 Evaluation Findings 51 system losses, theft and unpaid tariffs. Zambia's electricity tariffs fall far below those of other countries in the region (see Table 10 below). program. However, little was done to address the technical capacity gaps in line ministries responsible for the implementation of reforms. Political will to move toward cost-reflective tariffs is lacking. In February 2016, the GRZ reversed a previous tariff increase to an average of USD cents/kwh. 84 As part of this agreement, the lifeline tariff would have been extended from 100 kwh to 300 kwh, while other commercial and residential consumers would pay higher fees. Although the Bank is currently funding a Cost of Service Study to support ZESCO in proposing a tariff increase, its potential impact will be subject to changes in the political environment. Although the PPA is supported by a guarantee from the GRZ, the current macroeconomic context calls into question whether such agreements could be honored across multiple investments. In terms of constraints to the development of the sector, current tariff levels discourage the conclusion of new PPAs with private power producers and limit the viability of projects for other clean sources of energy, including solar. Multi-Sector. The poor sustainability of outcomes from the Bank's PRBS program is attributed to institutional capacity risks and insufficient ownership. These risks were exacerbated by an overestimation of the ability of MoFNP to influence the activities of other ministries. Weaknesses in technical capacity were identified as risks to project implementation for each of the PRBS projects. The risk was deemed mitigated through training provided to the MoFNP on various elements of budget planning and execution under the PEMFA Some donors suggested that, given Zambia's MIC status, continued high poverty levels are a reflection of poor ownership rather than insufficient resources. In the implementation of PRBS, ownership was found to be lacking in three areas: (i) implementation of IFMIS, ongoing since 2004; (ii) budget execution reforms; and (iii) decentralization. 85 All three items featured in a High Level Dialogue in 2009 through which the PRBS group raised concerns about ongoing delays. More recently some progress has been made in implementing IFMIS and the TSA, but development partners note that continued irregularities and slow progress raised doubts as to whether the underlying changes are substantive or merely procedural. Donors also expressed concern over variable levels of ownership across the Sector Working Groups. Although ownership of the PRBS agenda was strong within MoFNP, it was somewhat weak among the line ministries, some of which regarded the PRBS program and the Sector Working Groups as requiring that they cede autonomy to MoFNP. 86 In the latter years of the Program, the GRZ gained access to the Eurobond market on favorable terms. These funds could be obtained quickly and were subject to fewer conditions. Access to the Eurobond market was identified by other donors as an additional factor underpinning insufficient ownership of budget support targets. However, recent loans have been more expensive, 87 placing additional strain on the economy. In 2014, the PRBS Group eventually disbanded due to concerns that ownership and macroeconomic management were An IDEV Country Strategy Evaluation Table 10: Regional Tariff Levels Tariffs Zambia Tanzania Swaziland Zimbabwe US cents/kwh Lifeline kwh/ month n/a 0-50 Source: Zambia Energy Regulation Board

60 52 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report insufficient to justify budget support. Furthermore, some donors doubted the appropriateness of budget support given Zambia's MIC status. Social Sector. Although progress achieved at the outcome level under the fourth PRBS has been sustained, the mitigation of risks to sustainability at the project level was deemed moderately unsatisfactory. First, as noted above, line ministries including the Ministries of Health, Education and Local Government did not benefit from the TA provided to strengthen budget execution, project management and procurement. Political and governance-related risks identified during project appraisal were limited to the potential for a change in government in The project failed to address the misuse of funds identified in the 2009 OAG Annual Report with procurement irregularities constituting the most salient risk. 88 Ongoing initiatives were underway to strengthen the Anti-Corruption Commission, but no actions were taken to mitigate this risk within the Ministry of Health itself. Water Supply and Sanitation. The sustainability of outcomes for WSS projects has benefitted from the selection of simplified and uniform technical solutions to reduce the cost and complexity of maintenance as well as the establishment of commercial utilities and community maintenance committees to ensure continued operations. However, WSS projects have struggled to ensure financial sustainability of operations and reduce unaccounted for water. In the case of the Nkana WSS project, non-revenue water stood at 60 percent due to leakages, faulty billing and illegal activities. The 8 Centers project faced similar challenges in ensuring adequate collection of revenues and cost-recovery for water services. In the case of the NRWSSP, lack of consideration for the ability and willingness of project beneficiaries to pay for services has resulted in the failure to establish a local utility for operations and maintenance. Transport Sector. Relevant risks to sustainability of the Bank's transport sector projects include financial sustainability and institutional capacity. Institutional risks have resulted from recent management challenges identified at the Roads Development Agency, for which a 2008 OAG Report identified overprocurement and contract management irregularities, specifically with regard to high unit prices paid to contractors. 89 The report resulted in a forensic audit and dissolution of the RDA's Board in 2011while the Nacala Corridor Project was ongoing. The project addressed institutional capacity risks by providing TA for procurement, management of civil works and collection of baseline data. Independent audit and technical firms were appointed to oversee project implementation. For the Kazungula Bridge Project, these risks will be addressed through the creation of the Kazungula Bridge Authority, a separate agency which will be established after project implementation to own, operate and maintain the bridge. Financial sustainability and maintenance of the Nacala corridor was addressed through EU-funded design studies which were used to identify the most costeffective implementation option. The selection of an asphalt concrete paved carriageway is estimated to have reduced maintenance costs by two-thirds over other alternatives. In addition, the GRZ has adopted a Road Maintenance and Management Strategy ( ) which provides a methodology for road maintenance planning and encourages the use of Output and Performance Based Contracting. Funds for maintenance of the road are to be ring-fenced under the National Road Fund Agency. Crosscutting Themes Coverage of crosscutting themes, including gender, inclusive growth and green growth has been moderately unsatisfactory over the evaluation period.

61 Evaluation Findings 53 Table 11: Crosscutting Issue Rating Criteria Coverage of Gender within CSPs Gender mainstreaming across the portfolio Inclusive and Green Growth within CSPs Inclusive and Green Growth across the portfolio Overall Rating for Crosscutting Themes Rating Moderately Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory Moderately Satisfactory Moderately Unsatisfactory Coverage of Gender within the Bank's Strategies. Gender has received limited coverage across the CSP periods and has been moderately unsatisfactory with uneven coverage of genderspecific development constraints and few targeted interventions identified. The CSP and the Bank Multi-sector Country Gender Profile for Zambia (2006) identified specific gender-related issues within the development context, including increased poverty among female-headed households and lack of opportunities in education, asset ownership and economic activities. Initiatives to address these issues were to be incorporated into the Bank's projects through complementary grant funds. However, coverage of gender within the Bank's CSPs decreased after this period. No specific interventions pertaining to gender were identified under the JASZ. Discussion of gender constraints under the CSP has been limited to a lack of available resources for implementing the National Gender Policy. However, some specific gender-related interventions were identified, including building GRZ capacity to collect sex-disaggregated data, providing support for gender-responsive budget and conducting gender audits of key sectors in which the Bank is active (for example the proposed gender audit of the transport sector and inventory of women entrepreneurs). Significantly, none of the gender-specific knowledge work was eventually implemented. Gender Mainstreaming across the Project Portfolio. With respect to the mainstreaming of gender into the Bank's portfolio of projects, gender has been mainstreamed into project design for the majority of projects by identifying employment and income generating opportunities for women. However, in some cases, efforts to promote incomegenerating activities for women were limited by project design omissions, whereas other projects missed opportunities in this regard. Overall, mainstreaming of gender into project design is rated as moderately unsatisfactory. LOC projects identified female business owners as a key beneficiary group. It was expected that projects would increase lending to women-owned businesses. However, the project design did not allow for the specific targeting of women in that each loan was subject to the specific due diligence requirements of the financial institutions. Through the provision of TA, however, one financial institution developed a credit product which specifically targets female traders. Integration of gender into the Bank's agriculture projects has been weak. For example, women are noticeably absent in decision-making positions among irrigation committees established under the SIP. Furthermore, for the CWMIP, treatment of gender was limited to an initiative to collect disaggregated data, which was ultimately unsuccessful. Integration An IDEV Country Strategy Evaluation

62 54 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report of gender was more successful in the Water Supply and Sanitation sector, for which the Nkana and 8 Centers WSS projects promoted employment opportunities for women through income-generating activities, such as management of water points. The integration of gender into industrial and infrastructure projects was uneven. The Lumwana Copper Mine created employment opportunities for women through: (i) the contracting of women's groups for specific services; (ii) implementation of income-generating activities; and (iii) direct hiring and training of women for industrial jobs (for example driving trucks). Similarly, initiatives targeting the capacity development of female contractors and business operators have been integrated into the design of the Kazungula Bridge, Nacala Corridor and Chinsali-Nakonde Road projects. However, no such initiatives were considered in the case of the Itezhi-Tezhi power generation and transmission project. The experience of the Lumwana Copper Mine project suggests that more can be done to promote direct employment of women for semi-skilled jobs in the context of infrastructure and industrial projects. Inclusive and Green Growth within the Bank's CSPs. Coverage of inclusive and growth and environmental sustainability is most explicit under the CSP but decreased across subsequent CSPs over the evaluation period. Overall, coverage of inclusive and green growth within the Bank's strategies has been moderately unsatisfactory. The CSP explicitly addresses the issue of high levels of rural poverty, noting that a lack of investment in rural areas has left rural populations detached from the direct gains of growth and lacking access to inputs, services and infrastructure. These challenges were to be addressed through the Bank's agriculture and WSS interventions, as well as policy based operations targeting decentralization of governance. Environmental concerns were addressed in terms of industrial pollution in the Copperbelt region and environmental degradation caused by heavy reliance on farming in rural areas but no specific interventions were identified in this regard. Coverage of inclusive and sustainable growth is less satisfactory in the JASZ and CSP. Both strategies promote poverty reduction through the development of economic infrastructure, but the connection to reducing poverty in rural areas is not explicit. The CSP identifies the Bank's role cooperating with partners to implement the Zambia Chapter of the Global Pilot Project for Climate Change resilience and recognizes the increased pressure that climate change is placing on rural agricultural activities. However, no specific interventions are identified in this regard. Inclusive and Green Growth across the Project Portfolio. The themes of inclusive and green growth have been integrated across the portfolio through the promotion of socially and environmentally responsible investment and a reduction of disparities between urban and rural areas. Overall, integration of inclusive and green growth within the project portfolio has been moderately satisfactory. Promotion of socially responsible investment was achieved through: (i) implementation of Environmental and Social Management Plans for category 1 and 2 projects; and (ii) increasing the capacity of FIs to address environmental and social management issues in their portfolio. All Category 1 and 2 projects in the power, transport, private, and WSS sectors are subject to ESMPs, for which implementation has been satisfactory. Beneficiaries have also made additional efforts to improve their environmental performance. For example, the Lumwana Mine has successfully reduced its water usage such that, in 2013, it was the best performer among Barrick mines in terms of water used per ton of ore produced. 90 LOC projects, although deemed as

63 Evaluation Findings 55 Category 3, were complemented by TA to strengthen the content and implementation of the targeted FIs' Environment and Social Management policies. Other measures have been taken by private sector partners to promote social sustainability of projects. Both ITPC and Barrick have established trust funds to support the development implementation of local development plans in cooperation with surrounding chiefdoms and build the capacity of local governments. For example, the Lumwana Mine and ZESCO promoted local employment opportunities by working with chiefdoms to establish lists of eligible workers for temporary and permanent employment. As a result, Barrick estimates that between percent of its employees at the mine come from the local community. Furthermore, nearly all of the Bank's projects in the WSS sector have sought to reduce disparities in service delivery among urban and rural communities by targeting areas where access to water and sanitation had previously been very low. The Bank's PRBS projects in the multi- and social sectors have sought to further reduce regional disparities through decentralization of government and improved delivery of health and social services in underperforming districts. However, multi-sector projects have missed opportunities to promote green growth. Environmental sustainability was part of the PRBS Program under the 2010 PAF for which the approval and submission of an Environmental Mainstreaming Strategy and Action Plan, Climate Change Response Strategy and Environmental Management Bill were identified as targets. This element of the PAF, however, was not identified among the Bank's project objectives. Furthermore, some agricultural projects have inadvertently contributed to environmental degradation. Finally, although agricultural projects have been successful in terms of addressing inclusive growth within the project design, these projects have performed poorly in terms of managing environmental risks. As noted previously, the SIP did not adequately address impacts related to climate change such that two outgrower schemes are now facing challenges due to reduced water levels in Lake Kariba. Furthermore, the CWMIP is likely to cause environmental damage. Farmers in the Chingola, Mkushi and Kapiri Mposhi regions are gardening along dambos (a type of marshland) and perennial rivers, causing the dambos to dry up due to increased water extraction. Quality at Entry Quality at entry of the Bank's CSPs over the evaluation period was found to be moderately satisfactory. However, quality at entry for project design was found to be moderately unsatisfactory due to weaknesses in the clarity of project intervention logic as well as the realism of project design. Quality at Entry of CSPs At the Strategy level, the quality at entry of the Bank's CSPs for Zambia was assessed in terms of: (i) understanding of the evolving development context; (ii) clarity and realism of the intervention logic; (iii) selectivity and coherence of the project portfolio; and (iv) positioning and comparative advantage. All CSPs over the evaluation period are rooted in a robust analysis of the political context, macroeconomic context, dimensions of poverty, national development strategies and constraints to growth. Understanding of the country context has been underpinned by the use of consultation to inform CSP design. The design of CSP was not consultative, relying instead on its alignment with the PRSP and an opportunity provided for members of government, civil society and DPs to provide An IDEV Country Strategy Evaluation

64 56 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report feedback subsequent to drafting of the strategy. By contrast, development of the JASZ was rooted in a harmonized division of labor among the DPs, in consultation with GRZ, civil society and the private sector. Development of the CSP adopted a similar consultative approach but consultations were expanded to include emerging donors such as Saudi Arabia, China, Brazil and India. It should be pointed out that the CSP was finalized prior to the establishment of a new division of labor under the JASZ-II. Under the new division of labor, finalized in 2012, the Bank was retained as "lead" partners for Agriculture and WSS, notably at odds with the priorities that had been formulated under the CSP. 91 The clarity and realism of the intervention logic for the Bank's CSPs remained relatively weak until the CSP. The CSP did not identify a results matrix but instead provided a policy matrix which linked projects to sector-level policy objectives without identifying project outputs, outcomes or performance indicators. Similarly, the Bank did not develop a separate results framework for its interventions under the JASZ based on the intention to develop a joint monitoring framework with the GRZ which was never, in fact, delivered. By contrast, though certain challenges remain, the CSP presents a credible results framework in which specific projects are linked to strategy outcomes through measurable outputs, outcomes and performance indicators. However, an independent review of the quality at entry of the CSP found that the results Logframe continued to be impractical due to its scope and the fact that actual implementation of the framework would require onerous amounts of time and resources. The strategic selectivity and coherence of the Bank's portfolio has improved over time. The portfolio identified interventions across five different sectors. The JASZ again identified five different priority sectors under two different pillars: (i) infrastructure development; and (ii) the promotion of good governance. In contrast, the CSP narrows the Bank's strategic focus to just three priority sectors under two thematic pillars, that is: (i) Supporting Economic Diversification; and (ii) Supporting Economic and Financial Governance. Although the CSP has become more selective in thematic terms, the number of sectors covered under each portfolio has increased over time with interventions approved in just four sectors between 2004 and 2006, five sectors between 2007 and 2010, and seven sectors between 2011 and 2015 (see Table 12 below). Despite implementing activities in a broader range of sectors, the Table 12: Operational selectivity in Zambia CSPs CSP Priority Sectors (WSS, Agriculture, Social, Multi-Sector, Private Sector) Sector # Project percent Funds CSP Priority Sectors (WSS, Agriculture, Transport, Energy, Multi-Sector) Sector # Project percent Funds CSP Priority Sectors (Transport, Energy, Multi-Sector) Sector # Project percent Funds WSS Transport Energy Industry Multi-Sector Agriculture Multi-Sector WSS Transport Agriculture Finance Social Finance 0 n/a Agriculture Environment Transport 0 n/a Industry 0 n/a WSS Social 0 n/a Social 0 n/a Finance Energy 0 n/a Energy 0 n/a Multi-Sector 0 n/a Environment 0 n/a Environment 0 n/a Industry 0 n/a Source: SAP

65 Evaluation Findings 57 Bank's strategy has become more coherent since projects across a range of sectors now support a more limited number of strategic outcomes. For example, under the CSP, interventions in the finance, energy, transport, social and multi-sectors complement each other in removing barriers to private sector growth. The advantage of this approach is that it allows the Bank to address multiple facets of each targeted outcome. However, as indicated by an independent review of quality at entry for the CSP, such an approach risks dispersing resources too thinly among multiple sectors, thereby limiting the achievement of results across the sector. This risk is most apparent with the Bank's LOC projects, which have reached a relatively small number of businesses. In contrast, the SSTEP is expected to have a broader impact, reaching 4,000 youths. Quality and Realism of Project Design Overall, the quality and realism of project design was found to be moderately unsatisfactory, with issues identified across all sectors. In particular, problems were identified with regard to: (i) realism of the intervention logic; (ii) quality and realism of project design. Weaknesses identified with regard to the realism of the project intervention logic resulted from weak realism of the intended outcomes, given the scope and design of projects and lack of clarity in the intervention logic due to confusion between outputs and outcomes. In the case of LOC Projects, some targets, such as "reduced unemployment among youths 15-24" and "one percent GDP growth in targeted countries," were unrealistic given the size and scope of the projects (together, the three LOCs accounted for just USD 15 million in non-revolving funds). Furthermore, no credible project mechanisms were identified to reduce the cost of finance among subprojects. Beneficiary FIs confirmed that subprojects supported through the LOC were assessed according to their regular due diligence processes. Otherwise, the FIs may not generate sufficient profit or would be exposed to undue risk. Accordingly, there were no means of ensuring that specific underserved groups would benefit from the funds instead of the FIs' existing clientele. FIs targeted under the LOC projects also questioned why the funds were not provided on a revolving basis given the targeted number of businesses to be reached. Revolving funds would have allowed the FIs to reach a larger pool of SMEs, especially considering the long tenor of the LOCs (10 years in the case of ZANACO), against the relatively short tenor of the loans provided to SMEs (on average, less than two years). The intervention logic for the Bank's projects under the PRBS program was limited by confusion among outputs, targets, triggers and outcomes. As such, it was not always possible to determine how GRZ activities addressed the PAF targets or to identify lessons learned. Furthermore, in an effort to be selective, the Bank often limited its focus to specific areas of the PAF. However, disbursements were triggered by performance against the PAF as a whole, calling into question whether donors truly pooled resources to achieve a limited number of shared objectives. Issues with respect to the realism of project design arose from the quality of feasibility studies underpinning the project designs, resulting in an underestimation of project costs. In the case of infrastructure projects, design challenges arose due to the quality of engineering works. Quality and use of Feasibility Studies. The majority of projects in the agriculture sector were not supported by feasibility studies to assess cost and implementation arrangements. In the case of the SIP, the funds provided were sufficient to fully implement only one of the six planned outgrower schemes. In the An IDEV Country Strategy Evaluation

66 58 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report case of PRODAP, the project design and appraisal did not clearly identify specific locations for investments and did not consider the high dispersal of coastal communities. With respect to implementation arrangements, no consideration was given to the availability and capacity of local suppliers and contractors, which frustrated project implementation. In contrast, the design of WSS projects was informed from lessons learned identified through previous interventions, including the adoption of demand-driven approaches, an assessment of financial sustainability for maintenance, sensitization of communities and simplicity and uniformity in the selection of equipment. Feasibility studies were used to identify water demand, stakeholders' ability to pay and the collection of baseline data. However, the cost of improvements to sanitation infrastructure was underestimated, resulting in a low uptake of improved sanitation technology among beneficiaries and diminishing performance for these components. Furthermore, the RWSSP and Nkana WSS projects did not fully consider the ability or willingness of communities to pay for water services, as well as operations and maintenance fees, resulting in increases in unaccounted for water in the case of Nkana and lack of a sustainable operations and maintenance structure in the case of the RWSSP. Engineering Works and Project Modalities. The Bank's projects in the transport sector have been supported by detailed studies which identify cost-effective solutions to regional trade issues. Furthermore, projects in the transport sector have incorporated lessons learned from previous initiatives in other countries, including the importance of integrating TA and community sensitization into the project design. However, initial engineering and design work completed for the Nacala Corridor Road project was insufficient, resulting in delays as additional engineering work was undertaken. The design and implementation of the Itezhi-Tezhi power project was similarly supported by detailed feasibility studies. Other initiatives undertaken to control a range of risks include negotiation of a takeor-pay PPA and the use of Engineering, Procurement and Construction (EPC) Contracts. However, one issue noted in the design of the project is the decision to erect a 220 KV transmission line from Itezhi- Tezhi to Mumbwa, necessitating the installation of transformers at the Mumbwa substation, thereby increasing the project cost and complexity. Given that additional investment is expected in the Western Region, a 330 KV line would have facilitated future expansion. The Ministry of Energy expressed dissatisfaction with the progress achieved in the Itezhi-Tezhi power project to date owing to implementation delays relative to the complexity of the project. Given that the required infrastructure was already in place and owned by ZESCO, it was questioned why Itezhi-Tezhi was implemented as a PPP instead of a more technically complex project. Itezhi-Tezhi was regarded as "low-hanging fruit" within the GRZ's Power Sector Development Master Plan. GRZ stakeholders felt that Itezhi-Tezhi could have been implemented more efficiently as a public sector project, diverting private resources to a project for which a PPP could yield greater efficiency. Although MoFNP has established a PPP division, it was noted that its staff lacked the sector expertise necessary to identify projects for which a PPP modality can bring added value. Supervision The frequency of supervision has increased over the evaluation period. A portfolio review indicated that, between 2002 and 2004, projects were supervised, on average, once every two years. However, the 2015 CSPCR indicates that, between 2011 and 2014, the number of projects supervised twice annually grew from five to eight projects out of an ongoing portfolio of 13. Over that period, the percentage of supervision missions conducted by staff at ZMFO increased from 67 percent to 75 percent. Increased supervision has also contributed to an increase in quarterly reports,

67 Evaluation Findings 59 which were found to be available for 50 percent of the ongoing projects in 2011, and 85 percent of ongoing projects in The feedback from the government on the Bank s supervisions shows that quality of interaction with the Bank has improved over the evaluation period, particularly after the establishment of the Country Office. There are instances where supervisions have been effective in improving the quality of the portfolio by taking steps to discontinue ageing and problematic projects (for example PRODAP) by restructuring the projects, limiting the components to suit the funds available and identifying alternative sources of funding to complete all components as with the original plan (for example, Small Scale Irrigation projects). Leveraging and Co-financing Throughout the evaluation period, leveraging and co-financing have been identified in the Bank's CSPs as a means of maximizing the relatively small quantum of ADF resources available. A portfolio review indicated that approximately 55 percent of projects approved throughout the evaluation period have included some degree of co-financing from partners. Furthermore, the 2015 CSPCR notes that, over the strategy period, the Bank managed to secure co-financing of 220 percent of the original ADF allocation. With respect to the leveraging of funds, the Bank has served as the lead arranger for three projects: (i) the Itezhi-Tezhi Power Generation and Transmission Project; (ii) The Nacala Corridor Road Project; and (iii) the Kazungula Bridge. Through these projects, the Bank was able to leverage its own contribution by factors of 16, 3.68 and 1.82, respectively. The Bank has also secured co-financing from emerging donors, including India Exim Bank, the Development Bank of South Africa and the OPEC Fund for International Development. An IDEV Country Strategy Evaluation

68

69 Conclusions and Recommendations 61 Conclusions and Recommendations Conclusions The Bank s Country Strategies and Programs have been well-aligned with both national development plans and the Bank's comparative advantage. The Bank's CSP priorities have reflected contemporary national development plans, including the FNDP, SNDP and Vision The Bank's CSPs have also responded to changes in national development priorities over time. Removal of constraints to growth through economic infrastructure development and strengthening of economic governance reflects the objectives of both the SNDP and Vision 2030, but also align with the Bank's comparative advantage in infrastructure development and policy-based operations. The Bank's portfolio has become more coherent, adopting an integrated approach to development challenges. Although interventions have been approved in a greater range of sectors across each CSP period, the realism of the strategy logic has improved due to greater coherence of the portfolio. Projects across different sectors now address different facets of a limited number of strategy objectives. Development of economic infrastructure is now being addressed in terms of regional transport and energy infrastructure, trade facilitation and skills development. Strengthening of economic governance is being addressed through complementary initiatives in the multi-sector and financial sector to enhance the regulatory environment and increase access to finance. The Bank has largely delivered planned outputs, but the achievement of outcomes has been limited by project design weaknesses and delayed implementation. With the exception of the transport and agriculture sectors, the delivery of outputs has been satisfactory. Transport sector projects have faced considerable delays but project outputs are likely to be achieved. The achievement of project outcomes, however, has been less satisfactory, reflecting the weaknesses in project design, including (i) inadequate targeting of beneficiary needs; (ii) lack of realism in intervention logic; and (iii) weaknesses in project assumptions. LOC projects were unable to influence the due diligence requirements and lending practices of FIs and did not address constraints arising from the low productivity of Zambian SMEs. Agriculture projects faced challenges in implementation as project design was not supported by detailed feasibility studies, including credible project costs and concrete implementation arrangements. On the other hand, policy-based operations failed to address poor ownership of reforms and capacity constraints among line ministries. Implementation delays have also affected the achievement of outcomes. The timeliness of project implementation was found to be unsatisfactory across all sectors. Whereas progress has already been made in addressing delays to loan effectiveness and first disbursement, the ongoing portfolio has been experiencing challenges arising from multiparty co-financing arrangements. In particular, projects in the power and transport sectors are delayed due to the need to harmonies terms, conditions and procurement arrangements among multilateral, bilateral and emerging partners. As experienced An IDEV Country Strategy Evaluation

70 62 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report in the Itezhi-Tezhi power project, these delays can increase costs and prolong the achievement of outcomes. The Bank has contributed to increasing access to basic services and improving the business environment. However, opportunities for upscaling have not been leveraged. At the country level, the Bank has contributed to improving access to basic services and infrastructure, including water, sanitation, power and health. In contrast, efforts to strengthen PFM have been less successful. Notably, these gains have not resulted only from direct investments in each sector. Increased access to services and business development have also resulted from large private sector investments, including the Lumwana Mine and Itezhi-Tezhi power plant. However, opportunities to complement and upscale the initiatives undertaken by private sector actors (for example, the corporate social responsibility initiatives by Barrick Gold at the Lumwana Mine) have not been addressed. Political and governance risks are becoming an increasingly important factor in the sustainability of projects. Whereas risks to the sustainability of earlier projects implemented in the agriculture and WSS sectors stemmed from project design issues, institutional capacity and weak ownership among beneficiaries, more recent interventions are vulnerable to political and governance risks. These risks include regulatory changes implemented without consultation, irregularities in procurement, and lack of political will to address inadequate tariff regimes. For example, the sustainability of LOC projects were limited by the introduction of interest rate caps and increased capital reserve requirements, whereas investments in the power sector will face challenges due to the continued lack of cost-reflective tariffs. Whereas the Bank's activities have attempted to promote inclusive and green growth, opportunities to mainstream gender have not been fully leveraged. The Bank's projects have attempted to promote sustainable investment as well as the reduction of disparities between urban and rural areas. However, opportunities have not always been leveraged to generate business development and employment opportunities for women. LOC and agriculture projects demonstrate that such initiatives should be supported by targeted project mechanisms whereas the Lumwana Mine project demonstrates that more can be done to promote direct employment opportunities for women within infrastructure and industrial projects. Recommendations Based on the above findings and conclusions, IDEV proposes the following recommendations. 1. Continue to support private sector regulatory reform and build capacity among the accountability functions of government. Budget support and TA interventions have been successful in promoting private sector regulatory reform and strengthening of accountability functions, partly due to adequate ownership among implementing bodies. These issues continue to be relevant, particularly the need to strengthen audit and procurement capacity. Furthermore, strengthening external accountability functions can complement the implementation of financial control systems to reduce irregularities and improve public financial management. 2. Strengthen the Bank s role in donor coordination, analytical work and policy dialogue.

71 Conclusions and Recommendations 63 The Bank, with its unique positioning in Africa and among the DPs, is expected to play a key role in donor coordination, analytical work and policy dialogue. The policy dialogue needs to be supported by rigorous analytical work, which is also demanded by government in several areas of public policy including private sector development, public private partnerships, and regulatory systems with policy predictability. The donor coordination efforts are currently subdued due partly to the withdrawal of several cooperating partners from key areas of assistance including budget support. Coordinated efforts are required to address deficiencies in the monitoring and evaluation system at the country level to enable systematic reporting of results with active participation of all line ministries, and the Bank should contribute to the ongoing initiative in this regard. Finally, policy dialogue should also be implemented as a means of addressing increasing political and governance risks to the sustainability of the Bank's projects, particularly with regard to ensuring adequate consultation with the private sector to inform regulatory changes. 3. Address constraints to private sector involvement in service delivery. Private sector participation in service delivery, particularly in the energy sector, will play an increasing role in addressing Zambia's infrastructure bottlenecks. However, inadequate tariff structures and macroeconomic pressures create disincentives to private sector participation. The Bank should identify, through analytical and policy dialogue in particular, opportunities to mitigate the impact of these constraints through a range of instruments, including lending, TA for project selection and guarantees for service delivery and purchasing agreements. 4. Identify opportunities to upscale development outcomes from private sector investments. Although the Bank has helped improved access to services and infrastructure through public sector projects, private sector investments have yielded similar results through infrastructure works and corporate social responsibility initiatives. As demonstrated by the experience of the Itezhi-Tezhi Power Project and the Lumwana Mine, additional development impacts and private sector development could be realized through complementary infrastructure investments surrounding these projects. Integrating lessons from past experience into the CSP, the Bank should more systematically identify opportunities to upscale and complement such initiatives, thereby further contributing to service delivery, infrastructure improvements and business development. 5. Deepen the integration of gender in Bank s operations and engage in policy dialogue on gender with the government in collaboration with cooperating partners. Even though the Bank s analytical work on gender in Zambia had recommended specific areas for intervention, those were not implemented in earnest. In the increasingly relevant context of feminization of poverty and HIV/AIDS pandemic, gender violence, women s unequal access to education, health services, and resources including land and credit, it is imperative that the Bank take concrete steps to integrate gender in its operations with respect to design, implementation and the achievement of results. The Bank, with other cooperating partners, should engage in policy dialogue with the government and assist in the implementation of the National Gender Policy 2014 with a focus on gender An IDEV Country Strategy Evaluation

72 64 Zambia: Evaluation of the Bank's Country Strategy and Program Summary Report auditing, gender responsive national planning and budgeting, and the establishment of a system for monitoring gender outcomes at country level. 6. Promote synergies within the Bank program and in coordination with other donors between projects which improve the productivity of small businesses and increase access to finance. Increasing the pool of SMEs that generate sufficient revenue to afford formal financial products remains an obstacle to improving access to finance. Opportunities for creating linkages between interventions which increase production of high value crops, incomes and access to markets among underserved groups with LOCs or TA for SME finance should be explored. Such linkages may yield synergies in terms of access to finance, business development and economic growth. Outgrower and farmers club schemes provide a proven means of improving agricultural productivity and diversity while linking farmers to markets and increasing both incomes and access to finance. Attention should be paid to ensuring that targeted project mechanisms are included to ensure that the benefits are gender inclusive. 7. Identify means of harmonizing project implementation approaches with other cofinancing partners as well as within projects that possess both private and public sector components. Addressing regional and domestic infrastructure constraints will likely necessitate continued cooperation and co-financing. The experience of the Kazungula Bridge and Itezhi-Tezhi Power Project demonstrates the need for the Bank to work closely with other partners by identifying harmonized terms, conditions, engineering and procurement approaches for co-financed projects in order to avoid implementation delays, cost-overruns and output quality inconsistencies. Furthermore, the design and implementation of co-financed projects with both public and private components should be mindful of differences in the approval processes for these two sources of funds in order to avoid both unnecessary delays and additional costs. 8. Increase the capacity of the GRZ for project selection, design and engineering, particularly for infrastructure projects and Public Private Partnerships. Inadequate project design and engineering has frustrated the implementation of projects across several sectors, including the transport sector, for which inadequate project design and engineering can result in considerable project delays as well as procurement disputes. Assistance in implementing high quality feasibility studies, which identify reasonable project costs, would also be beneficial for the Bank's ongoing support to the WSS sector, for which underestimation of the project complexity and costs have limited the successful delivery of improved sanitation facilities for past interventions. As additional PPPs are implemented in Zambia, technical assistance should be provided to improve project selection and ensure that PPP arrangements are providing added value. Finally, greater attention should be devoted to identifying and assessing sensitivity to political and governance risks, including tariff structures, regulatory changes and ownership. 9. Identify and mitigate operational issues surrounding multinational projects aimed at regional integration. The implementation of regional projects has been subject to excessive delays caused by coordination issues at the regional level, as seen in the case of PRODAP, which was discontinued in course of implementation. Delays in the full implementation

73 Conclusions and Recommendations 65 of one-stop border post constrain the movement goods, services and people, affecting the competitiveness and economic growth of the region in general and Zambia in particular. Based on the identification of constraints, the Bank should establish linkages between the CSP and the broader regional integration strategy to ensure bottlenecks are addressed at the right level either through dialogue at country level or coordinated approach at regional level. An IDEV Country Strategy Evaluation

74