IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA IDA IDA IDA-H7940)

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Document of The World Bank Report No: ICR IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA IDA IDA IDA-H7940) ON A CREDIT/GRANT IN THE AMOUNT OF SDR 30.7 MILLION (US$50 MILLION EQUIVALENT); A FIRST ADDITIONAL FINANCING CREDIT IN THE AMOUNT OF SDR 9.5 MILLION (US$14 MILLION EQUIVALENT); AND A SECOND ADDITIONAL FINANCING CREDIT IN THE AMOUNT OF SDR 16.6 MILLION (US$25 MILLION EQUIVALENT) AND GRANT SECOND ADDITIONAL FINANCING GRANT IN THE AMOUNT OF SDR 16.6 MILLION (US$25 MILLION EQUIVALENT) TO THE GOVERNMENT OF MALAWI FOR A MALAWI THIRD SOCIAL ACTION FUND (MASF 3) APL II (LDF MECHANISM) Social Protection and Labor Global Practice Africa Region March 23, 2015

2 CURRENCY EQUIVALENTS (Exchange Rate Effective November 23, 2014) Currency Unit = Malawi Kwacha MK 1.00 = US$0.002 US$1.00 = MK 490 FISCAL YEAR July 1 June 30 ABBREVIATIONS AND ACRONYMS ADC AEC AF APL BA CAS CDD CMC COMSIP CRC CRW CSC DESC EDO EIMU ESMP ESWAP GDP GoM ICR IDA IFMIS IFR IPC ISM ISR LA LACE LAMIS LAPA LDF MAC MASAF MDG M&E MGDS MLGRD MTR PAM Area Development Committee Area Executive Committee Additional Financing Adaptable Program Lending Beneficiary Assessment Country Assistance Strategy Community Driven Development Community Management Committee Community Savings Investment Promotion Citizen Report Card Crisis Response Window Community Score Card District Environmental Subcommittee Environmental District Officer Education Infrastructure Management Unit Environmental and Social Management Plan Education Sector Wide Approach Gross Domestic Product Government of Malawi Implementation Completion and Results Report International Development Agency Integrated Financial Management Information System Interim Financial Report Internal Procurement Committee Implementation Support Mission Implementation Status Report Local Authority Local Authority Capacity Enhancement Local Authority Management Information System Local Authority Performance Assessment Local Development Fund Minimum Access Conditions Malawi Social Action Fund Millennium Development Goal Monitoring and Evaluation Malawi Growth and Development Strategy Ministry of Local Government and Rural Development Midterm Review Performance Assessment Measure ii

3 PDO PWP PWSP RAP RRP SDR SOE TST VDC UNICEF Project Development Objective Public Works Program Public Works Subproject Resettlement Action Plans Rapid Response Program Special Drawing Rights Statement of Expenditures Technical Support Team Village Development Committee United Nations Children s Fund Regional Vice President: Makhtar Diop Senior GP Director: Arup Banerji Practice Manager: Manuel Salazar Project Team Leader: Ida Manjolo ICR Team Leader: Peter Ivanov Pojarski iii

4 MALAWI MASAF III APL II (LDF Mechanism) TABLE OF CONTENTS Page Data Sheet...v A. Basic Information...v B. Key Dates...v C. Ratings Summary...v D. Sector and Theme Codes... vi E. Bank Staff... vi F. Results Framework Analysis... vii G. Ratings of Project Performance in ISRs... xiii H. Restructuring... xiv 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned...22 Annex 1. Project Costs and Financing Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Bank Lending and Implementation Support/Supervision Processes Annex 5. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 6. List of Supporting Documents Annex 7. Map republic of malawi LIST OF TABLES Table 1: Weighting of Outcome Ratings against Original and Revised Project Objectives Table 2.1: Public Works Participants by District and Public Works Cycle Table 2.2: Public Works Subprojects: Number of Projects and Outputs by Year and by Type Table 2.3: MASAF III APL II Project Direct Beneficiaries Table 2.4: Training Interventions Details Table 2.5: Studies and Evaluations Carried out under the Project Table 5. 1: MASAF 3 APLII Project PDO and Selected Key Performance Indicators Table 5. 2: PDO level indicator results table iv

5

6 DATA SHEET A. Basic Information Country: Malawi Project Name: Project ID: P L/C/TF Number(s): ICR Date: 11/23/2014 ICR Type: Core ICR Lending Instrument: Original Total Commitment: Revised Amount: Environmental Category: B Adaptable Program Loan Borrower: Malawi Third Social Action Fund (MASAF III) APL II (LDF Mechanism) IDA-44830, IDA , IDA-51420, IDA-H7940 GOVERNMENT OF THE REPUBLIC OF MALAWI SDR million Disbursed Amount: SDR million SDR million Implementing Agencies: Ministry of Finance/MASAF/LDF Technical Support Team Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 02/25/2008 Effectiveness: 03/24/ /24/2009 Appraisal: 05/05/2008 Restructuring(s): 06/30/2014 Approval: 06/20/2008 Mid-term Review: 09/12/ /09/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Closing: 09/30/ /30/2014 Satisfactory Moderate Satisfactory Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Quality of Supervision: Satisfactory Implementing Agency/Agencies: Satisfactory Overall Bank Performance: Satisfactory Overall Borrower Performance: v Satisfactory

7 C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: Yes No Satisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration General agriculture, fishing, and forestry sector General education sector Other social services Sub-national government administration 3 3 Theme Code (as % of total Bank financing) Decentralization Other public sector governance Participation and civic engagement Social safety nets E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Kundhavi Kadiresan Michael Baxter Practice Manager/Manager: Manuel Salazar Christopher J. Thomas Project Team Leader: Ida Manjolo Michael Baxter ICR Team Leader: Peter Ivanov Pojarski ICR Primary Author: Julie Van Domelen vi

8 F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) i. The Development Objective is to improve the livelihoods of poor households within the framework of improved local governance at community, local authority, and national levels. ii. Revised Project Development Objectives (as approved by original approving authority) iii. To improve the livelihoods of poor and vulnerable households and to strengthen the capacity of local authorities to manage local development iv. (a) PDO Indicator(s) Indicator 1: Indicator Value (quantitative or qualitative) Date achieved Comments (incl. % achievement) Indicator 2: Value (quantitative or qualitative) Date achieved Comments (incl. % achievement) Indicator 3: Baseline Value Original Target Values (from approval documents) Formally Revised Target Values (Additional Financing 1) Formally Revised Target Values (Additional Financing 2) At least 120,000 households annually increase their average incomes 0 120,000/year (40% female) An additional 300,000 foodinsecure households have access to predictable income. Total: 900,000 households (40% females) 2008/2009 and 2009/2010 public works program (PWP) cycles reaching a total of 546,758 households (47% females) 1,475,125 households 1,362,444 (49%) Actual Value Achieved at Completion or Target Years 2,163,944 (51%) October 6, 2014 Fully achieved target surpassed (147%), target % of women beneficiaries surpassed (128%). Defined as number of households benefitting from public works transfers. At least 70% of local authorities were able to set objectives and achieve at least 70% of their annual targets by midterm review (MTR) 38% 70% 68% Substantially/fully achieved (97%). Indicator completed at MTR. January 23, 2012 Public works subproject (PWSP) beneficiaries with savings of at least 50% of PWSP wage one year after participation (of which % female) (added at Additional Financing 1) vii

9 Value (quantitative or qualitative) 33,750 (40% female) 48,750 78,758 (61% female) Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved target surpassed (167%). Target % of women beneficiaries surpassed (153%). Indicator 4: PWSP beneficiaries who were able to buy agricultural inputs following participation in the PWSP (of which % female) (added at Additional Financing 1) Value (quantitative or qualitative) 540,000 (40% female) 700,000 (40%) 748,725 (51%) Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved target surpassed (107%). Target % of women beneficiaries surpassed (128%). Indicator 5: Person days provided in labor intensive work in public works (added at Additional Financing 1) Value (quantitative or Qualitative) 10,800,000 26,700, million Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved target surpassed (176%). Indicator 6: Number of direct project beneficiaries (IDA core indicator 1 ) (added at Additional Financing 1) Value (quantitative or qualitative) 4,730,729 7,630, million Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved target surpassed (143%). (b) Intermediate Outcome Indicator(s) Indicator Baseline Value Original Target Values (from approval documents) Formally Revised Target Values (Additional Financing 1) Formally Revised Target Values (Additional Financing 2) Actual Value Achieved at Completion or Target Years Indicator 1: At least 70% of the annual PWSP cycles are completed according to plan Value (quantitative All cycles (9) 70% 70% 80% or Qualitative) completed Date achieved October 6, 2014 Comments 100% of planned cycles completed, with some deviations from calendars. (incl. % achievement) Indicator 2: At least 70% of the Village Development Committees (VDCs) with functional institutions address service gaps at the community level 1 IDA 14 core indicators were systematically added across the Bank s IDA portfolio to be able to report aggregate results. viii

10 Value (quantitative or Qualitative) 70% Estimate 80% of VDCs involved in implementation Date achieved October 6, 2014 Comments Fully achieved target surpassed (114%) Only those VDCs within subproject sites are active (incl. % achievement) in the subproject implementation. Framework for Intergovernmental Fiscal Transfer System to support service delivery of local Indicator 3: authority defined and ready for testing by MTR System approved by Value Cabinet in Finalized and Finalized and Finalized and (quantitative March 2013, and operational operational operational or Qualitative) to be presented to Parliament for approval. Date achieved October 6, 2014 Comments Fully achieved. MASAF III APL II developed and tested the framework. Cabinet has (incl. % achievement) approved, awaiting Parliament approval for full rollout. Indicator 4: Local Authority Capital Grants System defined and ready for implementation by MTR Developed and tested, and Value (Quantitative Revised and Revised and System capacity or Qualitative) operational operational developed building grants system working, Date achieved October 6, 2014 Comments (incl. % achievement) Indicator 5: Fully achieved. There was confusion between the already existing Capital Grants Transfers System and the grant design proposed under this component. The IDA mission of January 17 to February 2, 2011 clarified that this reflects a performance-based grant system. At least 70% of the appraised Community Action Plans reflected in District Annual Investment Plans by MTR Value (Quantitative 70% 100% 100% 62% or Qualitative) Date achieved October 6, 2014 Comments (incl. % achievement) Indicator 6: Value (Quantitative or Qualitative) Partially achieved (62%). Although all local councils had plans developed during the Project, one-third had expired plans with e delays in the process of updating those District Development Plans. Community-level tracking system that delivers information annually on baselines, targeting, and utilization of wage earnings from PWSPs. To be determined at MTR Annual tracking studies undertaken Tracking studies done in 2012 and 2013 Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved. Indicator 7: At least 70% of local authorities publicly disclose expenditures by MTR ix

11 Value (Quantitative 70% 63% or Qualitative) Date achieved January 23, 2012 Comments (incl. % achievement) Substantially achieved (90%). Indicator : Rural roads rehabilitated (km) (IDA core indicator) (Added at Additional Financing 1) Value (Quantitative 18,000 km 26,500 km 105,147 km or Qualitative) Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved - target surpassed (397%). Sectoral targets were indicative as community prioritization could not be known ex ante. Indicator : Area reforested (ha) (Added at Additional Financing 1) Value (Quantitative or Qualitative) No target set No target set 18,469 ha Date achieved October 6, 2014 Comments (incl. % n.a. achievement) Indicator : Area provided with irrigation and drainage services, new/rehabilitated (ha) (IDA core indicator) (Added at Additional Financing 1) Value (Quantitative or Qualitative) No target set No target set 5,489 ha Date achieved October 6, 2014 Comments (incl. % achievement) n.a. Indicator : People with access to improved learning environment (number) (Added at Additional Financing 1) Value (Quantitative or Qualitative) 12,000 50,000 50,000 21,683 Date achieved October 6, 2014 Partially achieved. 43%, including staff houses, classroom blocks, and VIP latrines. Latrines Comments were scaled down due to the existence of a sanitation project working in the same (incl. % achievement) areas, resulting in the target being no longer relevant. Indicator : Student classroom ratios in intervention areas with school construction (number) (Added at Additional Financing 1) Value (Quantitative or Qualitative) Date achieved Comments (incl. % achievement) 1:60 Not included in Additional Financing 2 Indicator dropped as most classrooms were replacements from earthquake destruction and would not have an impact on class size. Indicator : Classrooms built/rehabilitated (number) (Added at Additional Financing 1) Value (Quantitative n.a x

12 or Qualitative) Date achieved October 6, 2014 Comments (incl. % achievement) Substantially achieved (82%) Indicator : Teacher houses (number) (Added at Additional Financing 1) Value (Quantitative or Qualitative) 1,300 1,671 Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved target surpassed (129%). (IDA funded 808 and the Government of Malawi funded 868. Indicator : People with access to improved sanitation under the project (number) (IDA core indicator) (Added at Additional Financing 1) Value (Quantitative 6,529 No target set No target set or Qualitative) Date achieved October 6, 2104 Comments (incl. % achievement) n.a. Indicator : Safety Net Programs (PWSPs) implemented (number) (Added at Additional Financing 1) Value (Quantitative or Qualitative) 7,000 9,000 10,000 39,706 Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved target surpassed (397%) Indicator : Share of beneficiaries of PWSP complying with eligibility criteria (%) (Added at Additional Financing 1) Value (Quantitative or Qualitative) 90 n.a Date achieved October 6, 2014 Comments (incl. % achievement) Substantially achieved (87 97% achieved). 80% of beneficiaries among ultra-poor, 87% food insecure. Indicator : Beneficiaries paid in a timely manner: wages paid within two weeks of works (%) (Added at Additional Financing 1) Value (Quantitative or Qualitative) Date achieved October 6, 2014 Comments (incl. % achievement) Substantially achieved (78%), although significant improvement noted over time and final measurement found that although 70% were under 2 weeks, 98% were paid within 3 weeks. Indicator : Active micro-savings accounts (number) - % women (IDA core indicator) (Added at Additional Financing 1) Value (Quantitative 97,000 99,694 45,000 67,500 or Qualitative) (65%) (66% female) Date achieved October 6, 2014 Comments (incl. %achievement) Fully achieved target surpassed (103%) Indicator : Community savings and investment groups created that reach stage 3 (number) (Added at Additional Financing 1) xi

13 4,465 groups, Value (Quantitative 1,400 1,500 n.a. 1,106 clusters, or Qualitative) 135 cooperatives Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved target surpassed (298%) Indicator : Local authorities meeting specified criteria in financial management: non-qualified audited accounts (Added at Additional Financing 1) Value (Quantitative or Qualitative) 50% 60% 62% Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved target surpassed (103%) Indicator : Local authorities meeting specified criteria in procurement (functional Internal Procurement Committee [IPC], availability of procurement plan) (Added at Additional Financing 1) 80% have plan; Value (Quantitative 94% have 80% 80% or Qualitative) evidence of IPC consultations Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved ( % of target) Indicator : Communities satisfaction with support provided by local authorities, Area Development Committees (ADCs), and VDCs (Added at Additional Financing 1) 70% teacher Value (Quantitative 70% 70% housing and or Qualitative) 33% for PWPs Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved for teacher housing (100%), partially achieved for PWPs (47%). Local authorities with reporting mechanism for millennium development goal (MDG) Indicator : indicator targets (percentage) (Added at Additional Financing 1) Value (Quantitative or Qualitative) 70% 80% 53% Date achieved October 6, 2014 Comments (incl. % achievement) Partially achieved (66%). Outcome reporting is not yet carried out in all local authorities (LAs) Indicator : Funds lost to errors, fraud, and corruption: identified through audit exercise (%) (Added at Additional Financing 1) Value (Quantitative or Qualitative) Date achieved October 6, 2014 Comments (incl. % achievement) Full achieved Indicator : The LAs publicly disclose revenues, such as grants and expenditures by MTR (%) (Added at Additional Financing 1) Value (Quantitative xii

14 or Qualitative) Date achieved October 6, 2014 Comments (incl. % achievement) Fully achieved target surpassed (126%) Indicator : Beneficiaries who feel project investments reflected their needs (%) (Added at Additional Financing 2) 88% of public works Value (Quantitative 95 beneficiaries or Qualitative) satisfied with the PWP Date achieved October 6, 2014 Comments (incl. % achievement) Substantially achieved (93% ) Indicator : Grievances registered related to delivery of project benefits that are actually addressed (%) (Added at Additional Financing 2) Value (Quantitative or Qualitative) 75 n.a. Date achieved October 6, 2014 Comments (incl. % achievement) Indicator : World Bank and the Local Development Fund agreed that a full grievance system would be more appropriate to implement under MASAF IV than under crisis funding for Additional Financing 2, given time limitations and the emergency nature of this funding. Intended beneficiaries who are aware of project information and project supported investments (%) (Added at Additional Financing 2) Value (Quantitative or Qualitative) Date achieved October 6, 2014 Comments (incl. % achievement) Substantially achieved (94%) No. G. Ratings of Project Performance in ISRs Date ISR Archived DO IP Actual Disbursements (USD, millions) 1 04/09/2009 Satisfactory Satisfactory /18/2009 Satisfactory Satisfactory /07/2010 Moderately Satisfactory Moderately Satisfactory /13/2011 Moderately Satisfactory Moderately Satisfactory /27/2011 Satisfactory Moderately Satisfactory /27/2012 Moderately Satisfactory Moderately Satisfactory /06/2013 Moderately Satisfactory Moderately Satisfactory /01/2014 Satisfactory Moderately Satisfactory /12/2014 Satisfactory Moderately Satisfactory xiii

15 H. Restructuring In addition to the restructurings done during the approval process of the additional financings, the Project had only one restructuring. It was done just before project closing, on June 30, 2014, and was needed in order to apply some final reallocation and adjustment of amounts among components and categories. I. Disbursement Profile xiv

16 1. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN 1.1 Context at Appraisal 1. When Malawi Third Social Action Fund (MASAF III) Adaptable Program Lending (APL) II was appraised in late 2007, slightly over half of the country s 6.4 million people were living below the national poverty line and 22 percent were in extreme poverty. Poverty was more widespread in rural areas. Moreover, poverty was dynamic with high levels of transient poverty based on shocks and economic conditions. Most Malawians suffered from chronic food insecurity, relying on agriculture characterized by low yields and depending on rain-fed farming which increases vulnerability to weather-related shocks. The precarious food security situation had led to a vicious cycle of emergency appeals for humanitarian assistance. 2. Poor communities continued to face food insecurity and hunger and the lack effective access to basic services and infrastructure. For example, rural roads were often impassable, restricting access to markets. In the education sector, while substantial gains had been made in enrollment, educational infrastructure had not kept pace with these gains. Overcrowding and high student to teacher ratios affected educational quality and the lack of housing for teachers made it difficult to recruit and maintain staff in rural areas. For example, for the country s 13,000 mostly rural teachers, only 4,500 teacher houses were available, mostly in poor condition. 3. As a way to address some of these issues, the government of Malawi (GoM) strengthened its policy focus on safety nets. MASAF III APL II was in alignment with the government s poverty reduction strategy and the Malawi Growth and Development Strategy (MGDS) Social protection was one of the five key themes prioritized in the MGDS. The MGDS also acknowledged a central role for good governance, including improving service delivery and accountability at the local level through decentralization. 4. The World Bank s Country Assistance Strategy (CAS) (FY ) for Malawi, approved on January 4, 2007, supported the MGDS goals. MASAF III APL II aimed to contribute toward CAS outcomes by targeting poor and vulnerable households 2 through its community livelihoods support component (including public works activities as a social protection mechanism) and enhancing local authority (LA) capacity (to prepare for devolution of responsibilities and resources). It also sought to strengthen the operational links between the central and local governments through an improved fiscal architecture and integration of MASAF systems into the local governance system. 5. The rationale for Bank assistance was also built on the Bank s long involvement in social protection and community-driven development in Malawi, and in particular as a financier of the MASAF since its inception in In the period , the government invested, through the MASAF, a total of US$122 million in community infrastructure (education, health, water supply and sanitation, markets, roads, and bridges primarily). The Bank was well placed to continue to help the organization engage more fully in developing effective safety nets in Malawi, both for its 2 The National policy framework was laid out in the National Social Support Policy (NSSP) and identified three categories of the poor and vulnerable to target for assistance: (i) The moderately poor: 25% (3.5 million people); (ii) The ultra-poor with labour 5% (0.73 million people) and (iii) The ultra-poor and incapacitated:10% (1.5 million people). 1

17 historic involvement in the MASAF as well as its international and regional expertise in social protection and safety nets. 1.2 Main Changes in Country Context during Project Implementation 6. The project operated through a difficult period for the country, during which Malawi was facing difficult economic, fiscal, and poverty challenges. In that context, the project, which was designed to support the backbone social safety net program in the country, played a particularly important role. Turbulent period during : During this period, Malawi faced multiple shocks: adverse terms of trade, significant reduction in tobacco proceeds, governance setbacks, drop in donor inflows (especially budget support), and these were combined with inappropriate policy responses. These resulted in fiscal and external imbalances, and the severe foreign exchange shortages were beginning to choke the economy and shortage of fuel and other essential commodities. All this along with the falling domestic currency hit the poor. The new President acted swiftly to correct some of these concerns through policy actions in Cashgate and the macroeconomic position: In September 2013, a massive theft of public resources by a group of government and non-government individuals was exposed. In consequence, Malawi s fiscal situation deteriorated rapidly due to stoppage of aid flows and declining private sector confidence and the country s global reputation. This in turn led to a significantly enlarged budget deficit, high-cost domestic borrowing and resurgent inflation, stalling earlier macroeconomic gains and their beneficial contributions to poverty reduction. The suspension of budget support has dominated donor dialogue with the Government for the past year and a half. The loss of donor budget support following cashgate cost the equivalent of 10 percent of GDP. Poverty and inequality remain stubbornly high in Malawi. New official poverty estimates will only become available in 2017, but the expectation is that the trends will not differ dramatically from the findings of the Third Integrated Household Survey of 2010/11 which showed that over half of the population was monetarily poor and one quarter lived in extreme poverty. Between 2004/05 and 2010/11, national poverty levels declined only marginally (from 52.4 to 50.7 percent of the population), and although poverty fell significantly in urban areas (from 25 to 17 percent) it increased slightly (from 56.2 to 56.7 percent) in rural areas, where 85 percent of the population and 95 percent of the country s poor live. Rural income inequality also increased from 2004/05 to 2010/11, with the Gini coefficient worsening from 0.34 to Despite real GDP growth rates averaging 7 percent and household per capita consumption expenditure increasing by about 12 percent between 2006 and 2010, real consumption in some two-thirds of rural households actually declined. The 2013 Integrated Household Panel Survey shows that about a third of the population have either exited or entered poverty and in roughly equal proportions -- but the reasons for these movements, as well as for the persistence of rural poverty, are imperfectly understood. Recognizing this, the new Government has asked for WBG help to better understand the drivers and dynamics of rural poverty, and to reframe its anti-poverty strategy. A key obstacle to reducing poverty is low agricultural productivity. The majority of the poor remain locked in low productivity subsistence farming. Access to infrastructure, services, land, working capital, and market opportunities are all limited, but Malawi s 2

18 current strategy for increasing agricultural productivity centers disproportionately on fertilizer and seed subsidies, particularly for maize (through the Farm Input Subsidy Program, FISP). FISP, however, is not poverty targeted and the efficiency of its management is questionable. Addressing these subsidies is likely to have an impact on income diversification and household earnings. 1.3 Original Project Development Objectives (PDO) and Key Indicators 7. The project development objective (PDO) was to improve the livelihoods of poor households within the framework of improved local governance at community, local authority, and national levels. 8. The results framework included two key performance outcome indicators: a) At least 120,000 households annually increase their average incomes b) At least 70 percent of local authorities (LAs) are able to set objectives and achieve at least 70 percent of their annual targets by Midterm Review (MTR) 9. Output indicators were: a) At least 70 percent of the annual public works subprojects (PWSPs) cycles are completed according to plan b) At least 70 percent of Village Development Committees with functional institutions address service gaps at the community level c) Framework for an Intergovernmental Fiscal Transfer System to support service delivery of the LA defined and ready for testing by MTR d) Local Authority Capital Grants System defined and ready for implementation by MTR e) At least 70 percent of appraised Community Action Plans reflected in District Annual Investment Plans by MTR f) Community-level tracking system that delivers information annually on baselines, targeting, and utilization of wage earnings from PWSPs g) At least 70 percent of the LAs publicly disclose expenditures by MTR 1.4 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 10. An Additional Financing (AF) project reworded the PDO to make it more concise and measurable without changing its substance. The revised PDO was to improve the livelihoods of poor and vulnerable households and to strengthen the capacity of local authorities to manage local development. 11. The changed PDO did not substantively change the objectives of the project, but clarified some aspects, including areas for which the project was to be held accountable. While the project was conceived at a time when the Government was pushing plans for decentralization, and the project was seen as an instrument that would function well in such an environment, it had not been designed or intended as a decentralization project. The revised PDO therefore put the correct emphasis on improving the livelihoods of the vulnerable, and on building the capacity of the local authorities. 3

19 12. Four outcome indicators were added with AF 1: a) PWSP beneficiaries with savings of at least 50 percent of PWSP wage one year after participation (of which percentage female) b) PWSP beneficiaries who were able to buy agricultural inputs following participation in the PWSP (of which percentage female) c) Person days provided in labor intensive work in public works d) Number of direct project beneficiaries (International Development Agency [IDA] 14 core indicator 3 ) 13. Several additional results indicators were added to track progress on component outputs: a) Rural roads rehabilitated (km) (IDA core indicator) b) Area reforested (ha) c) Area provided with irrigation and drainage services, new/rehabilitated (ha) (IDA core indicator) d) People with access to improved learning environment (number) e) Student classroom ratios in intervention areas with school construction (number) f) Classrooms and teachers houses built/rehabilitated (number) g) People with access to improved sanitation (number) (IDA core indicator) 4 h) Safety Net Programs (PWSPs) implemented (number) i) Share of beneficiaries of the PWSP complying with eligibility criteria (percentage) j) Beneficiaries paid in a timely manner: wages paid within two weeks of works k) Active micro-savings accounts (number) - percentage women (IDA core indicator) l) Community savings and investment groups created that reach stage 3 (number) m) The LAs meeting specified criteria in financial management n) The LAs meeting specified criteria in procurement (functional Internal Procurement Committee [IPC], availability of procurement plan) o) Communities satisfaction with support provided by the LAs, Area Development Committees (ADCs), and Village Development Committees (VDCs) p) VDCs/ADCs actively involved in overseeing subproject implementation to address service gaps at the community level (percentage) q) The LAs with reporting mechanism for Millennium Development Goal (MDG) indicator targets (number) r) Funds lost to errors, fraud, and corruption: identified through audit exercise (percentage) s) The LAs publicly disclose revenues, such as grants and expenditures by MTR (percentage) 14. AF 2 left the PDO unchanged. Additional results indicators were added on component outputs: a) Beneficiaries who feel project investments reflected their needs (percentage) b) Grievances registered related to delivery of project benefits that are actually addressed (percentage) 3 IDA 14 core indicators were systematically added across the Bank s IDA portfolio to be able to report aggregate results. 4 Ibid. 4

20 c) Intended beneficiaries who are aware of project information and project supported investments (percentage) 1.5 Main Beneficiaries 15. Food-insecure households are the main beneficiaries of public works programs (PWPs) under MASAF III APL II. The PWSPs provide a safety net mechanism for those who are poor and with limited employment opportunities. The PWSPs are targeted at poor households as an annual intervention to address food security-related shocks that affect these households around the time they need cash to purchase agricultural inputs, grains, and other basic necessities. 16. Poor communities were the main beneficiaries of the infrastructure and services created through the PWSPs and community subprojects. AF 1 included restoring basic services and infrastructure in communities affected by the 2010 earthquake. 17. The institutional objectives of the project focused on building the capacity of Malawi s 35 LAs as well as VDCs and ADCs. Additional beneficiaries include national-level institutions associated with the capacity development of the LAs, such as the Ministry of Local Government and Rural Development, and sectoral ministries. 1.6 Original Components 18. MASAF III APL II is the title of the Bank-financed project implemented through the Local Development Fund (LDF), managed by a Technical Support Team (TST). The project comprises of three components in support of the three windows of the LDF. a) Community Livelihoods Support Fund with two subcomponents: (i) Local Authority Fund: Based on local councils annual investment plans produced through village participatory planning, this subcomponent finances labor-intensive subprojects in afforestation, village access roads, small gravity irrigation projects, watershed management, soil conservation, solid waste disposal, water drainage systems, small-scale irrigation projects, and gravity-fed water supply. Individuals who will participate in the various PWSP are selected based on community-based targeting. At the outset, poor households received twelve days of work at MK 200 per day to address food security-related shocks that affected them around the time they needed to purchase agricultural inputs, grains, and other basic necessities. (ii) A Community Fund supports participatory planning processes and direct community financing to improve the functionality of existing educational facilities, in particular the construction of teachers houses and additional classrooms. This subcomponent also entails interventions to promote a culture of savings and investments (Community Savings Investment Promotion [COMSIP]), in particular among participants of the PWP, to enable them get on a pathway to graduating out of poverty. Voluntary savings groups received training to engage in savings and revolving credit. b) Local Authority Capacity Enhancement Fund supports the development of a comprehensive framework for addressing capacity needs for the LAs to effectively manage the grants they receive, perform functions allocated to them, and prepare them to perform anticipated 5

21 responsibilities as devolution and decentralization proceeds. This includes developing capacity in the areas of planning, financial management, and human resource management that would enable them to receive and manage development block grants as well as implement priority programs such as the PWSP on behalf of the national government. c) National Institutional Strengthening Fund supports national-level cross-cutting issues aimed at improving accountability and transparency in the use of project resources. This includes: (a) a TST to oversee project implementation under the umbrella LDF and (b) knowledge generation and application, including monitoring and evaluation (M&E), as well as the preparation and dissemination to strengthen community participation in project implementation and document community experiences with service delivery. 1.7 Revised Components 19. There were no changes in the components or other design features of the original project. All AF was channeled through the original components. 1.8 Other Significant Changes 20. There were two AF projects. (a) AF 1 (US$14 million from IDA funded from the crisis response window [CRW]) was approved on June 15, 2010 in response to the impact of the global economic crisis and natural disasters. The AF aimed to (i) develop, implement, and monitor a scaled-up public worksplus-savings and investment program to help alleviate the poverty, social, and economic impact of the crisis and (ii) provide financial assistance to protect core spending on education infrastructure investments through the reconstruction/retrofitting of education structures damaged in the Karonga and Chitipa earthquakes. (b) AF 2 (US$25 million grant and US$25 million credit) was approved on July 12, 2012 under the Rapid Response Program (RRP), as part of an overall US$150 million stabilization package which included additional support for MASAF III APL II, general budget support, and AF for an irrigation, rural livelihoods, and agricultural development project. The government had taken bold economic reforms in early 2012 after a change in leadership, including liberalizing Malawi s foreign exchange regime and significant fuel and power price adjustments. The resulting inflation had severe effects on the poor, particularly due to rising food costs. 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.1 Project Preparation, Design and Quality at Entry 21. Design factors. The project built on the experience of the long institutional trajectory of the MASAF as well as the more immediate lessons learned from the implementation of the first phase of MASAF III (APL I). 22. Given the relatively uncertain pace and status of decentralization in Malawi, especially in the light of the stalled political decision on local elections, APL II did not propose to finance comprehensive policy reforms but rather lay the groundwork for the future using its community 6

22 driven development (CDD) approach to address synergies between communities and the local government. In the short term, the transfer of project resources to the LAs to manage public works stimulated and enhanced managerial capacities at that level, laying the groundwork for future decentralization. 23. Previous experience showed that public works cash transfers as part of drought response had a major impact on community livelihoods and food security, solidifying the growing acceptance of the need for social protection measures in response to shocks. An operational base had been established, permitting a scale-up of coverage and impact. 24. Previous phases of the MASAF had financed an extensive expansion of community-level infrastructure (schools, health facilities, and water points), but the shortage of skilled personnel and supplies reduced the impact of such investments. APL II focused less on creating new infrastructure and shifted from an open menu approach to targeted infrastructure investments mainly for building teacher housing which would not significantly increase recurrent costs but would have an impact on retaining teachers in rural areas and improving educational quality. 25. Assessment of risks. Project preparation identified a number of critical risks. The first of these was the pace of decentralization. As mentioned above, the main strategy for controlling this risk was to focus on developing LA readiness for greater responsibilities if and when those materialize. Another key risk was the complexity of different sectors and actors involved in implementation. The need for effective collaboration was to be addressed at the central government level by a Steering Committee led by the Ministry of Finance and composed of senior officials from all the key ministries. Given the prominent role of relatively weak LAs in channeling funds, a further risk identified was the failure to implement because funds could be misapplied. The project design addressed this by demanding accountability of funds used before any further funding can be provided as well as emphasizing capacity building of the LAs. 26. Overall quality at entry. Project preparation was carried out by a joint Bank-government team that built on long-standing collaboration of social protection and CDD issues in Malawi. Design and preparation activities addressed the main issues and sought to control outstanding risks. The project was prepared in a short time frame, about six months, to respond to the immediate needs of the population for social protection measures and because it could build on the base from APL I. The project components were responsive to the country context and balanced between seeking to deliver direct benefits to the poor and longer-term institutional goals. 2.2 Implementation 27. Project funding and time frame. The original US$50 million project was approved by the Board on June 20, 2008, signed on November 25, 2008, and became effective on March 24, AF 1 (US$14 million) was approved on June 15, 2010 and AF 2 (US$50 million) on July 5, 2012, bringing the total IDA envelope to US$114 million with an additional US$1.7 million in community contributions. The original closing date of September 30, 2013 was changed to June 30, 2014 at the time of AF 2 and the project closed on that date. 28. Project implementation issues. Project implementation was rated Satisfactory or Moderately Satisfactory throughout implementation. All issues that arose during implementation 7

23 were largely process-oriented rather than substantive in nature. For example, there were delays in hiring of and changes to the executive director position of the TST and other personnel. Shortages and delays in recruiting district-level procurement and financial management staff and overall high turnover and reassignments at the district level were issues identified throughout implementation. 29. Project implementation was also affected by external factors. Economic shocks and natural disasters increased the level of vulnerable households and prompted the need for the AF to satisfy these urgent needs. Moreover, the project lost a total of US$2.7 million due to appreciation of the U.S. dollar against special drawing rights (SDR). Following economic liberalization, Malawi experienced very high levels of inflation, reaching 38 percent in March This affected the costs of subproject inputs. For example, the unit cost of a teacher s house increased from US$15,000 in 2008 to US$ 22,000 in Project changes/restructuring. An AF allowed for additional funds to be called forth to address critical shocks, like economic crises and natural disasters. It also created an opportunity for formal project restructuring of the project. At AF 1, the original PDO statement was edited to clarify the main project development objective. Key performance indicators were added in both AF 1 and AF 2 to respond to IDA corporate reporting needs and to capture a broader range of outputs and outcomes. 31. Some of the safety net parameters were altered in AF 2 to enhance the ability of the poor and vulnerable households to cope with the negative effects of the economic reforms. Daily wage rates were increased from MK 200 to MK 300. The allowable number days was increased from 12 to 48 days, with beneficiaries working for 12 days per month for two consecutive months in each cycle compared to 12 days per year in the normal PWP. Coverage nearly tripled from 220,000 beneficiaries per year to 594,000 beneficiaries, and more predictable employment and income were provided by targeting the same participants over the two cycles of the program. 32. Midterm review. An MTR was conducted from January 9 to January 23, The main issues identified were (a) the need to restructure the PWP to be a productive social safety net which supports communities for more than one year to put them on a pathway to graduation (which was supported by AF 2); (b) the need to better align the PDO indicators to the PDO statement (also under AF 2); and (c) continued inadequate staffing capacity at the district level for financial management, internal auditing, and procurement staff, which required the government's urgent attention for the districts to effectively deliver on their development mandate. 33. Project closing. At project closing on June 30, 2014, 95 percent of the project funds were disbursed. The remaining balance was meant to fund a final round of public works. Initial discussions identified a needed wage increase at the beginning of 2014 to address the erosion in the real value of transfers over time. The effective purchasing power of the transfer had diminished due to the inflation following economic liberalization measures. While a wage rate increase was budgeted, the government ended up deciding not to raise rates, leaving some undisbursed funds. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 34. M&E design was built on a solid platform of previous experience in the MASAF. The M&E framework was based on a well-defined results framework and a comprehensive M&E plan with 8

24 clear roles and responsibilities. Monitoring would be supported by a management information system which extended to the LA level. Periodic assessments and evaluations were to include annual tracking studies, baseline studies, ex post evaluations, and impact evaluations. Participatory community monitoring and accountability approaches and systems using citizen report cards (CRCs) and community score cards (CSCs) were identified to ensure direct beneficiary feedback. 35. Project monitoring was consistently carried out at the local levels through multisectoral implementation support missions comprised of a team of 30 that visited the country s 35 districts every quarter, identified issues, and prepared a team report. A basic management information system was maintained, though district-level inputs could not be fully automated due to the lack of communications infrastructure in many rural districts. 36. Annex 2 contains a list of the various evaluations done throughout project implementation. The project was able to create a sufficient evaluative basis to draw conclusions about project performance. Of note: a) The MASAF developed a performance assessment system for the LAs which tracks seven functional areas of local government performance. The performance assessment measure (PAM) was tested in 2010 and then carried out in 2011, 2012, and b) A CSC was carried out in District-level personnel were trained and the process covered 14 of the country s 35 districts. The exercise was expensive in terms of personnel and logistics and turned out to be time-intensive. It was found to be a better instrument to gauge general service delivery quality than specific project outcomes and was subsequently taken up by the Ministry of Planning. c) Beneficiary assessments (BAs), public works tracking studies, technical audits, and impact assessments were contracted to external firms or carried out by multisectoral government teams. 5 The TST was able to incorporate the findings and adjust processes 2.4 Safeguard and Fiduciary Compliance 37. Safeguards. The MASAF III APL II project triggered four Safeguards Operational Policies: OP/BP Environmental Assessment, OP Pest Management, OP/BP Involuntary Resettlement, and OP/BP Forests. During MASAF III APL II, the responsibility for safeguards implementation was transferred fully to the district level. However, there arose issues of capacity to include environmental and social safeguards as part of the districts regular planning and implementation processes. Given the large number of PWSPs, District Environmental Subcommittees (DESCs) were challenged to do proper environmental and social screening and/or develop Environmental and Social Management Plans (ESMPs) for all subprojects in all communities. The COMSIP program was more successful at mainstreaming safeguard considerations into the business management aspects of the savings clubs formed. 38. To address shortcomings, several actions were undertaken. The government recruited 27 environmental district officers (EDOs) to coordinate the social and environmental activities at the LA level and to serve as secretary to the DESC. The project supported training for the EDOs, DESC, and local extension officers. Recognizing that limited follow-up on mitigation measures was in part 5 5 Draft report by the Development Economics Research Group under a working title The design of Public Works and the Competing Goals of Investment and Food Security see section 6, Lessons Learned, for more details.. 9

25 the result of a lack of earmarked funds, in 2012 the LDF allocated MK 50,000 per subproject for overall support to safeguards implementation and monitoring. The Ministry of Environmental Affairs and Climate Change began to undertake quarterly monitoring missions, which improved screening and implementation of mitigating measures. In addition, the project initiated a pilot to mainstream the use of soil stabilized bricks for building construction, as an alternative to baked bricks which are currently most commonly used in Malawi for the construction of rural buildings. 39. Procurement. At the national level, the TST was responsible mainly for the procurement of the institutional support activities. At the local level, district staff were responsible for contracting PWSPs and community-based project management committees (PMCs) for the social infrastructure subprojects. MASAF I and II had previously developed the operational systems for procurement at the local level and had proven that the local level, particularly that community PMCs were well placed to carry out simple procurement responsibilities. Overall, procurement functions were carried out in a satisfactory manner. The government s Office Director of Public Procurement rates all agencies on performance. The LDF has consistently received the highest rating. The Bank carried out procurement reviews twice every year and found that the system was solid, justifying an increase in prior review thresholds. In , there were changes in procurement personnel in the TST and delays in recruitment that caused a temporary downgrading of procurement performance. At the district level, the challenge was that there were no official procurement officer positions. Therefore, training had to be conducted for district public works engineers. In addition to district-level training, capacity building was carried out directly by the Bank s procurement staff through quarterly supervision and monthly clinics with procurement officers from all Bank project offices. 40. Financial management. Financial management was also carried out at the national, district, and community levels. Statement of Expenditures (SOE) reviews were done consistently, including financial management field visits jointly with the Bank and TST staff. Financial supervision reports and audits were remitted in a timely fashion and found to be acceptable. There was significant difficulty in administering the system in May 2011 when the existing software system crashed and was considered inoperable. It took one year to switch to a more reliable system and reenter historical data. During this transition year, financial management performance was downgraded and the Bank decided there was insufficient technical backing to permit utilization of interim financial reports (IFRs) so the system was changed to SOEs. Utilization of the SOEs was challenging in a decentralized program due to delays in gathering all the necessary information to get reimbursements. 2.5 Post-completion Operation/Next Phase 41. Post-completion subproject operations and maintenance arrangements. Eligible PWSPs were designed to be labor-intensive and generate few recurrent costs. For example, over half of the PWSPs were in roads, focused on maintenance and repair of existing rural feeder roads rather than building new roads. This was an acknowledgement of the systemic issues with road maintenance that have plagued Malawi. The high labor intensity of the PWSPs meant that only basic maintenance could be applied rather than any more significant investments. For example, in bridges and culverts, basic maintenance could upgrade the status of the road and its durability. 10

26 42. There has been some progress in addressing systemic issues in road maintenance at the sectoral level, with the creation of the National Road Authority and the Road Fund Administration in 2006 to provide dedicated funding for road maintenance. However, overall funding remains inadequate. A tracking study found that local residents observed maintenance happening on only a quarter of the completed roads. MASAF III APL II has helped fill the funding gap, providing maintenance and repairs to a significant share of rural feeder roads in the country. 43. Community subprojects were focused on building teacher housing and limited classroom construction focused on repair and reconstruction of infrastructure affected by the earthquake. Operation and maintenance is the responsibility of the Ministry of Education, with support from local parent-teacher associations. The additional classrooms were replacements with teachers typically already assigned, with few recurrent costs generated. 44. The temporary income benefits from the PWSPs were designed to provide for immediate consumption needs of the vulnerable population. The innovative feature of the COMSIP sought to create a savings culture and mechanism for PWP beneficiaries to extend the economic impacts of the temporary income to interested households. The COMSIP contributed positively to financial saving behavior of group members with 84 percent of members reporting an increase in household savings. Moreover, savings is used to lend for income-generating activities. The COMSIP model requires a member to save 40 percent of their total money accumulated at the end of the year in the group, unlike other models that guarantees the saver an opportunity to withdraw all savings. The amount left in the group is security for the group to continue working and growing together. This promotes the longer-term sustainability of the mechanism. 45. Sustaining institutional capacity and follow-on operations. MASAF III APL II built institutional capacity at several levels. The most important of these during the implementation of MASAF III, given the significance of MASAF financing to local government administrative and capital budgets, was the decentralization agenda spear headed by the Ministry of Local Development and the Ministry of Finance to improve the procedures and process for the flow of financing from the Central Government to the Local Authorities. MASAF III supported the, Framework for Intergovernmental Fiscal Transfer System to support service delivery of local authority. In addition MASAF III financed critical staffing such as in financial management at the Local Authority District Offices to manage the finances. The LDF has been able to lobby with the Ministry of Local Government and Rural Development (MLGRD) to fill critical positions. For example, about 5 10 percent of all internal auditor positions which were filled in the LAs are now filled to 100 percent and procurement officers are now being deployed to all the LAs. The annual performance assessment which the MASAF III instituted has now become a standard procedure for the GoM to measure performance. 46. The COMSIP savings groups were created and supported under MASAF III and over the implementation period MASAF III supported COMSIP s institutional sustainability through the transition of the savings groups to clusters to formally constituted cooperatives. This cooperative structure invests ownership shares, pays dividend, and organizes continuing capacity building. At the national level, the COMSIP Cooperative Union Limited is a member-owned organization registered under the Cooperative Societies Act No. 36 of 1998 to provide institutional support to the COMSIP groups and cooperatives. 11

27 47. The project was part of the IDA financing strategy using a three-phased APL instrument ( ) to improve service delivery efforts by communities supported by local governments and sector ministries through a CDD approach. APL II was expected to satisfy the following three triggers to get into APL III: (a) LAs able to set objectives and achieve their annual targets; (b) annual PWP cycles completed according to plan; and (c) LAs with reporting mechanism for MDG indicator targets. These triggers were achieved, but it was determined that given that the follow-on operation would focus mainly on developing a national safety net system, it would be considered a new program rather than a continuation of the APL series. 48. A follow-on operation, Strengthening Safety Net Systems MASAF IV (US$32.8 million IDA credit), was approved on December 18, Building on the experience of MASAF III APL II, the new project is designed as a second generation safety net system that builds a safety net platform to harmonize, coordinate, and deliver safety nets based on the experience of the LDF mechanism as well as more recent approaches underway in the country on a unified registry system. It would continue to fund PWSPs as well as the COMSIP program and would add a social cash transfer program for labor-restrained households building on experiences of other donors in Malawi. MASAF IV would harmonize and coordinate the approaches, targeting, and systems of these programs and would continue to be implemented through the LDF TST. 3. ASSESSMENT OF OUTCOMES 3.1 Relevance of Objectives, Design and Implementation Rating: Satisfactory 49. Relevance of objectives. Project objectives were highly relevant to the country context at the time of appraisal and remained relevant throughout. The safety net focus addressed the needs of the chronically food-insecure populations and community-based funding of essential social infrastructure. In particular, teacher housing addressed a core sectoral issue that was impeding quality education in the rural areas. MASAF III APL II was aligned with the prevailing poverty reduction and economic growth strategy of the GoM and the Bank s CAS. 50. Relevance of design. Relevance of design was substantial. Project design was built on an experience of the MASAF as the main safety net and CDD organization in the country. The safety net design reflected global best practice for its attention to timing with agricultural cycles, high labor intensity, targeting approach using community-based selection, and below market wages to ensure the most vulnerable benefitted. Design was tailored to the specific opportunities and constraints of the decentralization process in Malawi. Successful innovations, like COMSIP, were scaled up. As the Malawi social assistance system developed, the design kept reflecting this evolution in the next stages of the project design. 51. Relevance of implementation. Implementation remained sensitive to the changing context in the country. Most importantly, two AFs were processed to address immediate social protection needs from economic shocks and natural disasters. While most of the ongoing Bank supervision (ISRs) evaluated implementation as Moderately Satisfactory, a review of the issues in the ISRs shows that the ratings resulted from different (not recurring) issues, that were addressed in the 12

28 course of implementation. Therefore, the overall relevance of the implementation is considered substantial. 3.2 Achievement of Project Development Objectives Rating: Satisfactory 52. The project aimed to reach the poor and vulnerable with temporary income, social infrastructure, and savings mobilization in the context of improved local governance and public sector management. The results chain posits that providing short-term temporary employment through public works can provide a social protection benefit to help vulnerable households cope with chronic poverty and transient shocks. Moreover, this short-term injection of income can create longer-term impacts on household economic well-being through savings mobilization and investment. These are particularly important given that it was a period of time when the country faced a number of shocks and the project played a role in helping communities cope with the shocks (see Section 1.2 above). At the institutional level, through a combination of learning by doing and capacity building, the LAs would be able to effectively take on greater development responsibilities leading to the government being more responsive to local communities over the longer-term. 53. According to the August 2006 ICR Guidelines (revised October 5, 2011), the project outcome should be assessed against both the original 6 and revised 7 PDOs. Assessing achievement of project development objectives prior to restructuring 54. Prior to the rewording of the PDO at AF 1, the project was on track to achieve its development objectives. The project had disbursed US$24.3 million, equivalent to 51 percent of the original credit or 21 percent of the total final project cost. The results framework included two PDO indicators. PDO Indicators Prior to Restructuring At least 120,000 households annually increase their average incomes (40% females). At least 70 percent of the LAs are able to set objectives and achieve at least 70 percent of their annual targets by MTR. Status as of Restructuring 2008/2009 and 2009/2010 PWP cycles reached 546,758 households from the two public works cycles, more than twice the targeted number of 240,000 households (47% females). 68 percent as measured by the first Local Authority Performance Assessment (LAPA) 6 Original PDO: To improve the livelihoods of poor households within the framework of improved local governance at community, local authority, and national levels. 7 Revised PDO: To improve the livelihoods of poor and vulnerable households and to strengthen the capacity of local authorities to manage local development. 13

29 55. The restructuring was not substantive in nature and project components remained unchanged. Since all indicators met their targets as of the restructuring date, this portion of the project is rated Satisfactory. The bulk of the analysis of achievement of the PDOs will focus on status and impacts as of the end of the project. Assessing achievement of project development objectives at project closing 56. A summary of outputs is available in annex 2. Project outcomes were focused on four areas: (a) provision of temporary employment and income through safety net cash-for-work; (b) longerterm income effects through savings mobilization, (c) improved access to basic services through community-driven basic infrastructure, and (d) enhanced LAs capacity to manage development. All PDO indicators were substantially achieved and most were significantly surpassed. Provision of temporary employment and income through safety net cash-for-work 57. MASAF III APL II generated 47 million person days over the life of the project. These safety net investments benefitted 2.2 million households cumulatively over 9 public works cycles. Each public works cycle reached an average of about 434,000 beneficiary households or about 15 percent of households nationwide, a relatively high coverage rate for safety net PWPs. These outputs had significant positive impacts on participating households. The BA found that 89 percent of the PWP respondents indicated that the PWP had contributed positively to household welfare. Impacts were noted in the following areas: Improved food security: The BA (2011) and the Second Tracking Study (2012) reported that 58 percent and 54 percent of beneficiaries, respectively, used public works cash earnings to meet immediate food needs. Over 740,000 public works beneficiaries were able to buy agricultural inputs following participation in the public works, surpassing the project target. Evaluations confirmed that the PWP households increased their spending on farm inputs, in particular fertilizer. The Second Tracking Study recorded improvements in maize production per hectare from 13 to 27 bags. Improved living conditions: Twenty-nine percent of beneficiaries moved from temporary houses (grass thatched, unburnt bricks; poles; and mud structures) to more permanent structures. Access to education services: About 10 percent of the beneficiaries used their earnings to pay for children s education and other education-related expenses. Access to health services: Cash payments enabled access to health care as beneficiaries reported to have paid for medical bills using the PWP earnings. Accumulation of assets: Tracing studies also found that 11 percent of public works beneficiary households spent cash on household assets. 58. There is some evidence of increased impact with the shift to greater number of days per year for households in AF 2. With participation in two cycles in the year, more beneficiaries were able to save some of their earnings. However, it was difficult for workers to commit to full participation in both cycles due to competing demands from their small-holder plots and small businesses. About 30 percent of the same workers participated in the multiple cycles with most choosing to rotate through the multiple cycles in the same year. 14

30 Longer-term income effects through savings mobilization 59. The COMSIP program offered services to savings and investments groups that were formed following participation in PWSPs. The theory was that these cash payments could prompt longerterm savings and investments by poor households if they were accompanied with training and social mobilization. At project closing, almost 100,000 households were members of the COMSIP saving groups. The project supported the formation of 4,465 groups (30 35 members), 1,106 clusters (3 5 groups), and 135 cooperatives comprising clusters that have saved sufficient amounts and wished to become a more formalized organization. A total of 3,685 groups received training grants for training in financial literacy, business management, and environmental and social safeguards. The aggregate volume of savings mobilized during the life of the project was estimated at MK 1.34 billion (US$2.7 million at current exchange rates). 60. The formation of these savings and investment groups had a positive impact on the wellbeing of the participating households. Based on a self-assessment of poverty, the percentage of members who reported to be better-off was higher with the years in existence of the savings group, from 67 percent for older groups to 48 percent for newer groups. Specific outcomes include the following: Increases in savings: Seventy-nine percent of members reported savings of at least 50 percent of their public works wages one year after participation, surpassing the project target. Increases in income: Per the COMSIP impact assessment, 39 percent of beneficiaries attested to income gains from the COMSIP activities. Impacts were higher in the groups that received more extensive COMSIP training. Improved food security and social protection of the household: These income gains translated into half of the beneficiaries stating that they now had more food/maize. Twentynine percent of the COMPSIP households found that they now had recourse to savings to help with family emergencies. All respondents reported that they had bought at least one household asset the past year, using the COMSIP savings and loans as well as business income. Access to credit: The COMSIP improved access to loans for group members. The percentage of members with access to loans increased over time. For example, from 57 percent for recently formed groups to 85 percent for groups with several years experience, with frequency of borrowing between two and three times. The average loan value was MK 11,864. Promotion of entrepreneurship culture: The COMSIP groups used loans mainly for starting a business or buying business stock/working capital and purchasing agricultural inputs. Thirty-eight percent of beneficiaries stated that they had expanded their business activities because of the COMSIP. Improved access to basic services through community-driven basic infrastructure 61. MASAF III APL II created or repaired a substantial number of public assets in poor communities through labor-intensive public works and community-driven social infrastructure subprojects. The project financed 39,706 PWSPs, over half of which were in roads (100,000 km), afforestation (6,500 ha), irrigation (5,500 ha), environmental sanitation (2,050 ha), land 15

31 conservation (2,100 ha), and fisheries (1,900). Community-based social infrastructure subprojects included 871 teachers houses, 200 classrooms, and 16 VIP latrines. 62. Impacts of these infrastructure investments include: Roads: The technical audit found improved access in rural communities which positively contributed to improvements in the rural economy. Teacher housing: The BA found that staff houses constructed under the project contributed to improvements in both teacher and student performance. Over half of the teachers moved from houses that leaked during the rains, affecting their preparedness. Two-thirds indicated that they had more time to prepare for lessons due to electricity availability. Ninety-nine percent of teachers occupying staff houses expressed satisfaction with the houses, which is a key motivating factor for improved performance. Classrooms: The technical audit study indicated that the project has contributed to better quality of education as enrollment of pupils in schools improved due to increased and improved infrastructure. Strengthened local authorities capacity to manage development 63. MASAF III APL II had a broader institutional objective of supporting capacity development at the local level. With over a decade of experience in directly funding the communities, the project represented a gradual evolution toward greater support of and responsibilities devolved to the LAs in part as a way to support the country s decentralization trajectory.. The project supported the LAs and communities through learning-by-doing that is, transferring direct subproject management responsibilities to them as well as through training and capacity building. Training was provided to over 40,000 local personnel in various core functional topics. For example, MASAF previously financed an accountant position in each LA to support sub-project fiduciary activities. With this project, district administrative staff took over disbursement and reporting responsibilities. The Project also sought to support district level participatory planning efforts, including the creation of district development plans. The PDO indicator in this area, that least 70 percent of local authorities were able to set objectives and achieve at least 70 percent of their annual targets by midterm review, was fully achieved. 64. MASAF III APL II was instrumental in developing and testing Malawi s fiscal transfer system, including supporting the technical work in developing the design of a fiscal transfer and capital grants system. Financial and project management computerized systems were developed for districts as a whole and not just for the MASAF project. And more importantly, the LDF served as the initial test for actually transferring capital funding to the district level.. The LDF succeeded in becoming a recognized conduit for other donor financing, amounting to US$245 million committed toward various activities in the four windows of the LDF. 8 Based on the success of communitymanaged school infrastructure subprojects, the Ministry of Education has taken the policy decision that all primary school construction, whether financed externally or through the national budget, 8 The commitments include Malawi government MK 5.4 billion (US$38.7 million); Education Sector Wide Approach (ESWAP) MK 16 billion (US$50 million); World Bank US$114 million; African Development Bank US$27.7 million; and KfW 11 million (US$15 million). 16

32 should adopt the MASAF community-driven approach and be channeled through the LDF. To date, the LDF community window manages sector-wide primary schools investment activities, with the total funding between 2012 and 2013 equivalent to US$36.9 million. 65. There has been some noted improvement in the LA performance. The PAM was used annually to measure capacity in several key areas (participatory planning and budgeting, financial management, procurement, service delivery, supervision, and M&E). The composite performance of councils improved from 37 points in 2010 to 46 points in In 2013, the highest scores were in procurement and M&E, with the lowest scores in financial management. Despite this progress, the assessment pointed to weaknesses in vacant positions of key personnel as well as the slow pace of the human resource devolution, continued high level of reassignments, and lack of equipment for data collection. These issues need to be addressed from a decentralization perspective and extend beyond the ability of a single project like the MASAF to correct. The Counterfactual 66. While it is not easy to construct a quantitative counterfactual, it can be concluded with a high level of certainty that without the project a huge number of Malawi s poor would have been worse off, would have received no poverty-targeted direct cash and food support, fewer opportunities to improve the assets of their communities, fewer opportunities to get on a graduation path out of poverty, and weaker local government mechanisms for outreach. The safety net programs that functioned in Malawi prior to MASAF would not have been Government led, and would have remained as scattered and varied agendas and definitions of development goals. Without MASAF, the ravages of HIV/AIDS in that time frame, also causing a decimated national capacity to deliver services, would have been even more catastrophic. In the face of the negative effects of HIV/AIDS MASAF provided a secure and stable highly professional and consistent delivery mechanism to reach each and every district and its most vulnerable populations with income and food security and a sense of hope. Without the project, more than 2 million poor households may not have increased their average annual incomes. More than 78,000 poor would not have been able to save, and at least 300,000 to 400,000 poor would not have been able to provide for their needed agricultural inputs. Looking at the indicators for the community investments and savings group, it can also be concluded that close to 100,000 people (among which close to 70% women) would not have been able to get on a sustainable path out of poverty if it weren t for the project. The project also improved the capacity of local authorities and without it 30% of the local authorities would not have been able to set objectives and achieve their targets. 3.3 Efficiency 67. As presented in annex 4, the project was efficient. Public works safety net program parameters were consistent with best practice for efficiency with high labor intensity, low administrative costs, below-market wages to promote effective targeting, and broad coverage. Unit costs of the constructed social infrastructure was less than comparator projects. The COMSIP program reports continued increases in savings mobilization and generally high repayment rates from the on-lending to group members. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 17

33 68. The project was relevant to the country context and in line with the government s economic growth and poverty reduction strategy and the CAS of the Bank. It was agile in responding to economic shocks and natural disasters as they arose. There is ample evidence to confirm achievement of the PDO to improve the livelihoods of poor and vulnerable households and to strengthen the capacity of the LAs to manage local development. 69. Applying the split evaluation rating, efficacy in meeting development objectives was substantial both prior to and following restructuring. Relevance and economic efficiency were substantial and consistent throughout the full implementation period. Therefore, the overall outcome rating for both the pre- and post-restructuring portions of MASAF III APL II is rated as Satisfactory, per Table 1. Table 1: Weighting of Outcome Ratings against Original and Revised Project Objectives Against Original PDOs Against Revised PDOs Overall Rating Satisfactory Satisfactory - Rating value Weight (% disbursed before/after PDO change) Weighted value (2 x 3) Final rating (rounded) - - Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 70. Poverty impacts. Public works were targeted at the poor. For example, 87 percent of the households used to run out of food before the next harvest when they first entered the program, only 15 percent had gone beyond primary school, and 8 percent relied on skilled employment and fishing meaning that as many as 92 percent relied on piece works, small-scale businesses, and farming as small holder farmers. 71. Income transfers were not designed to be sufficient in and of themselves to lift households out of poverty, but provided an important protective element to prevent destitution, support critical expenditures (for example on health and education), and accumulate basic assets. These actions over the longer term contribute to poverty reduction. Through investments in small businesses and purchase of productive assets, whether group or individual based, the COMSIP program has provided an additional path toward poverty reduction for the poor. 72. Gender. Women were prime beneficiaries of MASAF III APL II. The selection of public works participants was expected to be gender sensitive as specified in the 2010 project implementation manual. Program eligibility required that at least 40 percent of public works participants be women. Actual performance surpassed this, with women representing 51 percent of public works beneficiaries. In addition, 61 percent of the COMSIP members were women. The COMSIP program also included a focus on empowerment and leadership, including orientation on 18

34 gender-based violence issues. The COMSIP impact assessment found that participation in the COMSIP contributed to a decline in physical violence against women in their families. 73. Social development. The project used participatory techniques and promoted social inclusion. Transparency at the local level was furthered through the use of social audits, public posting of approved projects and other information, and a hotline for any complaints. Within the COMSIP program, members committed to eight responsibilities, one of which was the elimination of social injustice. The project also sought to mainstream participatory aspects of governance through its technical support to the LAs. Training in participatory planning and budgeting and the application of social safeguards was provided to the LAs. The PAM system revealed steady progress in the application of participatory planning and budgeting throughout the lifetime of the project, with national composite scores rising from 45 to 61 points in this functional area. (b) Institutional Change/Strengthening 74. At the national level, the MASAF/LDF continues to be one of the more efficient and effective government institutions in Malawi. The project supported institutional advancements in two areas principally. First, the project consolidated the MASAF/LDF s social protection role as a crisis scale-up vehicle. And second, MASAF III APL II was the key vehicle to operationalize the LDF mechanism, the core fiscal transfer mechanism to support eventual decentralization of capital budgets. Apart from the MASAF/LDF itself, there were institutional impacts on other nationallevel institutions. The project developed the PAM tool that is now used by the MLGRD as a standard to evaluate local government functional capacity 75. MASAF III APL II had a significant institutional impact at the local level. For the first time in Malawi, the LAs were given an indicative planning figure and transparent allocations that they could link to their local development plans. MASAF III APL II remains the most significant development budget that the LAs directly manage. In addition, the project was responsible for most of the training of the LA technical staff carried out in the country, including M&E officers, accounting officers, public works staff, and so on. The project encouraged team work and collaboration by establishing multisectoral district teams, whereas earlier sectoral reporting was directly to the central level with little collaboration across sectors. Moreover, the LDF has become a strong lobbying voice for the central government to increase staffing allocation to the LAs, with some effectiveness. 76. On the private side, the COMSIP has gone beyond the basic formation of savings groups to creating clusters and ultimately cooperatives. These cooperatives are part of a cooperative union that has created a stronger and more formal institutional structure, including ownership of shares by members. These cooperatives have also begun to access commercial credit, for example, from the Malawi Development Fund. 77. At the community level, MASAF III APL II continued the tradition of training and empowering community groups in program management, project formulation, implementation, M&E, financial management, and procurement. The BA reported that the skills and experiences gained by the PMCs have given them confidence to attract and manage other development projects. Other programs have used the same committees (United Nations Children s Fund [UNICEF], nongovernmental organizations). 19

35 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME Rating: Moderate 78. The government remains firmly committed to continuing and scaling up the achievements of MASAF III APL II under the new operation. Institutional capacity developed under the project will help support sustainability of the impacts. 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 79. The Bank team worked closely with the government to prepare the project in a timely fashion. The Bank preparation team had a good skill mix, adequately supporting the technical needs of the project. Please note that the Bank helped develop the social protection lens, bringing in international and regional experience in safety nets to inform design. Fiduciary readiness was confirmed during preparation, building on the MASAF s experience in managing Bank funding. Risks were adequately identified and mitigated. For example, the overall lack of funding for recurrent costs had hampered effectiveness of previously constructed schools and health centers. The eligible menu of community subprojects was limited to address this issue. 80. There were some minor shortcomings in ensuring quality at entry. The PDO could have been more clearly worded, prompting a rewording at AF 1. Nonetheless, the overall results readiness of the M&E framework was rated above the average in a results readiness portfolio review of all Bank social protection operations in FY Also, MASAF III APL II project appraisal document was selected among the cohort of projects reviewed to provide good practice examples of results frameworks, results monitoring tables, and institutional M&E arrangements. (b) Quality of Supervision Rating: Satisfactory 81. During the ICR mission, the government commented universally that the Bank team was very responsive during implementation. Bank supervision was effective in focusing on continual capacity development, for example, through regional study tours and incorporation into the Bank s topical courses. Fiduciary supervision went beyond ensuring compliance to seeking to build government skills in this area. For example, the Bank s procurement specialist in the country office held quarterly procurement clinics inviting all procurement officers, which created a network of government procurement specialists who could support each other. In terms of safeguards, every mission brought a safeguards specialist. The Implementation Status Reports (ISRs) were used proactively to highlight key issues and downgrade performance where necessary. 20

36 82. There were some minor shortcomings in supervision. The Bank has been slow to reinstate using IFRs despite the installation and operations of a new, more robust accounting system. This may be overcautious given the added administrative burden of processing the SOEs in such a decentralized program. Also, although a safeguard specialist was included in the team, the Bank could have paid closer attention to implementation of a simplified environmental checklist for PWSPs even though these were mostly repair and maintenance programs. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 83. The Bank task team ensured quality at entry and provided effective supervision throughout implementation. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 84. The government has been highly supportive of the project and considers the MASAF as a flagship program of poverty reduction in the country. The government prioritized the extension of safety nets to its vulnerable populations, including two AF operations to increase funding in response to economic shocks and natural disasters. The Ministry of Finance was able to mobilize significant additional donor funding to be channeled through the nascent LDF mechanism supported by the project, increasing its legitimacy and profile as the main decentralization mechanism. 85. There were minor shortcomings in government performance. These were mainly external factors not specifically directed to the project but that nonetheless had an impact on implementation. For example, the lack of elected local leaders between 2003 and 2013 weakened the legitimacy of the LAs as expressing the will and preferences of their citizens. Delays in staffing and disruptive transferring of staff continued to adversely affect local government capacity. The Local Authority Service Commission is now in place and has started to recruit staff and implement some needed human resource reforms. Economic liberation measures included the devaluation of the Malawi kwacha by about 50 percent which inflated the unit costs of subprojects. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 86. The TST of the LDF was the main implementing agency. The TST was highly committed and retained sufficient technical staff to effectively oversee project implementation. In particular, the team was able to manage a complex operation working through a decentralized structure with many implementing partners of varying capabilities. The TST provides technical support to the LAs to improve their performance. The TST responded quickly to the demands of preparing two AF operations within crisis contexts. Procurement, financial management, and auditing functions were carried out in compliance with project rules and operating procedures. Also, the M&E framework 21

37 was effectively followed, including reporting on key indicators and undertaking a significant number of studies and evaluations that created a sufficient basis for proving impact and improving operations. 87. There were minor shortcomings in performance. Financial management suffered from the computer crash of the accounting program resulting in the shift to using the SOEs which encumbered the disbursement process. Execution of public works cycles were often delayed by a month or two versus the planned calendar. There were delays in appointment of key personnel. Planning of expenditures in the final year could not be fully achieved, leading to cancellation of remaining balances. Also, despite significant attention on mainstreaming environmental screening by the LAs and by community groups implementing the COMSIP investments, as well as the additional mitigation funding to subproject costs, the PWSPs could have benefitted from more systematic review. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 88. Both government and implementing agency performance is rated as Satisfactory. Any shortcomings did not affect the overall achievement of the PDOs. 6. LESSONS LEARNED Lessons learned in public works safety net interventions 89. MASAF III APL II confirms the feasibility and impact of a large-scale safety net in Malawi. This adds to the body of evidence on the potential for safety nets in low income countries and in the Africa Region in particular as an essential component to stabilize household incomes in the face of shocks and chronic poverty. 90. Safety net program parameters face trade-offs between cost, coverage, generosity, and durability of the infrastructure created. There is no right formula for this, but rather it depends on the country s circumstance and priorities. The project began with 12 days of public works employment per participant, lower wage rates, and high labor intensity to distribute the income benefits as widely as resources would allow. As shocks called for both greater need and greater resources, coverage was scaled up, wage rates were increased, and the number of eligible days went from 12 to 48 per year. High labor intensity increased the share of costs going to household transfers, but in the case of rural roads it limits interventions to maintenance rather than any significant upgrading in the quality of the road itself. In general, safety net programs should experiment with these parameters to find the correct mix as they face changing circumstances. 91. Malawi s public works safety net achieved significant participation from women. The focus on unskilled labor, largely road maintenance and afforestation, at less than market wages tended to increase women s participation, as well as clear program guidelines and outreach to promote women s access. 22

38 92. A recent yet incomplete research 9 on the outcomes of the PWP under MASAF III brought up some controversial aspects and was reviewed by the ICR team. As noted by the research team, the assessment has some data and methodological issues that resulted in inconclusive findings. Nonetheless, the preliminary analysis of MASAF III indicated that some of the effects may pose questions as to how the program influenced the food security and the use of agricultural inputs by the participating and non-participating communities. The studies also indicate that the number of days of work and the amounts of cash transferred may have been too small to have an influence. While inconclusive, this evidence was already used as a lesson in the design of MASAF IV, which increases the focus on an integrated Social Protection System, increases the number of days worked, adds in social cash transfers to targeted poor, finances community support and investment groups and modifies the PWP to make it more focused and targeted for better effects on the most vulnerable to enable them to graduate out of poverty. 93. Finally, public works will definitely remain an important part of the social assistance tool kit in the country. The PWP is scalable in times of crisis and can be directed to the right geographic locations and target groups. However, more local level capacity building will be needed to ensure predictability in terms of timing of payments as well as identifying the chronic poor for longer-term inclusion within the annual PW cycles. Lessons learned in mobilizing savings 94. Voluntary savings by individuals and households should be encouraged as an asset building and risk diversification, risk insurance and investment promotion opportunity. Savings groups should be demand driven. The level of household savings can provide a tangible indicator of graduation capability of a household. Malawi s COMSIP program is an innovative example of extending safety net benefits into activities to further promote graduation. It has a unique focus on voluntary mobilization of savings first. Savings provide an important component of self-insurance against future shocks as well as an asset base to grow investments. Rules concerning buying shares of the group and requiring minimum savings levels ensures that this is not seen as a handout. Credit comes only after there is buy-in. It is clear that not all safety net participants are in the same position or have the same level of interest in participating in savings groups. However, the COMSIP has continued to expand and has deepened the impact of the safety net program. 95. Training has been shown to be an important aspect of the COMSIP success. The impact assessment found that performance is stronger in most aspects for the 41 percent of groups that have been supported by the COMSIP training programs. Demand for the more intensive training modules surpassed available funding. Given the high return on these training activities, future funding for training in the COMSIP should be a priority. Lessons learned in community-driven social infrastructure 96. The project continued to affirm over a decade of experience with community-based management of small-scale infrastructure. This phase limited the eligible menu of subprojects in part to mitigate the risk of lack of recurrent cost funding from the central government, and in part to respond to a gap of financing and a request from the Government. While this limited community 9 Draft report by the Development Economics Research Group under a working title The design of Public Works and the Competing Goals of Investment and Food Security 23

39 choice, it resulted in needed investments, like expansion of rural teachers housing and replacement of classrooms destroyed by the earthquake, without overburdening the national budget. Once the need for these specific types of investments was gone, however, the project (e.g. the next phase MASAF IV) returned to a wider array of community choices. Lessons learned in capacity development 97. The capacity development under MASAF III was mainly targeted at the local levels. The issue of the pace of decentralization was important to a project like MASAF III APL II but was outside of its direct control. Decentralization in Malawi has been more aspirational that operational. MASAF III APL II has been key in helping the government work out operational systems and build confidence in the district level. For example, at the outset, there was little confidence in the central government about the viability of transferring funds for direct management by district offices. Now, the LDF mechanism has channeled multiple donor and government programs to the LAs. 98. The MASAF built capacity of districts to respond to demands of communities with or without decentralization. Even with limited decentralization, the local levels can constructively engage and manage investments. The project built local capacity in two ways, both of which are important. Learning-by-doing through direct management of project planning, implementation by local governments and communities, and training programs reinforced this capacity development. 99. A couple of more general lessons can also be derived from the project s implementation. Firstly, it should be mentioned that a type of massive intervention as MASAF, which basically serves as a main social protection program of the country, should be accompanied by or integrated with a broader option for policy dialogue. By virtue of sheer size and importance, MASAF III was influenced by policy decisions. Projects of that type should not only be recipients but also shaping agents of policy, which could be done through including a subcomponent for policy dialogue, or linking the project to a separate source of funding for policy analysis and dialogue. Secondly, even though Malawi will continue to require a good social protection program it is important to look at ensuring financial sustainability through increased domestic allocations to the program. As part of implementing the successor project, the Bank should engage with the authorities about ensuring financial sustainability. 24

40 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$, millions equivalent) Components Appraisal Estimate (US$, millions) Actual/Latest Estimate (US$, millions) Percentage of Appraisal Community Livelihoods Support Fund Local Authority Capacity Enhancement Fund National Institutional Strengthening Fund Original MASAF III APL II Additional Financing 1 Additional Financing 2 Total Contingencies Total Project Costs (b) Financing Source of Funds Appraisal Estimate (US$, millions) Actual/Latest Estimate (US$, millions) Percentage of Appraisal Original MASAF III APL II Additional Financing 1 Additional Financing 2 Total Borrower Local Communities IDA Total

41 ANNEX 2. OUTPUTS BY COMPONENT Component 1 Table 2.1: Public Works Participants by District and Public Works Cycle Name of Local Authority 2008/ / / 11 Public Works Program Year 2011/ / 13 Phase / 13 Phase / 13 Phase / 13 Phase / 14 Chitipa 2,790 1,843 3,830 3,315 7,547 8,559 8,759 8,377 8,619 27,884 Karonga 7,428 3,876 6,620 5,712 14,082 15,053 15,640 15,761 16,575 52,771 Rumphi 3,781 2,373 3,770 3,050 7,576 7,576 7,328 7,611 9,553 28,126 Nkhata Bay 4,330 2,712 4,184 3,708 9,527 9,575 9,575 9,575 8,506 34,036 Likoma ,754 1,766 1,791 1,791 1,761 5,090 M'mbelwa 15,063 12,197 9,160 9,407 26,736 26,736 25,585 26,195 26,573 99,299 Mzuzu City 2,597 1,632 2,611 2,707 6,489 6,591 6,590 6,603 7,134 22,627 Kasungu 12,779 6,988 8,386 7,242 20,078 20,171 20,501 20,063 19,660 75,644 Kasungu Munic , ,155 2,155 2,155 2,155 1,605 7,431 Nkhotakota 7,021 4,026 5,385 4,954 13,240 13,291 13,291 13,291 14,285 47,917 Ntchisi 6,485 3,564 3,658 3,252 8,214 8,270 7,789 8,214 9,603 33,443 Dowa 11,273 4,647 7,873 6,907 19,970 14,303 13,994 14,308 18,676 65,581 Salima 7,525 4,313 6,910 5,851 16,187 16,187 16,194 16,187 18,562 56,973 Lilongwe Rural 19,370 12,347 15,670 13,765 37,317 37,713 38,369 38,369 35, ,182 Lilongwe City 0 8,626 9,381 8,214 21,434 23,035 23,014 23,014 21,528 70,690 Mchinji 7,491 6,040 6,002 6,407 16,798 16,954 17,251 16,954 18,083 59,692 Dedza 16, ,610 8,413 23,727 28,069 21,233 21,233 22,967 96,626 Ntcheu 12,180 7,368 9,432 7,244 20,088 20,142 20,207 20,207 17,169 76,454 Mangochi 22,832 13,711 14,391 12,404 31,886 33,851 33,722 32,990 29, ,075 Mangochi Town ,696 1,696 1,696 2,637 1,696 Machinga 13,869 6,570 11,651 10,172 28,716 28,886 28,873 29,314 11,784 99,864 Balaka 9,055 11,819 9,054 7,970 19,461 19,562 19,562 20,440 19,769 76,921 Zomba Rural 19,334 12,697 11,778 10,292 26,792 29,214 29,214 19,516 25, ,107 Zomba City 2, ,250 2,031 4,572 4,572 4,572 4,522 5,369 16,530 Chiradzulu 8,615 9,355 9,520 6,876 19,005 19,025 19,025 18,845 15,985 72,396 Blantyre Rural 12,144 6,730 8,216 7,204 19,516 19,783 20,046 20,004 14,844 73,593 Blantyre City 7,200 6,810 11,050 8,923 21,988 25,158 25,141 22,000 23,386 81,129 Neno 4,597 2,955 4,496 3,979 10,308 10,308 10,308 10,343 14,244 36,643 Mwanza 4,684 5,887 2,681 1,759 3,742 3,823 3,869 3,547 4,546 22,576 Thyolo 16,328 7,643 10,683 11,616 25,203 31,957 32,005 22,005 24, ,430 Luchenza Town ,120 1,051 1,189 1,989 1,929 1,989 2,810 5,605 Mulanje 18,060 8,114 11,000 9,665 27,087 27,087 27,727 25,525 23, ,013 Phalombe 17,418 8,548 7,736 6,875 18,196 20,460 14,003 19,021 17,891 79,233 Chikwawa 10,641 13,071 10,282 9,620 26,903 27,103 25,962 27,209 20,479 97,620 Nsanje 10,316 11,148 9,475 8,329 24,088 23,059 23,280 22,465 23,539 86,415 Total 315, , , , , , , , ,812 2,163,944 Total number of beneficiaries reached under the normal PWP 1,004,453 Total number of beneficiaries reached under the Rapid Response PWP 603,679 Total number of beneficiaries reached under the additional cycle of the PWP in 2013/14 555,812 Total number of beneficiaries reached under MASAF III APL II project ,163,944 Total 26

42 Table 2.2: Public Works Subprojects: Number of Projects and Outputs by Year and by Type Program Year Total # of Projects # Roads km # Forestation ha # Irrigation ha # Environment/ Sanitation # Land Conservation # Fisheries 2008/09 2,185 1,317 7, , /10 2,082 6,675 2,216 1, /11 2,387 8,933 3, /12 1, /13 ERP Phase /13 ERP Phase /13 ERP Phase /13 ERP Phase /14 ERP Additional Cycle 6,182 3,581 14,642 1,128 1, ,292 3,739 9,113 1,122 1, , , , ,289 3,808 13,103 1,092 3, , , TOTAL 39,706 20, ,147 6,519 18,469 1,676 5,489 2,052 2,107 1,930 Table 2.3: MASAF III APL II Project Direct Beneficiaries Category of Beneficiaries Achieved Norm/ Standard Total Beneficiaries Remarks Primary School Staff Housing Project people/ house 4,040 Based on assumption that 5 people per household would benefit directly from the staff house = 808*5 PWP (Normal) 1,004,453 5 people/ household 5,022,265 It is assumed that on average 5 people would benefit directly per household from the income = 1004,453*5 PWP (Rapid Response) 603,679 5 people/ household 3,018,395 It is assumed that on average 5 people would benefit directly per household from the income = 603,679*5 27

43 Category of Beneficiaries Achieved Norm/ Standard Total Beneficiaries Remarks Public Works Additional Cycle 2013/14 555,812 5 people/ household 2,779,060 It is assumed that on average 5 people would benefit directly per household from the income = 603,679*5 Based on assumption that 5 people per household would benefit directly from the staff house = 63*5 Staff houses under crisis Response 63 5 people/hou se 315 Classroom blocks under crisis response -100 blocks (200 classrooms) pupils/ classroom 12, classroom blocks (200 classrooms), with 60 pupils per class = 200*60 Project training activities Various Various 39,469 Trainings for PMCs; LA staff; and nationallevel stakeholders Community savings and investment 99,964 n.a. COMSIP group members originated from the PWP and as such these are counted under the PWP Total beneficiaries 10,875,544 Component 2 Trainings and Workshops Conducted 1,956 SMC members trained on simple principles of project management, including procurement and environmental and social safeguards 138 DESC members trained on environmental and social safeguards 3,685 COMSIP groups and 70,892 individuals trained on group dynamics, financial literacy, and business management 27 EDOs trained on environmental and social management 101 accounts assistants trained on financial management 165 IPC members trained on public procurement procedures and processes 120 artisans trained on modern construction techniques 28 directors of public works from councils trained on the solar photovoltaic system 240 LA staff trained on how to prepare Strategic and Implementation Plans 60 LA staff trained on project proposal writing skills 128 LA staff trained on the Public Service Performance Management System 33 District commissioners and chief executive officers trained on effective leadership and management 35 district monitoring and evaluation officers trained on project monitoring arrangements and PWP surveys and data analysis 590 DEC members oriented on the local government system and decentralization 1,204 district council members trained on the community project management process 18,148 VDC members trained on their roles and the village action planning process 28

44 11 LDF-TST staff trained on strategic supervisory and leadership skills 64 MLGRD staff oriented on the civil service performance management system 19 LDF-TST staff trained on environmental and social safeguards 33 LAs oriented on the operation of the Deloitte-managed Tip-Offs Anonymous facility 140 LA staff trained on the project management module of the Integrated Financial Management Information System (IFMIS) 35 directors of planning and development oriented on the revised District Development Planning System handbook Table 2.4: Training Interventions Details Name of Training Target group Conducted By No. Trained Dates Procurement training Key district staff and IPC members All LAs 165 August 2010 Training on environmental safeguards EDOs All LAs 168 March 2011 Training of M&E officers M&E officers All LAs 24 Orientation on the Public Service Performance Management System Training of TST staff in environment and social safeguards District commissioners Directors of Administration Human resource officers Representative from devolving sectors Directors Specialists Officers Department of Public Service Management and the MLGRD Ministry of Environmental Affairs and Climate Change November 2011 June 2012 April 11 and 12, 2012 Sensitization of TST staff on the Deloitte-managed Tip- Offs Anonymous facility All TST staff Anticorruption Bureau 33 April 13 and 14, 2012 Trainers workshop on improved construction techniques Directors of public works Building supervisors Instructors from technical colleges National Construction Industry and Buildings department 67 May 21 to 25, 2012 Training of local contractors (artisans) Artisans Technical colleges supported by district councils Training of PMCs SMCs/PMCs in 30 education districts Extension workers and LA staff 33,240 Orient district commissioners and chief executive officers on the newly developed Performance Based Grant System District commissioners and chief executive officers Malawi Institute of Management (MIM) 1, By June 2013 By June 2013 February

45 Name of Training Target group Conducted By No. Trained Dates Orient directors of planning and development on the revised District Development Planning System hand book Directors of planning and development MLGRD 35 April 2013 Conduct refresher training workshops for LA IPCs Members of IPC Office of Director of Public Procurement 175 May 2013 Train accounting personnel in financial management Accounts assistants from LAs Staff Development Institute 70 June 2013 Total 39, Component 3 Table 2.5: Studies and Evaluations Carried out under the Project Title Firm/Consultant Date Public Works Program Baseline Survey Report LDF TST May 2010 LAPA Baseline Data Final Report DR Consulting Services July 2010 Comprehensive Report of the CSC Process LDF TST October 2010 Information System Strategy Plan Study Bumas International June 2011 PAM for Local Governments in Malawi Annual 2011 Assessment Tracking Study on Public Works Cash Transfer in Malawi Inspired Thinking Inc. November 2011 Wadonda Consult January 2012 Comprehensive Communication Strategy for the LDF LDF TST January 2012 Beneficiary Assessment Study of the MASAF III APL II - LDF Mechanism Project Centre for Development Management January 2012 Technical Audit of Community & Local Assembly Window Subprojects Implemented from 2009 to December 2011 Romana CNM-YBJ (JV) Consulting Engineers February 2012 Development of a Performance Grant System Malawi Institute of Management September

46 Sector Budget Review Study Malawi Institute of Management September 2012 LAPA for Local Governments in Malawi 2012 Annual Assessment Inspired Thinking Inc. December 2012 Tracking Study on the Public Works Cash Transfer Program DR Consulting March 2013 Annual Report on the LAPA for 2012/2013 MLGRD November 2013 Meta Analysis of MASAF III APL II Project Studies LDF TST February 2014 Savings and Investment Promotion Intervention Impact Assessment Study Centre for Development Management March 2014 Trend Analysis of Key PWP Livelihood Indicators LDF TST May 2014 Beneficiary Assessment and Technical Audit for C risis Response Program for LDF RUO Consultants June 2014 Construction of School Blocks and Teachers Houses under the ESWAP through the LDF LDF TST September

47 Cost effectiveness of safety net transfers ANNEX 3. ECONOMIC AND FINANCIAL ANALYSIS 1. The cost effectiveness of the safety net transfers are a function of the accuracy of targeting, the labor intensity of the public works, wage rate, and the general overhead costs of the program. Together these determine the overall share of program resources that actually reach the poor through transfers. 2. MASAF III APL II cost effectiveness of transfers is high. a) Accurate targeting of beneficiaries, with over 80 percent of beneficiaries having characteristics consistent with the ultra-poor, meant there was very little leakage to the nonpoor. b) The high labor intensity, on average 80 percent, was based on a restrained list of eligible subprojects that ensured that only high labor intensive investments were made and overhead costs were relatively low (14 percent over APL II including a valuation for all government staff working on the program as well as all institutional support expenditures). This compares favorably with international experience in cost-effective PWPs. c) Wage rates were set below market rates, rising from MK 200 to MK 300 over the course of implementation compared to the current government minimum wage of MK 550. d) By shifting to multiple cycles per year, the average annual transfer to a beneficiary increased from MK 2,400 (12*MK 200) to MK 14,200 (48*MK 300) for workers who participated fully throughout the year. e) Administrative overheads were very low by international standards. The National Institutional Strengthening Fund was 10 percent of total project costs, and included both administrative costs as well as institutional strengthening to central level agencies and the project s M&E activities. The LDF retained 5 percent for those funding the sector-wide classroom construction program. Cost effectiveness of the infrastructure investments 3. According to the technical audit, a comparison between the MASAF/LDF approach and other construction models shows that the MASAF approach is more cost effective for constructing similar projects as compared with other partners. For example, the cost of a block with two classrooms which was fitted with 60 desks was about US$22,000 under the MASAF/LDF mechanism. A similar classroom block constructed under the Education Infrastructure Management Unit (EIMU) of the Ministry of Education, Science, and Technology costs about US$27,095, excluding 35 percent of external works and consultancy fees (US$9,483). This implies that the EIMU approach is 66 percent higher than the MASAF/LDF approach, yet it does not include desks as is currently the case with the MASAF/LDF approach. Likewise, for a teacher s house, the unit cost under the MASAF/LDF approach is US$25,000 while under the EIMU it is about US$30,240 plus 35 percent for external works and consultant s fees. Even with these cost savings, the quality of community subprojects like schools and teacher housing was found to be good on average in the technical audit. 32

48 Economic analysis of COMSIP activities 4. The COMSIP program does not provide funding directly for credit and business investment activities, rather this comes from members savings. The COMSIP only funds training and social mobilization. The COMSIP groups charge an interest of between 10 and 30 percent on individual member loans. This is lower cost capital than either micro-finance institutions or private money lenders. 5. There is no aggregate data collected on repayment rates for the COMSIP internal lending to members, though anecdotal reports suggest high repayment rates supported by the tight bonds and social control as well as the savings requirement for all members. Several indicators point to the viability of the businesses created and the solvency of most COMSIP groups. The COMSIP co-operatives have been experiencing a steady growth in recent years, both in terms of membership and savings. The impact assessment found that the COMSIP has a positive impact on the level of savings by individual members. The study found that on average members from old supported groups have four times the amount of savings than new members. About 70 percent of the COMSIP members access loans through the group, and on average have repaid and taken out credit a second time. The majority of business investments by the COMSIP savers has been in the areas of selling of food crops, cash crops, livestock, milk, and fish. There is no data on the number of potential savings groups that have dissolved. Due to the rules and structure of the program, groups have generally been sustainable. Performance is stronger for the 41 percent of groups that have been supported by the COMSIP training programs. 33

49 ANNEX 4. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES (a) Task Team Members Names Title Unit Lending Janet Chido Bvumbe Program Assistant AFMZW Simon B. Chenjerani Chirwa Senior Procurement Specialist GGODR Ben Chirwa AFTH1 - HIS Alfred Sambirani Chirwa Population & Health Spec. AFTH1 - HIS Fenwick M. Chitalu Financial Management Specialist AFTME - HIS Lori A. Geurts Operations Analyst GHNDR Chrissie Kamwendo Senior Operations Officer AFMZW Nginya Mungai Lenneiye Country Manager AFMZW Michael N. Mambo Senior Education Specialist AFTEE - HIS Masud Mozammel Senior Communications Officer ECROC Donald Herrings Mphande Lead Financial Management Spec GCFDR Suleiman Namara Senior Social Protection Economist GSPDR Eva K. Ngegba Program Assistant GHNDR Naa Dei Nikoi Operations Adviser LCRDE Krishna Pidatala Senior Operations Officer GTIDR Vivek Srivastava Lead Public Sector Development GGODR Hardwick Tchale Senior Agriculture Economist GAGDR Supervision/ICR Zeria Ntambuzeni Banda Communications Officer AFREC Anush Bezhanyan Practice Manager GSPDR Janet Chido Bvumbe Program Assistant AFMZW Simon B. Chenjerani Chirwa Senior Procurement Specialist GGODR Alfred Sambirani Chirwa Population & Health Spec. AFTH1 - HIS Fenwick M. Chitalu Financial Management Specialist AFTME - HIS Lori A. Geurts Operations Analyst GHNDR Kjetil Hansen Senior Public Sector Specialist GGODR Chrissie Kamwendo Senior Operations Officer AFMZW Nginya Mungai Lenneiye Country Manager AFMZW Muna Salih Meky Senior Education Specialist GEDDR Francis Kanyerere Mkandawire Financial Management Specialist AFTME - HIS Masud Mozammel Senior Communications Officer ECROC Donald Herrings Mphande Lead Financial Management Spec GCFDR Maggie Mwaisufanana Mshanga Receptionist AFMMW Suleiman Namara Senior Social Protection Economist GSPDR Harriet Nattabi Water & Sanitation Specialist TWIAF Eva K. Ngegba Program Assistant GHNDR Nadege K. Nouviale Program Assistant GSPDR Berk Ozler Senior Economist DECPI Krishna Pidatala Senior Operations Officer GTIDR Vivek Srivastava Lead Public Sector Development GGODR Hardwick Tchale Senior Agriculture Economist GAGDR Nko Etesin Umoren Resource Management Officer BPSGR 34

50 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of Staff Weeks US$, thousands (including travel and consultant costs) Lending FY Supervision/ICR Total Total

51 ANNEX 5. SUMMARY OF BORROWER'S ICR AND/OR COMMENTS ON DRAFT ICR Government of the Republic of Malawi MASAF III APL II PROJECT PROJECT COMPLETION REPORT 1. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN 1.1 Project Context at Appraisal At the time of Project design the population of Malawi was about 12.8 million people spread over the area of 118,484 square kilometers, with 85% living in the rural areas. More than 51 percent of the population lived below the poverty line of whom 22% was ultra-poor and 30% asset poor but with labour surplus. The Malawi Growth and Development Strategy (MGDS II) recognizes that strong and sustainable economic growth is key to reducing poverty and emphasizes Social Protection as one of the main areas of focus, a key ingredient for economic growth in an environment where poverty is prevalent. In order to achieve Social Protection objectives, such as protecting the livelihoods of the most vulnerable, food security, improving lives of persons living with chronic illnesses and others, the Social Protection programs in Malawi were designed with a long term view. Repeated shocks and depletion of assets is prevalent in rural areas of Malawi leading to pervasive vulnerability. Economic shocks have also exposed urban populations to vulnerability and poverty is increasing rapidly in the urban areas than in the rural areas. Government of Malawi (GoM) through the MASAF III APL II Project has made concerted efforts to move towards long term predictable social protection programs that help the poor households deal with risks and shocks in a coordinated and institutionalized approach. Guided by the Malawi Social Protection Policy, the designing, funding and implementation of Social Protection projects has changed. MASAF s contribution to the country s SP agenda has helped GoM to develop and implement sophisticated methodologies for poverty and vulnerability targeting; it has also helped build capacities of Local Authorities to involve and focus programming on poor and vulnerable households; it has also empowered communities so that they are heard and represented in decision making arenas and designed monitoring and evaluation systems for social protection indicators. 36

52 The objectives and design of the MASAF 3 APL II have also helped to mainstream social protection programs across sectors at the national and local level. It has also strengthened the delivery of social services in line with decentralization at the Local Authority level. In January 2008, the GoM prepared and circulated a Position Paper on the LDF that laid out the institutional architecture and the basis for financing local development using four windows of the LDF. The Fund was created to foster local government capacity building and promotion of transparency and accountability in the delivery of services by inputting the results from annual assessment exercise into a Capacity Development Program of the Ministry of Local Government and Rural Development. 1.2 RATIONALE FOR SEEKING WORLD BANK SUPPORT The GoM sought World Bank support for a Social Fund in 1995 for MASAF I as part of a strategy to support Social Dimensions of Adjustment as economic reforms got underway after the 1994 elections. A second phase of the MASAF commenced in 1998 when MASAF I was fully committed without meeting the expressed community demand. The MASAF I and II Projects received a total of US$120 million between 1995 and 2002 to support the construction of schools, water points, bridges, clinics and other small investments implemented by communities; Public works implemented by Local Authorities. Over 8,000 community sub-projects had been completed but there was still a large unmet demand for similar investments. Government of Malawi then developed a Community Empowerment and Development Project in 2003 called MASAF 3 to be implemented over a 12 year period to support the evolution of Local Authorities to support direct financing of community projects while at the same time building the capacity of the Local Authorities. The Bank undertook to finance a three-phased Adaptable Program Lending (APL) instrument during in order to improve service delivery at the Local Authority using the Community Demand Driven (CDD) approach. The plan was for the APL to support community efforts that contributed to the attainment of 12 out of 48 MDGs indicator targets identified as likely to benefit from these community efforts and contribute to Malawi s progress towards the attainment of Human Development-related MDGs within the national framework of MGDS. MASAF has been the main instrument for linking these central government service delivery efforts with community priorities. MASAF 3 APL II was designed to consolidate Community LA linkages to complement LA- Central Government linkages and function as a capacity building program for LAs so that they could support and report on community efforts in their contribution to the national attainment of MDGs hence promoting accountability and results measurement. It had strong synergies and complemented other projects in the Bank portfolio like those in agricultural sector e.g. the Irrigation, Rural Livelihoods and Agricultural Development Project. It supported poor farmers with Public Works earnings to access subsidized fertilizers and maize seed. 37

53 1.3 HIGHER LEVEL OBJECTIVES TO WHICH THE PROJECT CONTRIBUTES The Malawi Growth and Development Strategy is the overarching development strategy for Malawi. The MASAF 3 APLII Project subscribed to two key thematic areas, namely: Social Development and Social Protection and Disaster Risk Management. 1.4 DEVELOPMENT OBJECTIVES AND DESIGN The Development Objective of the MASAF 3 APLII Project was to improve the livelihoods of poor households within the framework of improved Local Governance at community, Local Authority, and National levels. Progress towards achievement of this objective was to be measured through continuous tracking of PDO outcomes and intermediate results presented in table 5.1 below. Table 5. 1: MASAF 3 APLII Project PDO and Selected Key Performance Indicators Project Development Objective To improve the livelihoods of poor households within the framework of local governance at community, Local Authority, and national levels. PDO Outcomes Outcome Indicators Use of Results Information 1. Increased household incomes. Beneficiaries of Public Works programs To identify community (Number) Person days provided in labor intensive work contributions to the attainment of specified MDGs 2. Capacity of LAs improved (Number) LAs able to set objectives and achieve their annual targets Intermediate Results per Results Indicators for each Component Use of Results Information Component 1. Community Livelihoods Support (CLS) Fund Communities and Local Authorities able to successfully manage (prepare, implement and evaluate) a poverty targeted public works program 2. Local Authority Capacity Enhancement (LACE) Fund Capacity at national and Local Authority level for longer-term planning and financing of service delivery increased 3. National Institutional Strengthening Fund Transparency and accountability for the use of resources at Community, Local Authority, and National levels improved. Amount of total transfers (USD) Safety Net programs being implemented (Number) Targeting errors (beneficiaries complying with eligibility criteria identified through verification exercise) (%) Beneficiaries paid in a timely manner (%) Framework for Intergovernmental Fiscal Transfer System to support service delivery of LA defined and tested by MTR. Local Authority Capital Grants System defined and tested by MTR Appraised Community Action Plans reflected in District Annual Investment Plans by MTR (%) Community level tracking system that delivers information annually on baselines, targeting, and utilization of PWSP wage earnings defined and implemented. Funds lost to errors, fraud, corruption (identified through audit exercise) (%) LAs publicly disclose expenditures by MTR (%) To establish project performance and identify opportunities for system improvement To assess project progress towards improving system performance To ensure that project maintains poverty focus and livelihood, to identify service gaps and inform decision makers to take corrective action To provide - mostly bottom-up - feedback to improve project and system performance The MASAF 3 APLII Project was part of the IDA financing strategy using a three-phased Adaptable Program Lending (APL) instrument ( ) to improve service delivery efforts by communities supported by Local Governments and Sector Ministries through a Community- 38

54 Driven Development (CDD) approach. Through this arrangement, the APLII was expected to satisfy the following three triggers to get into APLIII: i. LAs able to set objectives and achieve their annual targets; ii. Annual PWP cycles completed according to plan; iii. LAs with reporting mechanism for MDG indicator targets (number) (trigger for APL III). 1.5 REVISION OF PDO STATEMENT AND PDO LEVEL INDICATORS Revision of PDO statement: During a World Bank-Government mission to the Project undertaken in April 2010, the PDO statement was revised from the original statement: to improve the livelihoods of poor households within the framework of improved Local Governance at community, Local Authority, and National levels; to the current statement on which basis the Project has been assessed: To improve the livelihoods of poor and vulnerable households and to strengthen the capacity of local authorities to manage local development. The basis for rewording of the PDO statement was to better reflect the direct impact areas of the Project i.e. improvements in livelihoods and capacity of local authorities. REVISION OF PDO LEVEL INDICATORS: At Mid Term Review it was noted that PDO-level results indicators were only partially aligned with the PDO statement as the Key Performance Indicators did not provide direct measurement of improved livelihoods of beneficiaries and enhanced local government capacities. Two new indicators were introduced to replace outcome indicator number 3 LAs able to set objectives and achieve their annual targets and these are: (a) Number of Local Authorities with unqualified audit reports; (b) Percent procurement of goods and services at Local Authority level based on competitive bidding. 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES The implementation of the MASAF 3 APL II project was operating in a changing political and economic environment. The project was affected by both internal and external factors. 2.1 EXTERNAL FACTORS There are a number of external factors that affected project implementation. Some of the factors that were external to the project but had an effect on project implementation are as follows; Political Situation The project was operating in an environment where there were no elected councilors to provide political oversight at Local level. In the course of implementing this project, Malawi experienced different political outlooks. There was a souring relationship between donors and government leading to the suspension of development assistance to the country in The country also lost a sitting Head of State during the period under review creating political uncertainty. However, a smooth transition of power followed the constitutional order with the Vice President taking over as Head of State and Government. Towards the end of the project, the country conducted tripartite polls where amongst others, Councilors were elected bringing hope for improved local level 39

55 governance and service delivery. It should however, be noted that changes in political circumstances have generally not impacted negatively on the overall directions of reforms Economic Challenges Macroeconomic imbalances started to build up after 2008 in the context of the global economic crisis and deteriorating relations with donors. This combination of events led to fiscal and external imbalances that required adjustment by containing domestic demand. By end-2011, Malawi was grappling with severe fuel and foreign exchange shortages which started to choke the economy. In early 2012, real GDP growth for 2011 was estimated to have slowed to about 4.3 percent, and the outlook for 2012 and beyond was extremely bleak Exchange Rate Regulation In May 2012, Government took decisive policy measures to arrest the slowdown in the economy by implementing a comprehensive package of economic reforms. The GoM addressed the overvaluation of the exchange rate, with one-step 50 percent exchange rate liberalization, removal of restrictions on the setting of exchange rates by banks and foreign exchange bureaus for transactions with their customers and removal of the requirement to surrender tobacco export proceeds to the Reserve Bank. This allowed the transfer of the US dollars earned at the tobacco auction floors directly to seller s commercial banks which enable the stabilization of the forex market Exchange Rate Movements between USD/SDR The project lost a total of USD2, 685,224 in exchange rates due to appreciation of the US Dollar against the SDR, the currency in which the financing was denominated. For example, under the original financing of $50,000,000, the amount disbursed was $47,314,776 representing 100 percent disbursement High Levels of Inflation During the course of implementing the MASAF 3 APL II Project, Malawi experienced very high levels of inflation. As of March 2014, inflation rate was recorded at 24 percent and reaching a record high of 37.9 percent in March This affected the number of outputs under the project. For example, the unit estimated cost of a teachers house in 2008 at project preparation was USD15,000. This moved to USD 17,000 by July/August 2009 and in 2010, the unit estimated cost of a teachers house was USD22, 000 depicting an overall increase in prices of over 40 percent in a period of 2 years. This high rate of inflation caused a reduction in the number of teachers houses that were constructed with the credit financing from 1,000 to 808 houses under original financing. High inflation rates also undermine the real purchasing power of money. The wage rates for public works programme were fixed at MK200 per participant per day for the most part of the project and MK300 per day under the Rapid Response Public Works programme. But the high inflation rates reduced the real basket of goods that the beneficiaries could purchase with the wages. 40

56 2.2 INTERNAL FACTORS TO THE PROJECT Delays in Retirement of Imprests Liquidation of funds for public works programme was a setback during project implementation resulting in delays in retirement of imprests or submission of Statement of Expenditure Reports by Councils. This has consequently, led to delayed cash transfers to about 575,000 households for the last cycle of the 2013/2014 PWP. It is generally agreed that the introduction of the SoE financial reporting arrangement in the middle of project implementation in October, 2012, against original project design, was a major contributing factor to delayed retirement of the imprest amongst councils Inadequate Capacity at Council Level The design of the MASAF 3 APL II project was centered on, among other aspects, strengthening the national financial management systems. The design meant that the project would use government systems, particularly decentralized integrated financial systems for project implementation. Inadequate staffing capacity for financial management, internal auditors and procurement at the district level remained weak for the most part of project implementation causing delays Delays in Funding Flows The delays in the funding flows were due to the extended loop before the funds reach the communities. The funding loop was such that funds were being transferred from Credit Account at World Bank, to LDF Designated Account at RBM, LDF Holding Account at RBM, LDF Operating Account at Commercial Bank, NLGFC Holding Account at RBM, Local Authority Operating Account at Commercial Bank and finally Community Bank Account at Commercial Bank. Transfer of funds involved the following institutions LDF-TST, MOF (DAD), Accountant General, Reserve Bank of Malawi, NLGFC, Local Authorities and The Community. 2.3 MONITORING AND EVALUATION: DESIGN, IMPLEMENTATION AND UTILIZATION Monitoring and Evaluation System Design The MASAF 3 APLII Project Monitoring and Evaluation (M&E) system was designed to ensure better planning, targeting, and feedback to relevant stakeholders and enable timely decision making towards improving livelihoods of beneficiary household and governance at the local level. The Project was to rely on decentralised Information Management System (MIS), the Local Authority Management Information System (LAMIS) to capture Project outputs. The Project was also expected to facilitate harmonisation of reporting and strengthen interface between the Project M&E and local governments M&E system as a process towards improving accountability and results management. Social accountability tools particularly, the Community Score Card and Citizen Report Cards were to be the main tools for generating citizen feedback on the performance of the Project, while independent evaluators were to be used to generate evidence of Project outcomes through Beneficiary Assessments; Tracking studies and technical audits. The Results Framework was the main operative framework for tracking project performance towards achievement of the Project 41

57 Development Objective and respective key performance indicators, with key information sources including technical studies by independent consultant; an Impact Evaluation of the Public Works programme facilitated by the Bank and internal assessments such PWP surveys implemented by LA M&E officers and community score cards. Instead of using the LAMIS, it was decided that the IFMIS was going to be the main Information Management System that was going to be used for financial and physical reporting through integration of the project management module in the IFMIS. During the Mid-Term Review, the need for strengthening the role of LA M&E Officers in outcome porting was recognised and subsequently integrated in the monitoring and evaluation arrangements. The Project continued to use own Project reporting framework to satisfy the Project information requirements as harmonised framework could not be achieved Project M&E Implementation The Project M&E has been effective in generating credible evidence through technical studies including: Beneficiary Assessment of the MASAF 3 APLII Project; Beneficiary Assessment of the Crisis Response programme; Public Works Tracking Studies; Community Score Card; Performance Assessment Measures; COMSIP Impact Evaluation; and PWP Impact Evaluation. However, Citizen Report Cards were not able to be implemented within the Project time framework due to the extensive requirements for setting solid ground for implementation and institutionalisation of the CRC in the country such as training of Technical Support Team (TST) and engaging local counter parts consultants. The information particularly achievements under the Public Works Programme as generated through the MIS and technical studies also provided critical inputs into the designing of a successor Project, the MASAF 4 Project; The Project has made substantial impacts in enhancing the skills of LA Monitoring and Evaluation officers in tracking and reporting of outcomes which will go a long way in influencing continued culture of results management in Las (PWP surveys undertaken by LA M&E Officers). 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE Safeguards Compliance The MASAF 3 APL II project triggered 4 Safeguards Operational Policies: OP/BP 4.01 Environmental Assessment, OP/BP 4.09 Pest Management, OP/BP 4.12 Involuntary Resettlement, and OP/BP 4.36 Forests. The implementation progress for safeguard activities continuously improved over the course of project implementation from moderately unsatisfactory to satisfactory. The project prepared an environmental and Social Management Framework and a Resettlement Policy Framework to identify and mitigate any potential social and environmental impacts arising from the project activities. The implementation of the safeguard measures followed a decentralized approach where the local authorities and communities were responsible for social and environmental screening, preparation of the social and environmental management plans (ESMPs), and preparation of the Resettlement Action Plans (RAPs). 42

58 In the initial years of the project, implementation progress on safeguards was rated moderately unsatisfactory owing to the fact that councils and communities were not undertaking screening of the potential impacts. The Government moved in and recruited 27 Environmental District Officers (EDOs) who coordinate the social and environmental activities at local authority level and serves as secretary to the District Environmental sub Committee (DESC). A total of 2,666 sub projects were screened for social and environmental impacts under public works programme and a total 2,522 ESMPs were developed out of which 1,804 ESMPs were implemented under PWP. Under the Primary School Staff Housing Programme, all the projects were screened for social and environmental safeguards and the ESMPs were implemented. A total of 811 Resettlement Action Plans were developed and implemented on all programs including public works, primary school staff housing sub projects and construction of school blocks. The LDF TST organized training to all the 27 EDOs and provided refresher courses on an annual basis to 501 DESC members in conjunction with other projects that were funded by the World Bank in Malawi. The councils also trained a total of 1,316 Area Executive Committee members (AECs) who cascade the information to the grassroots. At community level, a total of 1,956 project management committee members were trained in social and environmental safeguards. The capacity building measures for safeguards implementation that have been developed in this operation will be continued in the next operation. Under the savings and investment activities that are coordinated by COMSIP, a total of 856 groups involving 18,320 members were also trained in social and environmental safeguards screening and preparation of the ESMPs. A total of 179 community facilitators were trained to monitor implementation of safeguards at community level. The TST provided earmarked funding of MK50,000 per sub project for infrastructure projects to help with implementation of mitigation measures at community level. The Ministry of Environment and Climate Change Management hired 27 Environmental Inspectors and placed in various districts to join Environmental Officers who are already established in most districts to strengthen capacity for social and environmental management and monitoring. The Ministry also continued to provide oversight and undertake quarterly monitoring of safeguards implementation process and local authority and community level Financial Management The original project cost for the MASAF 3 APL II was estimated at US $50 million. In the course of implementation there were two Additional Financing credits. The first Additional Financing was US$ 14 million and Second Additional financing US$ 50million bringing total project estimated cost to US$ 114 million (SDR73.4million). At the time of preparing the ICR, the disbursement rate of the project was estimated at 87% or USD101 million of which USD99 million had been spent. Financial Management was affected by a number of factors which included the following: Crashing of the financial management software (sun system) after two years of project implementation which forced the TST to operate on a manual system for over one year. Long loop in funding flow which led to delays in funds reaching the beneficiary local authorities and the communities. Delays in financial liquidations by the Local Authorities due to inadequate capacity which led to delays in replenishment of the designated account. High vacancy rates in the positions of DoF and IA at LA level 43

59 TST and Government have moved to address most of these problems and implemented the following remedial measures as follows: The LDF TST procured a new Financial Management System (TOMPRO), the Bank reviewed and found it to be satisfactory which has improved financial management and recording. The data for all previous transactions was restored in the system. The LDF TST continued to hand hold councils that had difficulties to do financial reports so that they may submit the reports on time. This has improved financial reporting. The LDF TST in conjunction with the NLGFC provided refresher courses to 101 accounting personnel in the Local Authorities and improved their capacity in financial management and reporting. Government recruited and trained Internal Auditors and Chief Accountants that has improved financial management and controls and quality of financial management systems at council level; Procurement Procurement at the Local Development Fund TST, Local Authority and the community levels were undertaken in accordance with laid down procedures. Procurement procedures and contracts administration were generally of good quality, reliable, timely and transparent. The project in conjunction with the Office of the Director of Public Procurement trained IPC members (165 members) and provided them with procurement refresher courses on Public Procurement procedures and processes to build their capacity in procurement. The project provided training to Local Authorities which in turn provided training to the communities on procurements and record management. All reviews of the procurement function found it to be satisfactory in the course of project implementation. The councils have been performing above average with regard to procurement. For example, in the 2012 Local Authority assessment (LAPA) report, the average performance under procurement was 77 percent while during the 2013 assessment; the average performance was at 73 percent. 3.0 ASSESSMENT OF OUTCOMES Rating: Satisfactory 3.1 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVES There is sufficient evidence to confirm that the MASAF 3 APLII Project has achieved its Development Objective to improve the livelihoods of poor and vulnerable households and to strengthen the capacity of local authorities to manage local development through the various interventions implemented under the Project. Table 5.2, below presents performance on PDO level indicator targets as of July

60 Table 5. 2: PDO level indicator results table # PDO level results indicator Baseline Target Achievement 1 PWSP beneficiaries (number) % female 815,557 (APLI) 46% 1,490,125 50% 2,163, % Number of Beneficiaries reached under 81,556 Up to direct support 298, ,394 2 PWSP beneficiaries with savings of at least 50% of PWSP wage one year after participation (of which % female) 3 PWSP beneficiaries that were able to buy agricultural inputs following participation in the PWSP (of which female) 4 Person days provided in labor intensive work in public works (number) 5 Number of Local Authorities with unqualified audit reports 6 Percent procurement of goods and services at Local Authority level based on competitive bidding N/A 48,750 50% 7 Direct Project beneficiaries 5,706,249 beneficiaries (APL I) 68% 714,000 (48%) 78,758 61% female 748,725 (51%) 6,735,820 days 19,500, million (APLI) days 12.5% 60% 62% N/A 70% 94% 7,630, million (cumulative annual beneficiaries) of On specific Project results based on PDO level indictors, the performance of the MASAF 3 APLII Project was as follows: Improvements in Household Income: The 2012 Public Works Tracking Study indicated that 52% of Public works beneficiary households reported to have increased household income as a result of their participation in Public Works programme while 79% of COMSIP group members (against the target of 49%) who were initially beneficiaries of PWP reported to have generated income above 50% of the initial income they received from PWP. Further, the Beneficiary Assessment (2012) reported that 74.1% of beneficiaries indicated that they would not earn more income through any source than PWP (implying that this was an eligible group targeted by the PWP); and that 65.3% of respondents reported to have satisfaction with income earned through community savings groups. The First Tracking study (2011) reported that over 86% of PWP households attributed their income to wages from PWP. Food Security Improvements: Improvements in food security have also been reported that 62% and 54% of PW beneficiaries (2011 and 2012 Tracking Studies respectively) used PW earnings to meet their basic food needs compared to 31% at baseline (PWP baseline survey report, 2009); Fifty percent (50%) of Savings and Investment group members indicated that they were no longer food insecure. The Project made substantial contribution towards accessibility of subsidised fertiliser (67% of PW beneficiaries directly benefited fertiliser coupons-tracking study 2013) with 34.6% and 30%; PW beneficiaries (2012 and 2013 Tracking studies respectively) reported to have procured (fertiliser and maize seed) using PW earnings compared to 24.9% at baseline (PWP baseline survey report, 2009). On the other hand, PWP households in Crisis hit areas (districts 45

61 affected with 2010 drought in the Southern Region) spent 40% of income on food; and 22% on fertilizer (Beneficiary Assessment and Technical Audit of the Crisis Response Programme, 2014). Improvements in Education Outcomes: Towards improvement of education outcomes, the 2011 Beneficiary Assessment indicated that 28.7% of beneficiary primary school teachers have moved from temporary mad and grass thatched houses to relatively permanent or better houses. This is likely to contribute towards improved performance of the teachers driven by satisfaction gained through better living conditions and ultimately improvements in educational outcomes in the rural areas. Expenditure patterns also show that a much larger proportion of PWP beneficiaries, 12.6% and 10.6% from 2011 and 2012 Public Works Tracking Studies respectively used earnings to meet education related expenses such as school fees and purchasing of exercise books compared to baseline 4.4%. The Project through the Primary School Staff Housing Project and the Crisis Response programme have contributed to improvement in education outcomes as follows: i. 34.1% of teachers reported to have reduced distance to school, with significant impact on performance of pupils; ii. For teachers who reported that the house conditions have significant effects on teachers performance (a) 62.3% of teachers indicated that they had more time to prepare for lessons with availability of electricity; (b) 50.7% moved from houses that leaked during the nightaffecting their preparedness for teaching (i.e. could not rest and sleep properly during rainy season); iii. Reduced pupil to teacher ratio in rural areas due to high retention of teachers as a result of permanent houses; iv. Enhanced teacher and pupil performance in rural areas (81% of houses occupied by head teachers thereby strengthening supervision on other teachers, as they are resident at schools); 36.2% of respondents reported that pupils can learn more with increased teacher commitment; v. Ninety-nine (99%) of teachers occupying staff houses expressed satisfaction with the houses, which is a key motivating factor for improved performance. vi. Enrolment rate has increased by 31 percent and 5% in Karonga and Chitipa between 2010 and 2013 as a result of the reconstruction of school blocks and teachers houses that were damaged by earthquakes vii. Retention rate has increased from 98.2 percent in 2010 to 99.9 percent in Chitipa in 2013 viii. Selection to secondary schools increased from 125 students in 2010 to 271 students in 2013 ix. School dropout declined from 0.8 percent in 2010 to 0.01 percent in In Karonga, the dropout rates for girls declined from 3% to 2 percent while that of boys was stable at 2 percent. Contribution towards Health Sector: The Project has also made notable contribution towards the health sector by enabling PWSP beneficiaries pay for medical bills using the PWP cash i.e. 10.6% and 2.7% as reported in 2012 and 2013 tracking studies respectively, compared to baseline of 2.6% at baseline (Baseline survey report, 2009). Improvements in Capacities of Local Governments: The Project tracked two key indicators using annual Performance Assessment Measures (PAM): (i) Number of Local Authorities with unqualified audit reports and (ii) Percent procurement of goods and services at Local Authority 46

62 level based on competitive bidding. The 2011 PAM reported that 62% of LAs were audited. The 2013 PAM indicated that 80% of councils had evidence of records on procurements using competitive bidding. LA s capacity to handle large volumes of resources has been improved as councils were able to manage large volumes of funds disbursed by the Project and effectively delivered Public works operations as well as supporting communities in the implementation of primary school infrastructure subprojects funded by the Project. The Annual Performance Assessment exercise has been instrumental in promoting good practices among councils as improvements have been noted in the key functional areas (i.e. Governance; Participatory Planning; Financial Management; Procurement; Supervision Monitoring and Evaluation; Service delivery; and Capacity building) that are assessed (PAM 2014). 3.2 ACHIEVEMENT OF OUTCOMES/OUTPUTS BY COMPONENT Project implementation is rated satisfactory as the Project successfully delivered its expected outputs Component 1: Community Livelihood Support Fund Local Authority Fund (Public Works Programme) Over the period of the five years the Project has been under implementation, a total of 2.2 million beneficiaries against the target of 1,490,125 have been reached with cash transfers, with a female representation of 50.7% achieved, and an average of 53.9% female representation achieved under the RRP (PWP survey trend analysis report, May 2014). Through the labour intensive works under the PWP, a total of 39,706subprojects have been implemented across the sectors, contributing towards improvements in access to socio-economic services Community Fund Improving Education Investments and Outcomes through the PSSHP The Project through the Primary School Staff Housing programme jointly funded by the Bank and Malawi Government supported construction of primary school education infrastructure with grants disbursed to LAs amounting to US$14.4 million. The objective of the staff housing programme was to enhance the quality of primary education through improved teacher to pupil ratio in the rural areas by attracting more teachers to teach in rural areas. Number of primary school staff housing constructed under the MASAF 3 programme increased from 118 staff houses 808 (MASAF 3 APL1,2008) staff houses constructed under the APLII Project. However, including government contribution, a total of 1, 526 of 1,735 staff houses have been constructed of which 70% were fitted with solar/connected to ESCOM grid. Of these 762 staff houses were funded by Government. The Beneficiary Assessment (2012) of the Project reported evidence of improvements in living conditions of the beneficiary teachers indicating that 28.7% of beneficiaries had moved from temporary houses (grass thatched, un-burnt bricks; poles; mud structures) to relatively better houses. The study further indicated that an enabling learning environment has been created in intervention areas through classroom blocks constructed under the Open Menu. In addition, the Community Score Card reported a score of 76% as excellent and very good on quality of structures under PSSHP. 47

63 Reconstruction of School Infrastructure in Karonga and Chitipa A total of US$5.4 million was allocated for the reconstruction programme in Chitipa and Karonga district councils. The reconstruction programme in Chitipa and Karonga was intended to restore the learning environment that was damaged as a result of earthquakes in 2009 which resulted in damage to school infrastructure in parts of Karonga and Chitipa. The Project disbursed a total of US$4.8 million to Karonga and Chitipa district councils and enabled reconstruction of 63 staff houses and 100 classroom blocks (200 classrooms) against the target of 66 staff houses and 122 classroom blocks (244 classrooms) in the two districts. Additionally, 10 school blocks; 21 teacher s houses; 1 kitchen; 1 girls hostel; 1 laboratory; 2 libraries; 4 office blocks; and 1 VIP latrine were rehabilitated in Karonga district. Improvements in key education performance indicators have also been reported (Beneficiary assessment of Crisis Response programme 2014) Community Savings and Investment Promotion (COMSIP) The Community Savings and Investment Component was intended to mobilize beneficiaries of PWP into savings and investment groups as a process towards improving household residence against shocks, as the households would eventually graduate out of dependency on safety nets. The Project funded proposals from the open menu process prepared by savers mostly from Public works programme and other community level investment. The Project disbursed a total of US$6.1million to the COMSIP Union for implementation of the component. The Project facilitated formation of 4,465 with 99,964 members comprising 33,712 male and 64,441 female. Evidence shows that 79% of COMSIP group members reported to have generated income above 50% of the initial income they received from PWP while 65.3% of respondents reported to have satisfaction with income earned through community savings groups (Tracking studies, 2013). Also, considerable proportion of COMSIP members (50%) reported to moved out of food insecure as a result of their involvement in savings and investment groups Component 2: Local Authority Capacity Enhancement Fund (LACE) The Local Authority Capacity Enhancement (LACE) component supported the development of a comprehensive framework for addressing capacity needs of Local authorities. The main outputs under this component are summarized as follows: (i) Local Development Fund Operational Frame Works that embraced LDF operational procedures and guidelines in form of manuals and hand books were developed shared to various stakeholders for use (ii) A comprehensive Local Authority Performance Assessment (LAPA) system in form of Minimum Access Conditions (MAC) and Performance Assessment Measures (PAM) was developed and used for three consecutive years since (iii) Targeted capacity building programmes devised and implemented based on the outcomes of annual assessments. (iv) Various capacity building interventions were implemented and reached over 23,892 individuals (refer Annex 2b) (v) 590 District Executive Committee members were offered different types of training (vi) 1,205 extension workers/frontline workers were trained in community project management process (vii) 18,148 Village Development Committees were trained on their roles and village action planning process. 48

64 (viii) Performance Based Grant system developed and piloted after MTR since it was introduced after MTR restructuring. A total of MK52.2 million for was disbursed to councils as capacity building grants (ix) Framework for Intergovernmental Fiscal Transfer System to support service delivery of LA was developed by Mid-Term Review, has received Cabinet approval and will be presented to Parliament. Evidence of improvements in the functionality of local governance structures and the LAs was reported particularly under the Beneficiary Assessment study and the Community Score card. The Beneficiary Assessment reported that team work and coordination at LA level has been enhanced as a result of Project implementation arrangements, while at the local level, improved capacity through training of Project Management Committees has been realized i.e. improved capacity to manage community subprojects. In addition, trained COMSIP groups are more coherent and have good performance in savings mobilization. Further, improved community participation has been achieved as the decentralized committees became more effective in mobilizing their communities in other development initiatives Component 3: National Institutional Strengthening Component The National Institutional Component The Project supported National Steering Committees and its subcommittee, the National Technical Advisory Committee to assess compliance of LA subproject proposals if in conformity with various sector norms as well as approval of financial allocations to councils. Technical Support in form of quarterly Implementation Support Missions to LAs and issue based missions were undertaken as part of continuous efforts towards ensuring that the Project was being implemented according to principles and procedures. The component facilitated project reviews through Joint Government-Donor Annual Reviews that were undertaken in 2010 and 2011 which allowed various Cooperating Partners to review the performance of the LDF Mechanism. This forum was an important platform for generating feedback on the performance of the Project. The Mid-Term Review of the Project was undertaken in 2012 during key stakeholder deliberated on Project performance and noted that the Project was on course towards achievement of the Development Objective. Implementation of studies was key in generating evidence on effectiveness of Project interventions in the beneficiary communities. Major studies included: Beneficiary Assessment of the MASAF 3 APLII Project (2012); Community Score Card Process (2010); Public Works Tracking studies (2012 and 2013); Technical Audit (2012); Beneficiary Assessment and Technical Audit of the Crisis Response programme (2014); Impact Evaluation of the COMSIP programme (2014); Implementation of Public Works Surveys in collaboration with LA Monitoring and Evaluation Officers ( ). These study findings were consolidated through a process of Meta-analysis which highlighted areas of common judgements on the performance of the Project as well as facilitating usage of the findings. Other studies were also undertaken as part of perfecting operational systems and these included: Information System Strategy Planning (ISSP) study; Financial Audits; Preparation of Development Communication Strategy; Development of LDF internet and intranet; and support towards rolling out of IFIMIS Project Management module to Local Authorities. 49

65 3.3 RELEVANCE OF THE OBJECTIVES, DESIGN AND IMPLEMENTATION Rating: Highly Satisfactory Relevance of Objectives Given the high poverty incidences in rural parts of the country and unstable macro-economic NLGFC situation in Malawi as evidenced by high inflation and interest rates and low food production among households (up to 2 million people being food insecure according to MVAC report 2013), the objectives of the project in all the three credits remained highly relevant. The project was highly relevant in providing education infrastructure through construction of teachers houses, restoration of learning environment for up to 50,000 pupils whose schools were damaged by earthquakes in Karonga and Chitipa and provided cash transfers to more than 2.1 million households Relevance of Design of the Project Deepening Decentralization The MASAF 3 APL II project was intended to roll out the Local Development Funding mechanism. It was intended to introduce a system of predictable and transparent grant financing mechanism and reporting for the Local Authorities. Although there was functional decentralization, fiscal decentralization has not fully taken place. In addition to that, throughout the project implementation period, there were no councilors to provide political oversight at council level. However, Malawi has conducted elections for Councilors through the 2014 tripartite elections which is expected to improve oversight of the councils and improve financial management Providing Long Term Socio Protection Programs The MASAF 3 APL II socio protection interventions were designed with a long term view, by among others increasing the duration of the PWP from 12 days to 48 days per year and scaling up the savings and investment activities to reach both rural areas where vulnerability seems to be increasing, as well as urban areas where pockets of poverty are growing at a faster rate than in rural areas. This approach remained relevant throughout the programme implementation by scaling up public works programme and savings and investment promotion Provision of Financing for Community Social Infrastructure This sub-component was expected to demonstrate ways of financing community investments in a way that does not undermine sector performance such building schools and clinics without the requisite recurrent funding. 3.4 JUSTIFICATION OF OVERALL OUTCOME RATINGS Rating: Satisfactory The achievement of PDO and the overall performance of MASAF 3 APL II is satisfactory. The relevance of MASAF 3 APL II remained substantial throughout, as reflected in both the Government and the Bank s core policy and country strategy documents. MASAF 3 APL II through 50

66 the public works program, construction of socio infrastructure and retrofitting of education infrastructure was the main vehicle for the delivering national safety nets, increasing incomes of the poor and improving their household food security and consumption and improving education outcomes at community level. The vast majority of the sub-projects portfolio is operational and delivering benefits to the population. Economic efficiency was substantial, as MASAF s CDD approach delivered basic infrastructure at a lower cost than comparators and safety net parameters follow international guidelines for cost-efficiency. The assessments of savings and investment activities found evidence of profits and asset accumulation. The PWP beneficiaries accumulated assets such as livestock, land and oxcarts. 3.5 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS Poverty Impacts, Gender Aspects and Social Development The PWP programme achieved its objective of increasing cash incomes and reducing food insecurity among poor communities. The programme provided employment opportunities for both men and women. The larger proportion of beneficiaries (60%) has been women (Beneficiary Assessment Study Feb, 2012). A large proportion of wages (55%) received by beneficiaries was spent on direct purchase of food and (14%) was spent on purchase of farm inputs (PWP Cash Transfer Tracking Study, January 2012). Hence, a large proportion of the wages go into purchases that are directly addressing the problem of food insecurity. The PWP sub projects were integrated with the COMSIP program by creating opportunities for community savers and entrepreneurs especially beneficiaries of PWP to increase their incomes as well as their livelihoods. Capacity building interventions under COMSIP have facilitated the mobilization of savings for investment which stood at MK893.8 million (U.S$2.2 million). COMSIP members of which 62% are women (Beneficiary Assessment Study Feb, 2012) who in Malawi are also more disadvantaged in terms of access to finance (GoM, 2012, Draft National Gender Policy) have benefited from financial services which have been used for investment in micro enterprises Social Development The COMSIP Programme has assisted in bringing people, especially those from the same geographical areas, together. During group meetings members discuss progress on achievement of social and community development indicators focusing on areas such as housing, education and health. Members take building better homes, accessing medical care and sending children to school as obligations of being in a COMSIP group. The PSSHP has contributed to retaining teachers in rural areas. Construction of Head Teachers houses on school premises has increased head teachers accessibility. (Beneficiary Assessment Study, 2012) Institutional Change/ Strengthening Institutional strengthening intervention implemented under the project mostly aimed at supporting institutional mechanism structures for implementing projects under the LDF. A system of assessing Local Authorities performances in form of Minimum Access Conditions (MAC) and Performance Assessment Measures (PAM) was devised to assist identification of LA capacity gaps in order to devise better targeted capacity development programmes with the goal of improving service 51

67 delivery in the Councils. The system has been in use for the past three years after the baseline assessment conducted in The assessment results revealed progressive LA performance improvement from average scores of 29 percent at baseline to 46 percent in ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME Overall Rating: Moderate The risks to development outcome were cumulatively rated moderate at the close of the operation because adequate mitigation measures were put in place at operation s design stage and interventions implemented during the life of the project. 5.0 ASSESSMENT OF BANK AND BORROWER PERFORMANCE 5.1 BANK PERFORMANCE Rating: Satisfactory Quality at Entry In all the cases, the projects identification, preparation and appraisal procedures were duly followed based on systematic consultation with the Government s Project Preparation Team. The Bank participated in the operationalization workshop of the LDF Mechanism project which was very critical to proper grounding of the institutional and management arrangements of the LDF at national, district and community levels. During the operationalization workshop, a joint Government/ Cooperating partner implementation support framework was agreed upon to help among other things facilitate the realization of a harmonized approach to local development financing, programming and monitoring Quality of Bank Supervision Bank supervision is rated as satisfactory despite the change of Team Task Leaders three times. The Bank managed to share Implementation Support Mission (ISM) objectives in good time. Team composition was generally acceptable although a greater part of the time of the midterm implementation of the project there was weak representation on the National Institutional Strengthening component of the project. Whereas the spirit of the adaptable program lending was to facilitate institutional development of local development financing and decentralized service delivery, the narrower focus on safety nets should not negatively affected the already made gains that would ensure that the LDF as a development financing and oversight agency is sustained. The Bank may wish to follow up on this dimension in future decentralization dialogue with the Government. 5.2 BORROWER PERFORMANCE Overall Rating: Satisfactory Performance of Ministry of Finance The placement of the LDF under the oversight of Ministry of Finance was strategic in term of resource mobilization and centrality of institutional role of the LDF as a mult-sectoral programming 52

68 and financing instrument for local development activities and service delivery. From the onset the Government established a Technical Support Team (TST) as a management agency to perform dayto-day management of the fund. The Ministry of Finance was able to mobilize resources from other donors into the LDF Performance of the LDF Technical Support Team The Technical Support Team has performed very well and met all the management requirements regarding the covenants agreed to between the Government and the Bank. The TST has been able to coordinate the programming of projects from Local Authorities and disburse funding within reasonable timescale. The TST has had tremendous impact on strengthening the capabilities of the Local Authorities in meeting social and environmental safeguards as well as conducting tracking surveys under the M&E function. The TST has exhibited high levels of competence and performance standards as demonstrated by the ability to prepare additional projects in times of crises within very short periods of time. 6.0 LESSONS LEARNED 6.1 POLICY LEVEL The MASAF 3 APLII (LDF Mechanism) was a project specially designed to kick-start a new intergovernmental fiscal transfer mechanism for development financing. The LDF mechanism has demonstrated that with a devolved fiscal transfer system, development outcomes at Local Authority level can be achieved more efficiently. Successful achievement of policy objectives is dependent on consistent commitment from the respective stakeholders in terms of meeting their share of obligations. The institutionalization of fiscal devolution has not been fully implemented because government has not yet devolved the development budget and the cooperating partners still insisting on discreet funding and accountability reporting. 6.2 DESIGN ISSUES The design of the MASAF 3 Project was meant to deepen decentralization and allow councils to plan, manage and implement activities within their priorities which are identified in consultation with the communities. This design required that certain minimum levels of capacity in terms of human resources, equipment and infrastructure existed at Local Authority level to support implementation. The project has assisted to mobilize the necessary capacity to implement projects in a decentralized manner. IEC plays a catalytic role in mobilizing communities and other stakeholders for community development and ensures transparency and accountability. It also assists in clarifying project processes and procedures that contribute to the smooth implementation of the Project. Low wages for the PWP have affected delivery of the initial intention of the project to reduce vulnerability and provide alternative employment to vulnerable people with labour. In future, it is imperative that the wage rate has to be responsive to the economic environment obtaining at the time. 53

69 6.3 INSTITUTIONAL In order to assist communities realize their development needs, efforts such as training and technical backstopping of the Local Authorities are vital and need to be sustained if the Local Authorities are to take over the role of development management. Where such efforts have been sustained and the right numbers of key staff supported, it has contributed to the successful implementation of subprojects and early completions. The public works programme is working as a platform for sustainable socio networks at community level, such as savings and investment and other groups that benefit the communities. 6.4 OPERATIONAL Some Financial Reporting arrangements can be unsuitable for community projects. The Statement of Expenditure (SoE) reporting arrangement for community projects has slowed down the implementation process as the communities have challenges to adapt to the new system. Handholding the communities to assimilate the new accounting method take time and the loop includes the councils to account to the LDF TST before LDF TST accounts to the World Bank. In future, the use of IFRs is advocated as a means of enhancing implementation. Backstopping of the Local Authorities by the LDF TST and the communities by the councils improves quality of infrastructure outputs at the local level. Capacity enhancement and consolidation of capacities over time has shown positive outcomes. The participation of Civil Society representatives proved vital in facilitating Community score card and should be encouraged in future processes. Clusters of COMSIGs are developing into Cooperatives. This is one of the most successful components of the project with signs of self-sustenance. Due to the benefits accruing to individual participants of the groups, mobilization among communities is self-propelling with more people joining in order to benefit from the COMSIP activities. If groups are linked to savings and investment, they can develop into sustainable entities. There is a need to improve quality of community assets created through PWP. This has been a challenge since MASAF I though the project has helped open up rural areas that were initially inaccessible to markets and better social services. Therefore, there is need to ensure that the assets created meet sector norms and standards. This can be improved by ensuring that adequate capacities are available at the Local Authority to support the implementation of the projects and ensure that standards and frameworks are also provided to guide implementation. 54

70 ANNEX 6. LIST OF SUPPORTING DOCUMENTS Bumas International Information System Strategy Plan (ISSP) Study. Centre for Development Management Beneficiary Assessment Study of the MASAF 3 APL II-LDF Mechanism Project Impact Assessment Study (IAS): Savings and Investment Promotion Intervention. D.R. Consulting Services Tracking Study on the Public Works Cash Transfer Program Local Authority Performance Assessment Baseline Data Final Report. Inspired Thinking, Ltd Local Authority Performance Assessment (LAPA) for Local Governments in Malawi 2012 Annual Assessment. Inspired Thinking, Ltd Performance Assessment Measures (PAM) for Local Governments in Malawi Annual 2011 Assessment. Local Development Fund Technical Support Team Construction of School Blocks and Teachers Houses under the Education Sector-Wide Approach (ASWAP) through the Local Development Fund (LDF) Trend Analysis of Key PWP Livelihood Indicators Meta Analysis of MASAF 3 APL II Project Studies Comprehensive Communication Strategy for the Local Development Fund Comprehensive Report of the Community Score Card Process Public Works Programme Baseline Survey Report. Malawi Institute of Management Sector Budget Review Study Development of a Performance Grant System. Malawi Social Action Fund 4 LDF Mechanism: Project Implementation Manual. October Ministry of Education, Science and Technology Education Sector Implementation Plan II (2013/ /18). Ministry of Local Government and Rural Development Annual Report on the Local Authority Performance Assessment (LAPA) for 2012/ Local Development Fund (LDF) Operational Manual. Romana CNM-YBJ (JV) Consulting Engineers Technical Audit of Community & Local Assembly Window Sub-Projects Implemented from 2009 to December

71 Ruo Consultants Beneficiary Assessment and Technical Audit for Crisis Response Programme for Local Development Fund. Prepared for the Government of Malawi. Wadonda Consult Tracking Study on Public Works Cash Transfer in Malawi. Prepared for the Government of Malawi. World Bank Implementation Completion Report: Third Social Action Fund Project in Support of the First Phase of the Community Empowerment and Development Program. Report No: ICR Project Appraisal Document: Malawi Third Social Action Fund (MASAF 3) APL II (LDF Mechanism) Project. Report No: MW Project Paper: Additional Credit on the Third Social Action Fund (MASAF 3) APL II (LDF Mechanism) Project. Report No: MW Project Paper: Additional Credit on the Third Social Action Fund (MASAF 3) APL II (LDF Mechanism) Project. Report No: MW Project Appraisal Document: Strengthening Safety Net Systems Project Fourth Malawi Social Action Fund (MASAF IV). Report No: MW.. All aide memoires and ISRs for the original project and two additional financings. 56

72 ANNEX 7. MAP REPUBLIC OF MALAWI 57