L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-48890,IDA-H Jun ,250,000.00

Transcription

1 Public Disclosure Authorized Independent Evaluation Group (IEG) 1. Project Data Report Number : ICRR Public Disclosure Authorized Project ID P Country Chad Project Name TD-Local Dev Prog Sup APL II Practice Area(Lead) Social, Urban, Rural and Resilience Global Practice L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-48890,IDA-H Jun ,250, Bank Approval Date 18-Mar-2011 Closing Date (Actual) 30-Jun-2015 IBRD/IDA (USD) Grants (USD) Public Disclosure Authorized Original Commitment 25,000, Revised Commitment 25,000, Actual 11,456, Sector(s) General agriculture, fishing and forestry sector(50%):health(15%):primary education(15%):water supply(15%):public administration- Financial Sector(5%) Theme(s) Rural services and infrastructure(50%):rural policies and institutions(25%):decentralization(25%) Public Disclosure Authorized Prepared by Reviewed by ICR Review Coordinator Group J. W. van Holst Pellekaan John R. Eriksson Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives Project Development Objective This project constituted the second phase of a three phase adjustable program loan (APL) to be implemented over a twelve year period (PAD, paras 7-8. Both the Financing Agreement (FA, Schedule 1) and the Project Appraisal Document (PAD, paragraph 21) for the Local Development Program Support Project 2 (LDPSP 2) stated that its Project Development Objectives (PDOs) were "to assist the Recipient in: (a) improving access to basic infrastructures and social services in targeted districts, and (b) improving the planning, management and monitoring by local communities and communes of decentralized investments". These PDOs will be used in this Review to assess the project's achievements.

2 b. Were the project objectives/key associated outcome targets revised during implementation? No c. Components The project had three components. Based on their description and appraised costs in the PAD these components are briefly described below.: (a) Capacity building of local communities and communes and support to decentralization (appraisal cost US$12.00 million; actual cost US3.55 million). The reduction in cost was due to the considerable shortfall in Government counterpart funding and a consequential lower IDA contribution. The aim of this component was to support the development of improved technical and fiduciary skills needed at the different decentralized levels and in the national institutions responsible for decentralization. There were two main activities, namely to (i) strengthen capacity of local communities and communes; and (ii) support decentralization which provided technical assistance to strengthen the capacity of the national institutions responsible for decentralization, focusing especially on newly elected leaders. (b) Decentralized financing of micro-projects (appraisal cost US$56.25 million; actual cost US$14.23 million). The large reduction in the cost for this component was also due to the considerable reduction in Government counterpart funding and the subsequent lower IDA contribution. The component's aim was to increase the availability of basic infrastructures in targeted districts. It supported targeted matching grant financing of demand-driven micro-projects based on Local Development Plans (LDPs) and Annual Investment Plans (AIPs). All the micro-projects would be identified through participatory processes and included in the integrated LDPs and Community Development Plans (CDPs). Micro-projects were intended to promote access to basic socio-economic services, incomegenerating activities, and sustainable natural resources management through the adoption of innovative technologies. Funding would be channeled to communes and local communities to finance: (i) socio-economic infrastructure micro-projects (e.g. education, health, water facilities); (ii) environmental and natural resources management micro-projects (e.g. acacia plantations, sustainable land management, Sahelian gardens); and (iii) rural income-generating micro-projects (e.g. improved seeds, agricultural equipment, drying facilities, small agricultural product transformation and storage facilities). (c) Project coordination and management (appraisal cost US$9.00 million; actual cost US$5.11 million). The lower cost was the result of a reduced project activity. This component was to be implemented by the national Project Management Unit (PMU) which would manage overall project implementation including general coordination functions with all national institutions, especially those charged with decentralization, environmental stewardship, natural resources management, and monitoring and evaluation. The national PMU (supported by seven regional PMUs) would also ensure compliance with the World Bank s procurement, disbursement, financial management, and safeguards policies and procedures. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Costs: The project's total estimated cost at appraisal was US$77.59 million (including contingencies). However the actual cost when the project closed was US22.89 million. The reduction in total cost was due to the reduction in activities in both of the project's operational components due to the considerable reduction in Government counterpart funding. Financing: At appraisal the project was to be financed by an IDA credit of US$13.8 million and an IDA grant of US$11.2 million. However, at the project's close only US$6.65 and US$6.00 million respectively of the credit and grant funding had been disbursed. It was estimated that the project's beneficiaries would contribute $2.25 million during implementation. However in the event no financial contributions could be made because the necessary strategic development plans were not prepared until shortly before the project closed (see Section 4, Objective 2) and the requirement for the beneficiary contributions was waived (see ICR, paragraph 24). Borrower contribution: At appraisal it was estimated that the Government would contribute US$50 million, but by the project's close the contribution was US$10.24 million Dates: The Data Sheet in the ICR records that the date for the Mid Term Review (MTR) was amended from 10/13/2014 to 2/17/2015 but paragraph 52 of the ICR states that the MTR "was not carried out". Indeed paragraph 16 of the ICR confirms that "it never took place". Restructuring of the project was approved on 5/27/2015 but the closing date of 6/30/2015 was not changed (see ICR, paragraph 16 and the Data Sheet). Restructuring: According to the ICR restructuring of the project to take account of its slow progress was to be discussed at the scheduled time for the MTR but for various reasons, including the country's insecurity and severe constraints on field visits, the MTR did not take place. A restructuring was approved only one month before the project closed. It had the effect of cancelling SDR8.7 million of IDA funding and the reallocation of SDR364,000 of the remaining funds from Component 1 (Capacity Building of Local Communities and Communes) to Component 3 (Project Coordination and Management) with a corresponding revision of the disbursement estimates

3 (ICR, paragraph 16). 3. Relevance of Objectives & Design a. Relevance of Objectives In 2000 the Government undertook a comprehensive review of all legislation and regulations relating to decentralization and suggested implementation and further legislative work. As noted in Section 2a these activities were supported by the Bank-financed Local Development Program Support Projects (LDPSP) structured as a three phase adjustable program loan over 12 years. The Bank's Interim Strategy Note for , which was current at the appraisal of LDPSP 2, stated that the Bank's strategy "will provide continued support to local development, either through phase two of the on-going Local Development Program Support Project, or through a new lending operation. Support will focus on the delivery of services based on needs expressed by the population/community, identified and delivered using a community-driven development approach" (paragraph 71). As the PAD for LDPSP 2 stated the Program was part of the World Bank's package of support for the Poverty Reduction Strategy Paper (PRSP) aimed at reducing poverty and promoting sustainable development in rural areas by empowering communities and decentralized authorities and improving access to basic services and economic opportunities at the local level and hence help to address cyclic food crises through the introduction of agricultural income-generating activities (paragraphs 3 and 8). The LDPSP 2 objectives were also relevant to the Bank's support for the Government's decentralization policy by first strengthening the capacity of local governments to plan, evaluate, finance, and manage local development activities, and second by reinforcing the institutional and legal framework for community development (PAD, paragraph 20). The decentralization program was meant to evolve from pilot activities in a few areas to a full-fledged national program within a clearly established framework of decentralization (ICR, paragraph 3). The project was also aimed at supporting the second axis of the Bank's Interim Strategy Note for Chad, namely "improving livelihoods and access to key social services". Finally, the LDSP 2 remained relevant to both the Government and Bank strategic objectives when the project closed in June The Government approved its National Development Plan for in May The Plan focused on (a) developing production capacities and job opportunities, (b) mobilizing and developing human capital and combating inequality, poverty, and social exclusion, (c) protecting environment and combating climate change, and (d) strengthening governance. The World Bank's Country Partnership Framework for Chad for FY16-20 focused on three themes: (a) strengthening management of public resources; (b) improving returns to agriculture and building value chains; and, (c) building human capital and reducing vulnerability. The CPF commented that the strategy provided a renewed emphasis on social services, with a focus on basic health, education and social protection (paragraphs 51 and 52). Rating Substantial b. Relevance of Design As noted in the PAD the LDPSP 2 was designed as part of a national poverty reduction strategy through promoting sustainable development in rural areas by empowering communities and decentralized authorities as well as improving access to basic services and economic opportunities at the local level through investments in a range of micro projects. LDPSP 2 was, following the Government's explicit request to the Bank, aimed at extending LDPSP 1 from 19 districts to a national program of in 63 districts. The two main design elements were community driven development (CDD) investments and capacity building to implement CDD in a decentralized framework. Communities and communes were supported and empowered to formulate development plans which could include micro projects. The project's design was therefore substantially relevant to the project's development objective of "increasing social and economic opportunities for the municipalities rural poor". However, as a practical matter the decision to design LDPSP 2 as a national program in 63 districts was a huge step up from 19 districts in LDPSP 1 and turned out to be problematic because the Government could not adequately support its implementation. While project's design was substantially relevant to the project's objective, the results matrix in the PAD (Annex 1) provided an inadequate summary of the project's design since it was only a listing of objectives, indicators and targets. The matrix failed to provide results chains showing how project inputs were transformed into intermediate and final outcomes..

4 Rating Substantial 4. Achievement of Objectives (Efficacy) PHREVISEDTBL Objective 1 Objective (a) improving access to basic infrastructures and social services in targeted districts Rationale Outputs 127 community micro-projects were completed compared with a target of 938 Nobody ended up working in renovated/constructed livestock production facilities under the project compared with a target of additional classrooms were built for primary level education resulting from the project's intervention compared with a target of 350 No land areas were restored and protected compared with a target of 1,300 hectare Neither income generation micro-projects nor environmental micro-projects were implemented during LDPSP 2. Outcomes 210,850 people were provided with access to improved domestic water sources under the project compared with a target of 690, ,650 people in project areas were provided with access to primary/basic health facilities compared with a target of 200,000 Rating Modest PHREVISEDTBL Objective 2 Objective (b) Improving the planning, management and monitoring by local communities and communes of decentralized investment Rationale Outputs Less than 45 percent of districts had community decision committees in place at the project's close compared with a target of 75 percent 8,409 persons at the local and communal levels were trained in community-based procurement and financial management compared with a target of 20,500 No legal texts on decentralization were approved by the Council of Ministers compared with a target of 12 Outcomes 45 targeted communities had prepared strategic development plans (local and commune) compared with a target of 75 - but these 45 committees did not complete their strategic development plans until shortly before the project closed

5 Rating Modest 5. Efficiency Neither the PAD nor the ICR prepared an economic analysis of the project. The reasons given in the PAD were that the activities to be proposed for the project could not be anticipated because of the demand driven nature of a CDD project and hence an analysis would not be possible, The ICR argued that an economic analysis was not possible because of inadequate data collected by the project. The ICR did, however, provide unit costs for some of the investments financed by the project and compared these costs with those incurred for similar investments in other Bank-financed projects in Chad. It was "inferred that LPDSP 2 was less efficient than other projects in terms of costeffectiveness for micro-projects, particularly for water supply" (paragraph 44). This Review concluded that this project's efficiency was negligible Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) Appraisal 0 ICR Estimate 0 0 Not Applicable 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of objectives and of design were both substantial. However, the efficacy of achieving the project's two objectives was modest because the results achieved were well below the targets established at appraisal. In addition some activities such as income generation micro projects were not implemented at all. The economic rate of return for the project was not estimated. In lieu of an economic analysis the costs for some project investments were compared with similar investments in other Bank-assisted projects in Chad, but it showed that the unit costs incurred for LDPSP 2 were higher than for comparable investments in the other Bank-assisted projects. Hence the project's efficiency was rated negligible. Overall the outcome of this project was rated as unsatisfactory. a. Outcome Rating 7. Rationale for Risk to Development Outcome Rating

6 The sustainability of this project will depend on the sustainability of the capacity of the institutions that were involved in the implementation of the project, such as community organizations and the non-governmental organizations that supported them in the selection, design and execution of community investments. It will also depend on the sustainability of the investments themselves which in turn will depend on cost recovery collections to sustain the financing of maintenance. The ICR reported that there was no study of the "willingness to pay for basic services by end-users" (paragraph 49.). It is therefore unclear if beneficiaries of the investments that were completed would be willing to pay for future maintenance. Financing for a continuation of LDPSP 2 to the proposed phase 3 of the program is doubtful because the Project Management Unit has not been maintained (ICR, paragraph 35) and, based on past experience, the Government is unlikely to provide any additional support of funds for phase 3. This Review concludes that the risk that the limited development outcomes that were achieved by this project will be sustained is high. a. Risk to Development Outcome Rating High 8. Assessment of Bank Performance a. Quality-at-Entry This project's design was based on the program's first phase project (LDPSP 1) which was an adjustable program loan (APL) that established the institutional basis for the program and hence for LDPSP 2. The design was based on a classical strategy of decentralization supported by community driven development (CDD) to provide services to remote areas in Chad that had seen little in terms of social or economic development. While LDPSP 1 had a moderately satisfactory outcome for 19 districts, increasing the scope of LDPSP 2 to 63 districts at the Government's request was overly ambitious and not consistent with the available implementation capacity. It was not a good decision by the Bank to agree with the Government's request to enlarge the scope of the project to a nationwide coverage and to expect that the Government would be able to provide about two-thirds of the total cost. Quality at entry is therefore rated as unsatisfactory. Quality-at-Entry Rating b. Quality of supervision During implementation there were only four complete supervision missions. There were a number of reasons for this. According to the ICR the project was supervised fairly regularly during the first year and half of project implementation ( ). Thereafter there was a hiatus of more than one year between beginning of 2013 and March This hiatus was explained by the late re-assignment of the project's management on the Bank's side. On the Government side, there were three changes in project coordinator and counterpart funding was not being disbursed by the Government and there were frequent changes in the project coordinator. The ICR argued that it was not justified for the Bank to send fully fledged supervision missions to Chad while the Government was not releasing funds (paragraph 51). During other periods security restrictions also constrained field visits for supervision missions. Nevertheless, despite the relatively small number of supervision missions, the Bank provided significant alternative forms of support to the project throughout its implementation. The ICR states that supervision missions were complemented by more focused support on financial management, procurement, and technical issues, as well as through regular communications and audio/video conferences between Bank staff and counterpart staff in Chad. In the Bank's view this support was modestly successful. On the other hand there were some shortcomings in the Bank's role (foreshadowed in the PAD) of sufficiently closely monitoring the project's progress on implementation of the activities and policy measures (paragraphs 73 and 74). For example, there were instances during implementation when forthright decisions were not made concerning the project's progress. Most importantly the Bank, in light of the substantial shortfall in the Government's counterpart funding that seriously undermined the number of micro projects financed (ICR, paragraph 24), was not able to engage the Government in a decision on restructuring the project at a proposed mid term review because of its preoccupation with security concerns. The MTR did not take place and a decision on a much needed restructuring was not agreed. In the event the project's accounts were restructured by a cancellation of SDR8.7 million of IDA funds and a reallocation of the remaining funds among project components just one month before the project closed (ICR, paragraph 16). Given the reduction of almost $30 million in the Government's proposed contribution of $50 million to the project, the Bank appeared weak by not cancelling the project well before the formal closing date. Despite some very positive interventions by supervision missions the overall supervision performance is rated as moderately unsatisfactory.

7 Quality of Supervision Rating Moderately Overall Bank Performance Rating 9. Assessment of Borrower Performance a. Government Performance The Government had, according to the ICR, prepared "a solid strategy for decentralization" which was "at the heart of the government s poverty reduction and equitable socio-economic development as enunciated in the Poverty Reduction Strategy Paper (PRSP). It also made a strong commitment to implementing the project (paragraph 55) and contributing $50 million to the project's cost of $77.25 million to support its request to the Bank that LDPSP become a national; program. However, according to the ICR, the "Government failed to (i) provide the required counterpart resources; it provided only about 20 percent of the resources it had committed itself to allocating to the project; (ii) act on the Bank recommendations, in particular as regards fiduciary matters; and (iii) maintain a high-level committed team for project management" because, for example, a series of three Government Ministers were responsible for the project during its four year implementation (paragraphs 55 and 56). The shortfall in funding delayed implementation at the start of the project and ultimately meant that numerous activities were not even implemented. Weak fiduciary management resulted in delays in procurement, audits and ineligible expenditures which are discussed in Section 11 of this Review. The difficulties with the maintenance of a high level committed team will be elaborated in the discussion of the implementing agency's performance. On the basis of this evidence the Government's performance is rated unsatisfactory Government Performance Rating b. Implementing Agency Performance The project's implementation was placed in the hands of the national Project Management Unit (PMU) for the LDPSP 1. Because of the substantial increase in the project's reach from 19 districts in LDPSP 1 to a national program of 63 districts in LDPSP 2 seven Regional Project Management Units (RPMUs) were established to support the national PMU. The ICR asserted that "the overall project governance arrangements, embodied in the PMU/RPMU set-up, proved weak" (paragraph 57). In particular there was insufficient stability in the project's coordination and management. For example, two project coordinators were removed in three years for wrongdoing, before an interim coordinator was appointed in July 2014 (Restructuring Paper, page 6). In addition there were frequent changes in staffing of the PMU and consequently coordinators and staff had insufficient time or inclination to plan project activities beyond the short term. Weaknesses in the PMU's management of monitoring and evaluation as well as fiduciary issues were serious and are discussed in subsequent parts of this Review. The ICR concluded that "the PMU/RPMU arrangement showed definite weakness right from early project implementation; it remained weak and unable to act decisively" (paragraph 57). This Review rates the Implementing Agency's performance as unsatisfactory. Implementing Agency Performance Rating Overall Borrower Performance Rating 10. M&E Design, Implementation, & Utilization

8 a. M&E Design According to the project's design (paragraphs 33 and 34 in the PAD) the monitoring and evaluation (M&E) unit of the PMU for LDPSP 1 would responsible for the project's monitoring and evaluation (M&E). The PMU was charged with first establishing the baselines for both (i) the consolidation zones (19 departments or districts) where activities during the LDPSP 1 had taken place and were to be further strengthened in LDPSP 2, and (ii) the extension or new zones (all other country departments/districts or 43 new departments/districts) to be covered by the project (ICR, paragraph 28). The design of the M&E system was to be the same as for LDPSP 1 with an additional provision, namely that the responsible ministry for this project (Ministry of Land Management, Urban and Housing - MLMUH) and the Ministry of Planning and International Cooperation - MPIC) would become jointly involved with in the project's M&E. b. M&E Implementation As the ICR noted, M&E implementation was expected to be "relatively straightforward" (paragraph 29). On the contrary at the start of the project there was a two year delay in training M&E staff and a further delay in establishing the baselines and in launching a computerized M&E system. A consultant was hired toward the end of 2014 to make the computer system fully operational which did not take place until close to the end of the project. However, this achievement had little impact because basic data were incomplete. c. M&E Utilization Since baseline data and observations on the project's progress were not available until the end of the project, M&E results were not adequate during the project's implementation. Monthly progress reports prepared were either not submitted or inaccurate. Hence they were not useful for assessing project performance ICR, paragraph 30) M&E Quality Rating Negligible 11. Other Issues a. Safeguards The ICR states that "LDPSP 1 was classified as environmental category B. Hence the micro-projects being financed under this project required a brief environmental and social impact assessment using simple procedures". According to the ICR for LDPSP 2 the environmental assessment was undertaken as part of micro-project preparation but because of limited financial resources available in the project neither the training of beneficiaries nor the remedial plans and actions were implemented. According to the ICR this situation occurred despite efforts by supervision missions to have the appropriate procedures implemented (paragraph 31). b. Fiduciary Compliance Financial Management: The ICR mentions that "All financial management (FM) missions underscored the weakness of financial management arrangements. These missions included action plans to improve the situation. But these plans were rarely heeded and acted upon by the PMU" (paragraph 32). There were continuing FM issues such as the lack of a fully functioning accounting and financial management system, inconsistencies in project accounts, an accounting software that was never fully functional, asset inventories that were not accurate, and ineligible expenditures. It was therefore not surprising that audits were delayed and the 2012 audit had serious qualifications including a lack of alignment of the closing statement for 2012 with the opening statement for 2013 and the existence of ineligible expenditures of CFAF267.4 million. At the project's close audits for 2013 and 2014 has still not been received and CFAF68 million had not yet been refunded to IDA. The extent to which FM was not under control is reflected in a recommendation of the last Bank supervision mission that technical assistance be retained to overcome the FM shortcomings.

9 Procurement: The procurement of goods and services experienced difficulties and delays due to the lack of expertise and high turnover on the part of PMU/RPMUs staff. Staff recruitment under Component 3 proved difficult because of the lack of adequate local expertise and a high turnover of PMU/RPMU staff.(icr, paragraph 34). While the project had chronic problems in this respect and it held up the project's implementation, some of the procurement was handled expeditiously using approved simplified procedures for community based projects. On the other hand the ICR acknowledged that the simplified procedures were never codified in a manual and compliance with approved procedures was difficult to verify ex post (paragraph 34). c. Unintended impacts (Positive or Negative) The Government's eagerness to rapidly expand the coverage of LDPSAP 1 from 19 districts to 63 overwhelmed its capacity to manage the project or to finance it. The negative consequence was an unsatisfactory outcome and an unintended impact of the project's design. d. Other None 12. Ratings Ratings ICR IEG Outcome --- Risk to Development Outcome High High --- Bank Performance --- Borrower Performance --- Quality of ICR High --- Reason for Disagreements/Comment Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR suggested an excellent set of lessons. In brief they were: Development objectives (PDOs) and the scope of interventions for projects supporting decentralization need to be realistic. PDOs should be less ambitious and more realistic for countries with limited implementation capacity and a fragile political environment. CDD operations need to adapt to the capacity of national institutions. For example, an important lesson emerging from both LDPSP 1 and 2 is that bottom-up development projects that use CDD approaches are challenging to implement and scale up, because they require extensive support with respect to capacity building. Multi-disciplinary expertise is required in support of decentralization. As they build on participatory mechanisms and reach out to all segments of the local population at the grassroots, decentralization programs are necessarily cross-cutting and therefore complex. Innovative procedures are necessary to work in the adverse context of fragile states. For example, the context of fragile states where the security situation often does not permit field work, it is necessary for both the Bank and the Government to adopt innovative ways of doing business. Project remedies should be clearly identified as different from overall dialogue between the Bank and the Borrower. The rules of Bank engagement with Government counterparts for each project should be clearly spelled out both internally and with the Government. For example, it should be made clear what the Bank aims to achieve without compromising either the individual project objectives or the broader dialogue with the Government

10 Realistic implementation risk assessment should result in considering alternative institutional arrangements. For example, the risk of failure or inadequacy of project arrangements was not flagged as high as part of LDPSP 2 implementation, and, therefore, no particular alternative arrangements were considered to mitigate this risk should it happen. This Review suggests an additional lesson: The Bank need not always comply with Government requests on project designs. In this case the Government insisted on a national scope for the LDPSP 2 following the LDPSP 1 pilot. The Government put pressure on the Bank by agreeing to contribute almost two-thirds of the total project cost. The Bank did not agree to the project's proposed national scope but pursued the excessively large project anyway. Irrespective of the Bank/country relations this was obviously the wrong decision and a costly one for the Government in terms of its reputation among the population. It was also costly for the Bank's reputation as a trusted partner. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR was well written according to the OPCS guidelines. The text reflected a thorough investigation of the project's problems and data availability. The presentation was candid, internally consistent, comprehensive but concise. The list of lessons was excellent and generally applicable to other similar projects. The only shortcoming was some missing data on Government costs by components in Annex 1 (a) which was probably due to the accounting weaknesses of the PMU when the ICR was prepared. Following a request from IEG the project's task team leader was able to provide the missing data. a. Quality of ICR Rating High