Public Disclosure Authorized IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/24/2010 Report Number : ICRR13256 Public Disclosure Authorized Public Disclosure Authorized PROJ ID : P075129 Appraisal Actual Project Name : Rwanda Project Costs (US$M US$M): 53.3 65.6 Demobilization And Reintegration Project Country: Rwanda Loan/Credit (US$M US$M): 25.0 30.5 Sector Board : SDV Cofinancing (US$M US$M): 25.6 30.7 Sector(s): Other social services (95%) Health (5%) Theme(s): Improving labor markets (25% - P) Conflict prevention and post-conflict reconstruction (25% - P) Public expenditure financial management and procurement (24% - P) HIV/AIDS (13% - S) Gender (13% - S) L/C Number: C3634 Partners involved : DfID, KfW, African Union Board Approval Date : 04/25/2002 Closing Date : 12/31/2005 12/31/2008 Evaluator : Panel Reviewer : Group Manager : Group: Roy Gilbert Kris Hallberg IEGSE ICR Reviews IEGSE Public Disclosure Authorized 2. Project Objectives and Components: a. Objectives: - as per the Development Credit Agreement (DCA): (a) to demobilize a large number of ex-combatants and support their reinsertion and reintegration into civilian life; (b) to facilitate the reallocation of public expenditures from military to social and economic sectors. [The objectives are similarly formulated in the Memorandum of the President (MOP), the only differences being that the MOP gives more details about the reinsertion and reintegration, and specifically refers to a number of 45,000 ex-combatants of the Rwandan Patriotic Army and an unspecified number of ex-combatants of Armed Forces of Rwanda (FAR) to be demobilized. This ICR Review uses the DCA statement of objectives, along with the target number of ex-combatants stated in the MOP.] b.were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? No c. Components (or Key Conditions in the case of DPLs, as appropriate):
A. Demobilization (appraisal cost - US$3.3m.; actual cost - US$3.3 m.) 20,000 ex-combatants of the RPA plus 25,000 ex-combatants of the Rwandese armed groups were to be demobilized through the following activities : (i) establishment of demobilization centers; (ii) transport of ex-combatants to demobilization centers; (iii) verification of ex-combatant status; (iv) provision of and identity card; (v) collection of socio-economic data; (vi) medical screening; (vii) distribution of vouchers to access the "recognition of service" allowance. B. Reinsertion (appraisal cost - 17.2m.; actual cost - US$17.2m.) All demobilized ex-combatants would receive a "Basic Needs Kit" expected to cost US$109 each, and consisting of: (i) food for a period of three months; (ii) seeds and tools; (iii) basic household items; (iv) transport to the community of settlement. Those who were former professional soldiers, would receive service allowances ranging from US$ 330 to US$1,100 each (depending upon rank). C. Reintegration (appraisal cost - US$18.4m.; actual cost - US$18.4m.) After six month demobilization, ex-combatants would receive Reintegration Grant of US$ 217 intended to finance agricultural and non -farm income generating activities, vocational and apprenticeship training, housing, employment, tools and education and scholarships. Reconciliation efforts through participation of reception communities through : (i) assessments of community perceptions of ex-combatants, and ex-combatants' perceptions of communities; (ii) expansion of current national and community reconciliation initiatives; (iii) information and sensitization of ex-combatants and communities through outreach activities, including training of counselors; and (iv) specialized counseling. D. Special Groups (appraisal cost - US$5.6m.; actual cost - US$8.5m.) Attention to groups composed of: (i) women; (ii) the disabled; (iii) the chronically ill; and (iv) child ex-combatants. E. Technical Assistance (appraisal cost - US$2.3m.; actual cost - US$2.3m.) studies and technical assistance in support of project implementation, including a beneficiary assessment of ex -combatants demobilized by an earlier program. F. Program Management (appraisal cost - US$4.0m.; actual cost - US$5.0m.) [This section summarizes the intended activities under each component, as described in the MOP. The ICR does not provide details of exactly what was done or how the funds were spent.] d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: By implementation completion, the Government of Rwanda (GOR) had spent US$4.4 million on the project--significantly more than the US$2.7 million expected at appraisal. Project closing was extended by three years to facilitate :(i) completion of vocational training; (ii) completion of second cycle of apprenticeship training; (iii) completion of second cycle of formal education support; (iv) implementing the construction of dwellings for disabled ex-combatants; and (v) repatriation, demobilization and reintegration of outstanding armed groups in the Democratic Republic of Congo (DRC). But the ICR does not explain why all these actions were not completed within the original 3.5 year time frame. 3. Relevance of Objectives & Design: The relevance of project objectives was substantial. At the time of appraisal, this project was part of the base case scenario of the 2002 CAS. The 2008 CAS focuses on two strategic themes: promoting economic transformation and growth and reducing social vulnerability. The latter includes the promotion of peace and social cohesion through demobilization and reintegration. Thus, project objectives remain consistent with the current CAS. The ICR refers to relevance in general, but not to specific government or Bank priorities or policies. The relevance of project design was modest. The demobilization, reinsertion and reintegration components were relevant instruments for achieving the project's first objective. Given the difficult state of the national economy at this time, however, the design would have been more relevant if the scale of these components had been calibrated with the absorptive capacity of the economy. To achieve an ambitious project objective of shifting public expenditures away from the military to social and economic programs, the project offered only one small and modestly relevant instrument, an economic opportunity structure study. 4. Achievement of Objectives (Efficacy): (a) Demobilize a large number of ex -combatants and support their reinsertion and reintegration into civilian life : modestly achieved. Demobilization was partially achieved under the project, reaching 66 percent of the original intended target (45,000 = 20,000 ex-rdf + 25,000 ex-ag), but only after project implementation had been extended to twice the original three-year period intended. The ICR reports that the target was lowered to 34,000, but this revision was not approved by the Board. The ICR reports high penetration of the project among potential beneficiaries, citing that 100 percent of FAR and RDF ex-combatants were assisted. At the same time, however, the ICR reports that unserved beneficiaries still remain among armed groups in the DRC. Evidence for reinsertion (limited to ex-combatants of the FAR) is thin. The ICR simply reports that the project provided them with reinsertion services in the form of education, vocational training and apprenticeships, without assessing the results of these efforts upon the lives of the beneficiaries and their reinsertion experiences. Similarly, instead of providing evidence of actual reintegration of project beneficiaries into their pre-conflict communities and work patterns, the ICR describes
only the delivery of project reintegration services to them -- vocational training, apprenticeships, adult literacy etc. Some evidence is provided by tracer studies, beneficiary surveys and assessments. These are partially summarized in Annex 5 of the ICR, but the evidence is incomplete and sometimes contradictory. For example, the summary of the 2007 beneficiary survey, provides no precise information on the results obtained. Comparisons between the conditions faced by ex-combatants in 2004 and 2007 are not given. The summary report of the 2008 final payment survey has contradictory findings. It states that all demobilized ex-combatants are fully integrated, yet notes that only 54 percent of ex-combatants from the DRC have been repatriated. (b) Facilitate the reallocation of public expenditures from military to social and economic sectors : modestly achieved. Military spending as a share of GDP fell from 3.4% in 2001 in 2001 to 1.9% in 2005 and 2006, but this fall is more the result of the 2002 cessation of hostilities following the peace deal between Rwanda and DRC. It cannot be attributed primarily to the project. The ICR provides no evidence of increased social and economic expenditures (the project provided only limited means to achieve this ). 5. Efficiency (not applicable to DPLs): Efficiency is rated modest. ERRs were not estimated for this project at appraisal or completion. Nor were unit cost standards or cost effectiveness targets established at the outset. The ICR text (p. 22) reports that 30% of project costs went toward managing the project itself. If correct, this would be an extraordinarily high overhead. But ICR cost tables show a much lower actual overhead of around 9.1%, up from the 7.9% originally planned at appraisal. a. If available, enter the Economic Rate of Return (ERR ERR)/Financial Rate of Return (FRR FRR) at appraisal and the re-estimated estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Although the project delivered substantial assistance for demobilization, evidence that the beneficiaries achieved reinsertion and reintegration is thin. There is evidence of falling military expenditures, but this cannot be attributed primarily to this project. a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The ICR notes that GOR remains committed to the demobilization, but the ICR does not provide evidence of government policy statements or budgets to support this. Although the formal peace accords between Rwanda and DRC still hold, security remains volatille, especially within the borders of the neighboring DRC. Armed groups there, such as the FDLR (Forces Democratiques du Liberation de Rwanda ), continue to destabilize the region, thereby challenging what this project has achieved. The sustainability of reintegration depends on ex -combatants' continued employment and engagement in economic activities, itself dependent upon conditions in the national economy. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: Quality-at-entry was moderately satisfactory. The Bank's response to GOR's request for assistance was prompt and it played an important role in helping to coordinate the support provided by different bilateral donors. The project design was wanting, however, for not providing a clear M&E system that could monitor the achievement of the intended results, rather than simply monitor the delivery of the demobilization and reintegration assistance requested. The quality of supervision was moderately satisfactory. Supervision missions were fielded regularly, providing technical support over the life of the project. Yet supervision's almost exclusive focus upon project implementation meant less attention to monitoring actual project results, particularly the demobilization and reintegration status of project beneficiaries. a. Ensuring Quality -at at-entry Entry:Moderately Satisfactory
b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: Borrower Performance was satisfactory. Ownership of the program was strong. By completion, GOR contributed 60% more counterpart funding than expected at appraisal. The ICR recognizes GOR's commitment to the full return of Rwandan armed groups operating in other countries, but that this proved far more difficult in practice than expected. Implementing Agency Performance was moderately satisfactory. The Rwanda Demobilization and Reintegration Commission (RDRC) performed well in the difficult post-conflict context. Staff turnover was significant. Initially, procurement processing was slow in the early part of implementation, but no cases of mis -procurement were identified. It contributed to delays and only half of the Credit was disbursed by the original closing date, which had to be extended for three years. a. Government Performance :Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: This project was prepared as an Emergency Reconstruction Loan (ERL). The Bank's OP 8.00 governing the rapid response to crises and emergencies requires appropriate oversight arrangements to ensure monitoring of emergency operations. In other words, M&E is very much part of an ERL such as this one. M&E design was modest. A management information system (MIS) was designed, but it only monitored project implementation (the status of beneficiaries and the budgetary support available to help them ), not the achievement of objectives (demobilization, reinsertion, and reintegration ). Successive improvements were made to MIS design, especially to make the database more secure, but its scope did not change. M&E implementation was modest. MIS was implemented throughout the program, but no indicators were added to monitor results. M&E utilization was substantial. M&E provided information on the status of the program deliverables and the number of beneficiaries served at the levels of the (national) technical secretariat, the provincial program office and the community itself. This information was used to adjust program focus in order to concentrate in those areas where performance appeared to be weakest. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Financial management functioned well for the most part, thanks to an local team experienced with World Bank financing procedures. Internal audits were weak at the outset, but strengthened as implementation proceeded. External auditors endorsed the project accounts finding no reason to provide qualified opinions. Safeguards : Following an environmental assessment, the project was rated Category B. The reintegration of ex-combatants was regarded as environmentally neutral and the project did not support major programs of civil works. Procurement : Attention was paid to the training of local staff, who became familiar with Bank procedures and guidelines. The ICR does not report any cases of mis -procurement. 12. Ratings: Outcome: Satisfactory ICR IEG Review Moderately Unsatisfactory Reason for Disagreement /Comments The ICR provides little evidence that the objectives of reinsertion and
Risk to Development Outcome: reintegration of ex-combatants were achieved. There is evidence of falling military expenditures, but this cannot be attributed primarily to this project. Moderate Significant The security situation in the neighboring DRC is volatile. Bank Performance : Satisfactory Borrower Performance : Satisfactory Quality of ICR : Moderately Satisfactory Moderately Satisfactory Unsatisfactory There were shortcomings in both project design and supervision due to a lack of focus on intended outcomes. Borrower Performance was satisfactory but the implementing agency suffered from staff turnover and delays in procurement processing. NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: As reported in the ICR: Sensitization, that is making ex-combatants aware of demobilization opportunities available to them, should begin very early in the post-conflict scenario, and be reinforced if the flow of volunteers falls. The community into which ex-combatants will be reinserted needs to be involved in the planning and implementation of the demobilization and reintegration activities. Psycho-social and mental health support need to an integral part of demobilization. To be effective, they have to be delivered in a timely manner. A demobilization and integration program requires a clear time horizon and exit strategy if it is to retain key staff who would otherwise leave if they perceive a program's continuity to be uncertain. IEG would add: Both the Bank and the Borrower need to remain focused upon the achievement of the intended outcomes. In this project, reintegration support was provided to a large number of ex -combatants, but little is known about whether these beneficiaries were successfully reintegrated into their communities. 14. Assessment Recommended? Yes No Why? To learn more about the actual results of this operation, especially through the reintegration experiences of the beneficiaries and the shift of public spending away from the military. 15. Comments on Quality of ICR: The ICR has a number of significant shortcomings : The ICR does not clearly explain and give full account of how project funding was used to finance the six components. The reader is left without a clear idea of how the money was spent with respect to reinsertion and reintegration, or how the two types of expenditures differed. Reintegration results are not presented in the main text of the ICR. They are partially summarized in the Annex 5, but the data provided is only partial. The ICR notes that IDA involvement in the demobilization program is consistent with the CAS, without specifying to which CAS it is referring, nor which specific CAS priority the project supported. Although stating that the project objectives are relevant to GOR policies, the ICR is imprecise about the specific policies it had in mind and upon what the relevance is based. The ICR refers (p. 15) to 25 evaluation studies that were carried out under this project, but does not report on their findings or methodology. The project cost and financing tables (Annex 1) provide data that are inconsistent with the appraisal figures in
the MOP, and the ICR's actual cost and financing tables are inconsistent with each other. a.quality of ICR Rating : Unsatisfactory