Actual Project Name : Rural Poverty Reduction Project - Pernambuco Country: Brazil. Project Costs (US$M US$M):

Transcription

1 Public Disclosure Authorized IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 04/29/2013 Report Number : ICRR13525 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized PROJ ID : P Appraisal Actual Project Name : Rural Poverty Project Costs (US$M US$M): Reduction Project - Pernambuco Country: Brazil Loan/Credit (US$M US$M): Sector Board : ARD Cofinancing (US$M US$M): Sector(s): Other social services (35%) Roads and highways (25%) General agriculture fishing and forestry sector (20%) Power (10%) Sub-national government administration (10%) Theme(s): Participation and civic engagement (25% - P) Rural services and infrastructure (25% - P) Rural non-farm income generation (24% - P) Rural policies and institutions (13% - S) Other social development (13% - S) L/C Number: L4625 Partners involved : Board Approval Date : 06/26/2001 Closing Date : 06/30/ /31/2010 Evaluator : Panel Reviewer : Group Manager : Group: John R. Heath Ridley Nelson IEG ICR Review 1 IEGPS1 2. Project Objectives and Components: a. Objectives: (I) ORIGINAL PROJECT The statement of project development objectives in the Project Appraisal Document (PAD) is similar but not identical to that in Loan Agreement (LA). According to the PAD, "The project aims to assist the State of Pernambuco to reduce currently high levels of rural poverty by: (a) improving well-being and incomes of the rural poor through better access to basic social and economic infrastructure and services and support for productive activities, using proven community-driven

2 development (CDD) techniques; (b) increasing the social capital of rural communities to organize collectively to meet own needs; (c) enhancing local governance by greater citizen participation and transparency in decision-making, through creation and strengthening of community associations and Municipal Councils; and (d) fostering closer integration of development policies, programs and projects at the local level, by assisting Municipal Councils to extend their role in seeking funding, priority-setting and decision-making over resource allocation" (pp. 2-3). According to the LA, "The objectives of the Project are: (a) to increase social and economic opportunities for the Municipalities' rural poor by improving access to basic, social and economic infrastructure through Community Subprojects; (b) to increase the social capital of rural communities to organize collectively and meet their own needs ; and (c) to foster local governance and citizenship through creation and strengthening of Municipal Councils, at the same time forging links with governmental and nongovernmental agencies, civil society, financial institutions and the private sector" (Schedule 2, page unnumbered). (II) ADDITIONAL FINANCING The additional financing that was approved in March 2007 had the same objectives as the original project. The statements of project development objectives in the Project Paper and the Loan Agreement were identical. According to the Additional Financing LA, "The objectives of the Project are: (a) to increase social and economic opportunities for the Municipalities' rural poor by improving access to basic, social and economic infrastructure through Community Subprojects; (b) to increase the social capital of rural communities to organize collectively and meet their own needs ; and (c) to foster local governance and citizenship through creation and strengthening of Municipal Councils, at the same time forging links with governmental and nongovernmental agencies, civil society, financial institutions and the private sector; and (d) to scale up the impact of the Original Project by using the social capital created by the Original Project to further increase incomes of the rural poor and by fostering closer integration of development policies, programs and projects in rural areas at the local level, by assisting Municipal Councils to extend their role in seeking funding from, setting priorities for and making decisions concerning the allocation of resources from other programs beyond the Project" (Schedule 1, p. 5). IEG evaluates the project against the statement of objectives contained in the PAD, because this makes clear that the project's overarching objective is to reduce rural poverty: sub-objectives (a)-(d) in the PAD are treated as intermediate outcomes; all five outcomes will be assessed. b.were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): 1. Community Subprojects (Expected cost at appraisal, US$34.3 million; Additional financing, US$31.2 million; Actual cost at closing, US$67.9 million). This component provided matching grants to rural community associations to identify infrastructure, social and productive investments (subprojects) that would improve community well-being, each subproject costing a maximum of US$50,000. There were three separate channels for financing subprojects: State Community Schemes (PAC). Rural communities submitted their investment proposals directly to the State Technical Unit (the project implementing agency), which screened and approved them and released funds to the beneficiary associations. Municipal Community Schemes (FUMAC). Decision-making on investment proposals was delegated by the State Technical Unit to project Municipal Councils, composed of community members and representatives of civil society and municipal authorities. At least 80 percent of Council voting members were potential project beneficiaries and civil society representatives. The Municipal Councils discussed, and sought to build consensus on priorities and approve community proposals, in the context of an indicative annual budget amount determined by the state. Pilot Municipal Community Funds (FUMAC-P). The State Technical Unit established an annual budget envelope, according to a distribution formula based on clear and measurable criteria (rural population, poverty levels and previous year's performance). Based on this budget, Municipal Councils submitted an Annual Operating Plan (Plano Operativo Anual) for review by the State Technical Unit. Upon approval, funds were

3 transferred to the Municipal Council, which was then responsible for managing their distribution to community associations and assisting them with implementation of subprojects. 2. Institutional Development (Expected cost at appraisal, US$2.7 million; Additional financing, US$5.4 million; Actual cost at closing, US$7.8 million). This component financed technical assistance and training to build the capacity of the Community Associations, the Municipal Councils and the implementing agency. It also included modest funding to support state institutional modernization and reform related to poverty reduction programs and policies. 3. Administration, Supervision, Monitoring and Evaluation (Expected, US$1.0 million; Additional financing, US$1.7 million; Actual cost at closing, US$4.5 million). This component financed the costs (excluding salaries) of project administration and coordination including supervision, monitoring and impact evaluation. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Costs Under the original project, the total cost was estimated as US$40.0 million at appraisal, rising to US$40.7 million at closing. Under the additional financing, the total cost was estimated as US$40.0 at appraisal, and was US$39.5 million at closing. Financing The loan agreement was amended twice to accommodate reallocation of funds between components and extensions of the closing date; none of these amendments involved Board approval. In 2005, US$8.0 million from the original project was reallocated, mainly involving a shift of funds from the Pilot Municipal Community Funds (FUMAC-P) subproject window (explained in Section 1c above) to the Municipal Community Schemes (FUMAC) subproject window. Under additional financing, a further US$1.3 million was reallocated between components, partly to allow for increased spending on operating costs and supervision. Borrower Contribution Adding funding from the original project to the additional financing, the expected Borrower contribution was US$19.9 million, and the actual contribution was US$21.1 million. The Borrower contribution included both the counterpart from the state government and the funds contributed by Community Associations; the ICR does not give their respective shares. Dates For the original project, the expected loan closing date (June 30, 2005) was extended twice: first, to June 30, 2006; and then to January 31, The first extension sought full disbursement of the Bank loan that was financing the original project. The second extension provided a bridge from the original project until the additional financing proposal was presented to the Board. The closing date of the additional financing was not extended and it closed at the end of its allotted three years. 3. Relevance of Objectives & Design: Relevance of Objectives (Rating: Substantial) Throughout the implementation period the relevance of this project s objectives was consistently endorsed in the Borrower and Bank's strategy. The overarching objective of rural poverty reduction was relevant because, when the project was appraised in 2001, 65 percent of rural families in the state of Pernambuco lived in poverty. Data from 1999 showed that 87 percent of rural households in Pernambuco lacked piped water supply, 54 percent were without sanitation, and 20 percent had no electricity. The Borrower attached particular importance to the aim of using the subproject process to leverage financing from sources outside the project (sub-objective (d) in the Project Appraisal Document) a new departure for this series of community-driven development projects, one that was relevant because it increased prospects for sustaining the drive to reduce poverty. The Bank s FY Country Assistance Strategy (dated May 24, 2001) identified poverty and inequality reduction

4 as the core of Bank assistance efforts and stressed well-targeted, decentralized programs, social capital formation and local integration of programs. The Additional Financing accorded with the FY CAS (dated November 10, 2003), which called for successive projects under the Northeast community-driven development program to finance basic infrastructure for the rural poor, support income generation, and promote closer integration of state and federal rural initiatives in participating municipalities. The Country Assistance Strategy that was current when the loan closed (covering the period 2008 to 2011) acknowledges the validity and the success of the series of Northeast community development operations of which this project is a part, "both in building social capital and in enabling poor communities to get access to water and electricity" (p. 10). However, the FY08-11 CAS did signal a change of emphasis, indicating that in response to demands from state governments in the Northeast, the Bank would henceforth address the development needs of rural municipalities by prioritizing the promotion of productive subprojects connected to high-value supply chains (e.g. contracts with supermarkets). This signaled a shift of emphasis away from the rural community infrastructure investments, promotion of which was central to the project. Relevance of Design (Rating: Substantial) Design relevance was enhanced by the incorporation of lessons learned from the two earlier community-driven development projects in this series of operations for Pernambuco. The project components were sufficiently few in number, clear in conception, and flexible in implementation to make it more likely that the objective and intermediate outcomes would be achieved. Attainment of project objectives was facilitated by a process that promoted community commitment to the investments by involving would-be beneficiaries in identifying subprojects, choosing between investment alternatives, and cofinancing of the subprojects that were selected. The components included the training and technical assistance to Community Associations and Municipal Councils that was needed to make them more effective and more sustainable agencies for local planning. Equally important was the leaning by doing that implementation entailed. The availability of a pre-existing menu of tried and tested subproject options helped to make these small-scale investments more feasible. The existence of three distinct windows for sub-project financing made it possible to tailor the project process to the institutional capacity of the various communities, with greater delegation of decision-making to communities with greater capacity (using the FUMAC and FUMAC-P windows, explained in Section 2c above). The transparent and participatory process of subproject selection helped to ensure that decisions reflected the will of the majority of would-be beneficiaries, reducing the scope for elite capture (although possibly limiting the opportunity for targeting investments to the poorest members of each community).project design ensured that around 90 percent of project funds flowed directly to beneficiaries, increasing the scope for attaining the overall objective of poverty reduction. 4. Achievement of Objectives (Efficacy): (a): Improve well-being and incomes of the rural poor through better access to basic social and economic infrastructure and services and support for productive activities, using proven community-driven development (CDD) techniques. (Rating: Substantial) Outputs Adding the results under the original project to those achieved under additional financing, 96,398 families were served (65 percent of the combined target) and 2,248 subprojects were approved and financed (86 percent of the combined target), from among the 2,541 proposals. Subprojects were distributed among the following categories: infrastructure (59 percent), social (35 percent) and productive (6 percent). Household water cisterns accounted for the largest number of subprojects (38 percent), followed by household sanitation (28 percent) and communal water supply (13 percent). Subprojects were selected from a menu of tried and tested options, helping to ensure they were technically sound. Ninety percent of subproject proposals were approved for financing following appraisal. A physical performance study in 2010 confirmed that, in most cases, subprojects were being soundly operated (ICR, p. 21). Under additional financing, beneficiaries reported that 60 percent of the subprojects delivered had satisfactory arrangements for operation and maintenance. The project team reported that, taking the two financing phases together, targets were not met because the average cost of subprojects was higher than expected owing to exchange rate changes and inflation; and also because the number of eligible families had been overestimated. Outcomes With respect to targeting, across the two phases of financing, 70 percent of the subprojects were implemented in

5 Area 1, which comprised 110 of the poorest municipalities in the project area (61 percent of all municipalities served by the project) (ICR, pp ). The ICR (p. 15) says that preliminary results showed an average increase of 22 percent in incomes between August 2003 and July 2004 (but this accounts for only one year in the eight years between loan effectiveness and closing; and it is not clear how much of the increase can be attributed to the project). Only 6 percent of the subprojects fell into the "productive" category, reducing the scope for increasing incomes and employment. The employment increment from productive subprojects (agricultural mechanization, agroindustry, grain processing and fish breeding) amounted to 9,507 jobs (ICR, p. iii). A 2009 survey of 300 productive subprojects reported that these subprojects had significant impact on quality of life of the beneficiaries, although this did not always translate into increased income (ICR, p. 43). Survey evidence from IPEA (2012) shows that in Pernambuco during , rural household per capita income rose from R$121 to R$193. This would include project areas and other rural areas in Pernambuco; it is unclear how much of the improvement would be due to the project as opposed to general improvements in the economy or other social protection programs. According to the Campinas Economic Foundation (FECAMP) survey of 2004, 86 percent of project beneficiaries reported significantly improved living conditions from 2000 (pre-project) to 2002/2003 (ICR, p. 15). The same study found that the provision of safe water access reduced water-borne diseases by 50 percent. The project team subsequently provided evidence that access to potable water in rural Pernambuco doubled from , from roughly 20% to 40%; this would include project areas and other rural areas in Pernambuco and reflect both results from the project and other government programs in improve access to water. (b): Increase the social capital of rural communities to organize collectively to meet own needs. (Rating: Substantial) Outputs The number of Community Associations and Municipal Councils formed under the project, and the training they received, may be construed as evidence of increased collective organization. Combining results under the original project and additional financing, 2,216 Community Associations were created (there was no target; there were around 4,500 communities in the project area). 101 Municipal Councils were created, against a target of 38. The number of training courses delivered to Community Associations and Municipal Councils by project end was more than eight times larger than the target. Outcomes The ICR (p. 16) says that social capital is displayed in social solidarity, confidence, mutual cooperation, linkages, access to diverse institutions and to the resources and information of a range of assistance programs outside the Bank-supported project. A Social Capital Index was part of the project design but the necessary data were not collected to assess change between the start and finish of project implementation. Most of the evidence presented for social capital formation consists of beneficiary opinions, and is sometimes mixed up with assessments of the benefits from subprojects. In 2004, the Campinas Economic Foundation (FECAMP) survey, which covered 8,602 beneficiary families and 493 control families, found that over 85 percent of beneficiaries agreed that subproject execution contributed to unify the community and 90 percent of the interviewees reported that the subprojects encouraged the participation of members during implementation and afterward. Also, beneficiary Community Associations compared with non-beneficiary Community Associations were more capable of: (i) resolving internal conflicts; (ii) responding to communal demands; (iii) effectively advocating for their membership; (iv) solving local problems for the community; and (v) mobilizing financial and human resources. (c): Enhance local governance by greater citizen participation and transparency in decision-making, through creation and strengthening of community associations and Municipal Councils. (Rating: Substantial) Outputs Women and indigenous groups were adequately represented in the Municipal Councils. Under the additional financing, the Councils became independent legal entities and there was a state-wide process of unifying the councils that had been generated by separate programs 98 percent of the councils are now unified. However, none of the Municipal Councils created were of the FUMAC-P type (see Section 2c above for explanation), which was, according to the project design, the most advanced model of decentralized decision-making and administration; the target was to create 14 of these councils. Outcomes There is some overlap with the previous objective with respect to the participation element. A survey with use of appropriate controls found that over 85 percent of interviewees were persuaded that communities had been unified by the project; other evaluations found that 85 percent of Community Associations view Municipal Councils as representing community interests. According to beneficiary surveys, community members said they were better

6 represented and that decision making about municipal planning was more transparent. The capacity of Community Associations to represent the community and resolve community problems was rated highly in 80 percent of cases (ICR, p. 19). The ICR (p. iv) cites evaluation studies which found that 85 percent of Community Associations view Municipal Councils as effective in representing community interests and channeling project information; 65 percent of Associations said that their operations had been strengthened through the work of the Municipal Councils. On the other hand, the most decentralized model of Municipal Councils (FUMAC-P) was not implemented. This the ICR attributes to (i) poor results on the ground; (ii) the Borrower s reticence in delegating sub-project prioritization and financing to Municipal Councils; and (iii) institutional roadblocks (legal and procurement) making FUMAC-P non-viable (pp vi-vii). (d): Foster closer integration of development policies, programs and projects at the local level, by assisting Municipal Councils to extend their role in seeking funding, priority-setting and decision-making over resource allocation. (Rating: Modest) Outputs "About 70 percent of the more advanced Municipal Councils were actively deliberating the resources from other programs" (ICR, p. 20). Outcomes Data from the ICR (p. 17) show that the project leveraged some US$25 million under its integration objective: less than envisioned at appraisal yet nonetheless 41 percent of the combined loans under the Original Project and Additional Financing. The project team subsequently reported that only about 6 percent of the Municipal Councils had actually received funding from other programs. Also, the ICR states (p. 18) that only 16 percent of Community Association members had established links with other associations. "There is no evidence that during project implementation the proposed partnership with credit institutions worked as planned" (ICR, p. 6). (e) Assist the State of Pernambuco to reduce currently high levels of rural poverty. (Rating: Substantial) The project team provided additional evidence that the incidence of extreme rural poverty declined from 37 percent to 22 percent of the population in Pernambuco. There are no doubt many factors affecting this reduction including the expansion of social safety net and rural water supply programs and a general economic improvement in Brazil. However, the extent of targeting and the evidence of improved access and incomes suggest that the project activities likely contributed to poverty reduction in the project area. 5. Efficiency (not applicable to DPLs): Given the demand-driven nature of the projects in this series the distribution of funds between the various categories of subproject could not be known in advance, explaining why a project-wide economic rate of return was not estimated at appraisal. The PAD (p. 14) makes it clear that efficiency should be assessed on the basis of: (a) cost effectiveness (logically, of infrastructure and social subprojects) ; (b) the financial rate of return to productive subprojects; and (c) the fiscal savings reaped from not providing infrastructure through other, less cost-effective, public programs. Benefit-cost analysis. At closing, the ICR reports that a financial rate of return was estimated for 29 subproject "case studies" (equal to 1 percent of all subprojects financed), comprising the four most common subproject types: water supply, domestic sanitation, farm tractors, and honey production. However, the cases were not randomly sampled; the project team noted that the cases represented subprojects considered by knowledgeable people to be moderately successful. Although the sampled units were not limited exclusively to very successful subprojects, they excluded unsuccessful projects and the overall share of successful projects is not known. For the cases studied, all of which were at least moderately successful, under different cost scenarios for productive subprojects, the financial rate of return for tractors was percent and for honey percent. The ICR estimates the benefit stream for water supply and home sanitation by imputing a value to the time of family members freed from having to fetch water, the incremental employment generated by water availability, reduced disease burden, and the increased value of houses due to the investment. The financial rate of return thus estimated was percent for water supply and 8-11 percent for home sanitation. The project team confirmed that operation and maintenance projections were included in the rate of return calculations. Fiscal savings. There is some evidence of the project s fiscal impact. Foregone expenditure for water distribution via tanker trucks are significant in those communities where water supply subprojects were executed. Some 49,000 families benefited from water supply subprojects at a combined investment cost of US$32.9 million. For virtually all of these families, the only viable alternative for water supply would have been tanker trucks at an overall monthly

7 recurrent cost of US$2.4 million. Cost-effectiveness. The region provided some comparative cost figures suggesting that public investments of a similar type were more costly than the investments made by communities. Project costs were reported to be between 47% and 77% of the non-project cases. However, it has been difficult to assess the comparability of these estimates to what was actually implemented for this particular project. Area targeting reduced the scope for leakage of project funds to the non-poor. The actual cost of project management (Component 3) was 6 percent of total project cost, consistent with claims in the PAD about the limited overheads involved in this approach to community-driven development. The price benchmarks that the implementing agency used for labor and building materials were market-based (ICR, p. 21). On the other hand, combining original and additional financing, actual spending on subprojects (US$66.9 million) was 102 percent of the expected amount, but only 65 percent of the expected number of families benefited from the project. Efficiency is rated modest. a. If available, enter the Economic Rate of Return (ERR ERR)/Financial Rate of Return (FRR FRR) at appraisal and the re-estimated estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal ICR estimate No No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The overall objective of reducing rural poverty was substantially relevant in terms of conditions in Pernambuco, and in relation to Bank and Borrower strategy. The relevance of design was also substantial, because the project components were tried and tested, and they were appropriate for reducing poverty. Although questions about attribution to the project remain, new evidence shows that rural poverty in the state did fall and a plausible case can be made that the various elements in the results chain were consistent with poverty reduction. Two of the four sub-objectives were, on balance, substantially achieved. However, there are weaknesses in the efficiency analysis particularly concerning the representativeness of the subproject sample. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Subprojects were selected from a tried and tested menu of options, increasing the prospect that they would be sustainable. Environmental screening processes were apparently adequate. Provisions for operation and maintenance were included in the contract between the Community Association and the State Technical Unit. Based on beneficiary self-report, 60 percent of the subprojects approved in the additional financing phase had adequate provision for operations and maintenance (ICR, p. v). The largest group of subprojects was for household water cisterns: it is reasonable to assume that families will have adequate incentive to maintain these; and, moreover, the costs and technical knowhow required for maintenance of these individual systems is undemanding. Unification of the various Municipal Councils set up by different programs (see intermediate outcome (c) in Section 4 above) may have helped to counter the tendency toward creating parallel structures that often weakens community-driven development. But the evidence that Municipal Councils have leveraged funds outside the project is insubstantial, reducing the likelihood that the project s benefits will be sustained. The lack of sound technical assistance during project implementation, and the pervasive issue of scant technical support in the rural Northeast (ICR, p. 7), raises concerns about the long-term viability of some project investments, particularly for productive subprojects. There are particular doubts about the sustainability of the productive subprojects, given the drought-prone nature of this region, and the limited availability of credit. At the ICR Review meeting, the project team told IEG that it was not clear what proportion of the Community Associations had obtained credit from commercial banks, acknowledging that the implementing agency did not actively pursue linkage of Community Associations (or individual members) to banks. On the other hand, in this Pernambuco operation, productive subprojects accounted for only 6 percent of all subprojects, for which reason risk is rated "moderate"

8 rather than 'significant". a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at Entry The project's objectives were relevant to Borrower and Bank strategy at the time of appraisal and afterwards; and its design was relevant to the attainment of those objectives. The Bank developed fruitful working relations with the state government during project preparation. The provision for targeting the poorest municipalities was sound. The Management Information System inherited from the previous project was strengthened (see Section 10 below). In support of the broad thrust toward decentralization of planning capabilities, the project set up eight Regional Technical Units in strategic locations statewide. The closeness of these units to the beneficiaries increased the speed and flexibility of project procedures and facilitated subproject supervision, helping to ensure that the expected outcomes were realized. The establishment of a management fee, equal to 1 percent of the budget of each subproject, made a modest contribution to funding the work of the Municipal Councils during project implementation. Quality at entry had three shortcomings. First, there was a lack of clarity about rules for the graduation of Community Associations: it was not clear whether this meant that Community Associations had matured to the point where they could stand alone without further funding; or whether it meant simply that Community Associations would not be eligible to receive funding for more than one productive subproject (a more limited definition; one that did not, moreover, rule out the possibility of successive rounds of funding for the same Community Association as long as this was for subprojects not classified as productive). Second, there was incomplete specification of baselines and targets and the provision for measuring the project s contribution to poverty reduction was limited. Third, there was no provision made for recruiting salaried Technical Advisors during project implementation, potentially compromising subproject viability (particularly for productive subprojects). Supervision The locally-based Bank team made regular visits to randomly picked Community Associations, Municipal Councils and subprojects, helping to ensure that there was close oversight of funding. There was good follow up on financial management and procurement problems. However, the Bank could have pressed for a mid-term review during implementation of additional financing, it should have included a water and sanitation expert in the supervision team given the large volume of small-scale water infrastructure financed under the project, and it could have pushed the implementing agency harder in order to secure the follow-up survey that was needed to provide sufficient data for the impact evaluation. a. Ensuring Quality -at at-entry Entry:Moderately Satisfactory b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: Government The Borrower was the state government (the federal government acted as guarantor). The state government remained committed to project objectives and approach throughout the period of preparation and implementation, despite three changes of administration. The state government underscored its commitment by leveraging US$25 million from other programs to expand project coverage and intensify the focus on the most vulnerable groups. At appraisal concern was expressed that the state government might not honor its counterpart funding promises; but an adequate level of counterpart funding was maintained throughout implementation. Implementing Agency The State Technical Unit (STU), which was the primary implementing agency, underwent a radical restructuring

9 during project implementation, devolving responsibilities to eight Regional Technical Units scattered across the state. This process of decentralization made the project process more flexible and agile, speeding up subproject disbursements. The rotation of state governments led to some loss of continuity. To its credit, the STU took the lead in exploring new approaches to productive investments under the Fair Trade initiative and also in seeking to extend project coverage to communities (quilombos ) composed of the descendants of escaped slaves. But the implementing agency did not promote productive projects as actively as it could have done and was lukewarm in its support for the impact evaluation that the Bank had proposed. a. Government Performance :Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: Design Insufficient attention was paid to the results framework: performance indicators were poorly designed and not enough provision was made for impact evaluation. The project had a good Management Information System (MIS), which produced the data needed for sound monitoring; but evaluation was weak. The MIS was designed to give on-line access to the Regional Technical Units and the program coordinating unit in Recife and a website was set up to provide information to the public on project implementation. Data could be entered locally by Community Associations, allowing for real-time monitoring of the entire subproject cycle. The MIS for this project improved on the approach taken in previous projects: first, by registering leveraged (i.e., non-project) resources from partnerships with others programs; second, by tracking the share of the rural poor receiving grant financing for productive subprojects; and, third, by recording Community Associations obtaining commercial loans. For purposes of evaluation, the project financed several studies, including a baseline and intermediate results study. The aim was to follow up with a second field survey that would allow for project impact to be estimated. Implementation The MIS was implemented according to plan and performed satisfactorily. Evaluation proceeded less smoothly. The follow-up survey was delayed owing to staffing changes in the project management unit and the state government and slow progress with competitive contracting of the study. The follow-up survey was initially deferred until implementation of the Additional Financing but had not gone ahead by loan closing so the anticipated formal impact evaluation never materialized. The governments in the various Northeast states resisted the multi-state evaluation that the Bank envisaged and it proved impossible to agree on a common design. As a partial substitute some other studies were implemented, including reviews of the physical performance of selected subprojects and estimates of economic and financial rates of return. Use The MIS was an invaluable adjunct to project management, data collection and analysis feeding directly into decision making. Thanks to the MIS, monitoring was exemplary. But assessment of the project's contribution to poverty reduction was impeded by failure to complete the anticipated formal impact evaluation. Given the size of the overall program of which this project is a part and the length of time it has been running, this pushes the M&E rating to modest. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards The project was Category B, with an Environmental Management Plan, which remained the same under additional financing. All subprojects were vetted for compliance with the Operational Manual, which included the Bank's Environment and Social Safeguards. Subprojects were also assessed against state and federal environmental laws.

10 The staff of the implementing agency included an environmental specialist. No safeguard violations were reported. Financial Management With one exception, financial management supervision missions rated the project "satisfactory": in June 2003, Brazil-wide fiscal constraints affecting most of the Bank's portfolio, led to a downgrade in the financial management rating to "unsatisfactory." The final financial management supervision (April 2010) concluded that the State Technical Unit's fiduciary performance was "satisfactory". Unintended Impacts No unintended impacts are reported in the ICR. 12. Ratings: ICR IEG Review Outcome: Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome: Reason for Disagreement /Comments Bank Performance : Moderately Satisfactory Borrower Performance : Moderately Satisfactory Quality of ICR : Moderately Satisfactory Moderately Satisfactory Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Lessons suggested by the ICR include the following: Community-driven development (CDD) can promote social capital formation if subprojects are rigorously and transparently selected and adequate training is given: "Tangible benefits from subproject investments strengthen collective capacity, generate a sense of citizenship, and raise the bar for local government accountability" (p. 28); Delegating responsibility for routine supervision to project monitoring unit regional offices (of which there were 8 in this project) helps increase project visibility at the local level, making the project more accountable to beneficiary families; and Productive subprojects are more likely to succeed if they are supported by business plans, allow for training and assess potential demand for the product. They are more likely to be sustainable if provision is made for linkage to banks. IEG adds: A complete assessment of the efficiency of CDD projects involves reviewing the evidence for cost-effectiveness, showing how this varies between subproject categories, and the extent of variation between the average for the project and the average obtained under other public programs. Intermediate outcomes, such as social capital, need to be properly quantified with well-designed indicators, baselines and targets. 14. Assessment Recommended? Yes No

11 15. Comments on Quality of ICR: The report is well written and comprehensive, distinguishing clearly between what was achieved under the original project and under additional financing. There is a thorough assessment of project design. A number of specific and informative lessons are proposed. Given the project objective of reducing high levels of rural poverty in the state, there is limited evidence presented on poverty outcome (perhaps reflecting the weak M&E). The economic analysis should have included data on subproject unit costs relative to other programs and the efficiency analysis should have been based on a larger number of subprojects and not excluded those that were unsuccessful. a.quality of ICR Rating : Satisfactory