FEBRUARY IS (MAIN REPORT)

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized QUALITY ASSURANCE GROUP FEBRUARY IS9 28 (MAIN REPORT) 46145

2

3 GiaffEVIATlQNS AND ACRONYMS AAA ACS ADB AFR AfDB APL ARPP ARDE BB BW C-AAA CAE CAS CASCR CEM CFAA CMU CODE CPAR CPIA CPPR CSR DGF DO DPL DPO DPR DRL EAP ECA ECO EMT ENV EP ERL ESW EXT FEU FIL FPD FSE Analytic and Advisory Activities Activity Completion Summary Asian Development Bank Africa Region African Development Bank Adaptable Program Loan Annual Report on Portfolio Performance Annual Report on Development Effectiveness Bank Budget Business Warehouse Country Analytic and Advisory Activities Country Assistance Evaluation Country Assistance Strategy CAS Completion Report Country Economic Memorandum Country Financial Accountability Assessment Country Management Unit Committee on Development Effectiveness Country Procurement Assessment Review Country Policy and Institutional Assessment Country Portfolio Performance Review Controller s, Strategy and Resource Management Development Grant Facility Development Objectives Development Policy Lending Development Policy Operations Development Policy Review Debt Service Reduction Loan East Asia and Pacific Region Europe and Central Asia Region Expanded Co-Financing Operation Energy and Mining Environment Economic Policy Emergency Recovery Loan Economic and Sector Work External Affairs Department Finance, Economics and Urban Department Financial Intermediary Loan Finance and Private Sector Development++ Finance Network++ FY GEF GPP GRAAA HD HDN HNP HR IBRD ICR IDA IDB IEG IFA IG IP ISR INF INV LICUS LCR LIL M&E MDGs MIC MNA MP MUV NLTA oc ODA OPCS PA PAD PDO PER PN PRC PPAR PREM PSDN Fiscal Year Global Environment Facility Global Programs and Partnership Global and Regional Analytic and Advisory Activities Human Development Human Development Network Health, Nutrition and Population Human Resources International Bank for Reconstruction and Development Implementation Completion Report International Development Association Inter-American Development Bank Independent Evaluation Group (formerly OED) Integrated Fiduciary Assessment Investment Grade Implementation Progress Implementation Status and Results Report Infrastructure Investment Operations Low-Income Countries Under Stress Latin America and the Caribbean Region Lending and Innovation Loans Monitoring and Evaluation Millennium Development Goals Middle-Income Countries Middle East and North Africa Region Montreal Protocol Manufacturers Unit Value Non-Lending Technical Assistance Operations Committee Official Development Assistance Operations Policy and Country Services Poverty Assessment Project Appraisal Document Project Development Objective Public Expenditure Review Policy Note Poverty Reduction Support Credit Project Performance Assessment Report Poverty Reduction and Economic Management Network Private Sector Development Network++ ++ In FY8, FSE and PSDN networks merged into FPD.

4 PSG PSL QAG QEA QER QSA RDV RETF RIL ROC SAL SAP SAR SDN Public Sector Governance Programmatic Structural Adjustment Loan Quality Assurance Group Quality-at-Entry Assessment Quality Enhancement Review Quality of Supervision Assessment Rural Development Recipient Executed Trust Fund Rehabilitation Loan Regional Operations Committee Sector Adjustment Loan Systems Application and Products in Data Processing South Asia Region Sustainable Development Network SDV SIL SIM SF SMU SND SP SSL SWAP TA TAL TF TTL URB Social Development Specific Investment Loan Sector Investment & Maintenance Loan Special Financing Sector Management Unit Sub-National Department Social Protection Special Structural Adjustment Lending Sector Wide Approach Technical Assistance Technical Assistance Loan Trust Fund Task Team Leader Urban

5 ANNUAL REPORT ON PORTFOLIO PERFBRIWIGNCE FISCAL YEAR 27 CONTENTS EXECUTIVE SUMMARY... i. I I1. I11. Iv. V. VI. Introduction... 1 Approvals and Portfolio Size and Composition... 2 A. Aggregate Trends... 3 B. Focus of Lending... 9 C. IDA and Fragile States D. IBRD E. Innovations in Lending F. Infrastructure G. Recommendations Portfolio Performance A. Development Outcomes of Projects B. Portfolio Perfomance and Measurement C. Recommendations Analytic and Advisory Activities A. Trends in Expenditures and Deliveries B. Quality of AAA C. Recommendations Development Outcomes of Country Programs A. Trends in Program and Project Outcomes B. Factors Contributing to Low Program Outcomes... 5 C. Divergence between Program and Project Outcomes D. Recommendations Recommendations BOX 2.1: The Portfolio Dynamics... 4 FIGURES Figure 2.1: Figure 2.2: Figure 2.3: Figure 2.4: Figure 2.5: Figure 2.6: Figure 2.7: Figure 3.1 : Trends in Approvals. Disbursements. and Portfolio (FY-7)... 3 Approval Trends in Multilateral Development Banks (2-7)... 4 Portfolio Concentration (FY2 vs. FY7)... 5 Preparation Time and Cost per Investment Lending Operation... 7 Preparation Time and Cost per Development Policy Operation... 8 Approvals for Fragile States (FY3-7) IBRD Approvals by Client Segment (3-Year Moving Average in Real Terms) Proportion of Operations with Satisfactory Outcomes... 18

6 Figure 3.2: Figure 3.3: Figure 3.4: Figure 3.5: Figure 3.6: Figure 4.1 : Figure 4.2: Figure 4.3: Figure 4.4: Figure 4.5: Figure 5.1: Figure 5.2: Figure 5.3: TABLES Table 2.1 : Table 2.2: Table 2.3: Table 3.1: Table 3.2: Table 3.3: Table 3.4: Table 3.5: Table 4.1 : Table 4.2: Table 4.3: Table 4.4: Table 4.5: Table 5.1 : Table 6.1 : Table 6.2: ANNEXES Annex 1 Annex 2 Project Outcomes by Region Major Improvers by Sector Board... 2 Major Decliners by Sector Board Proportion of Operations with Satisfactory Outcomes (IBRD vs. IDA) QEA8 Results by Quality Dimension Share of Country Services Expenditures (FY2 vs. FY7) Unit Cost in Real Terms of Dropped AAA Tasks (FY3-7) Preparation Time of AAA Deliveries (FY3-7) Quality of ESW Product Types Total Cost in Real Terms of Dropped AAA Tasks (FY3-7) Performance of Country Programs and Related Projects Factors Contributing to Unsatisfactory CAEs... 5 Sector and Aggregate Project Ratings (FYO1-7 CAEs) Gross and Net Disbursements (FY5-7)... 6 Approvals by Theme (FY-7)... 1 Approvals by Sector (FYOO-7) IEG Evaluations for Africa and Other Regions (FY4-7) Results of QAG Assessments for HNP FY4-7 IBRD and IDA Performance Gap by Sector Board FY7 Risk Ratings. QAG Review vs. ISR Reported Problem Projects vs. Actual Unsatisfactory Projects AAA Programs by Number and Cost (FY3-7) Core Diagnostic Reports Coverage (FY4 vs. FY7) Number and Unit Cost of ESW Deliveries by Output Type Quality of Country Programs Status of Recommendations from AAA Assessments Country Program vs. Project Portfolio Performance Status of FY2-6 ARPP Recommendations Consolidated ARPP Recommendations The Portfolio. An Overview Table Basic Portfolio Definitions and Data Sources... 6 STATISTICAL APPENDIX ACKNOWLEDGMENTS This report was prepared by Albert Martinez with inputs from Amnon Golan. Marc Blanc. Inder Sud. Jose Sokol. Vinay Bhargava. Roger Grawe. h a Hartmann. Fred Swartzendruber. Eugene McCarthy. Peter Ludwig. Brad Herbert. Irfan Aleem. Rema Balasundaram. and Melvin Vaz. Helpful advice was received from the Advisory Panel composed of Alain Barbu. Gisu Mohadjer. Sally Zeijlon. Richard Scobey. Raja Iyer. Hoveida Nobakht. Joel Maweni. Brigitte Duces and Tawhid Nawaz. Administrative support was provided by Conchita Castillo. Josephine Onwuemene. and Amelia Laya. Prem Garg. Director. and Xavier Legrain. Acting Director. Quality Assurance Group. guided the overall effort.

7 Annual ReDort on Portfolio Performance FY7 i EXECUTIVE SUMMARY 1. The Annual Report on Portfolio Performance (ARPP) provides the Board and Management with a strategic overview of the size, composition, and quality of the Bank s lending portfolio and Analytic and Advisory Activities (AAA) program. It also provides Senior Management with real time information to assess what is working well, or less well, together with recommendations on measures to sustain and improve quality and effectiveness of the portfolio and of the AAA program - two key vehicles for delivering results to clients. TRENDS IN APPROVALS AND PORTFOLIO 2. The Bank s portfolio at end-fy7 consisted of 1,485 operations with net commitments of $1.4 billion representing a decline in real terms compared to five years ago. Investment operations accounted for 95 percent of the portfolio since most Development Policy Operations (DPOs) close within one year of approval. There was a decrease in portfolio concentration; the ten largest borrowers accounted for half the portfolio in terms of commitments in FY7 compared to about 6 percent five years ago. 3. There are five trends and characteristics of Bank lending operations and portfolio worth noting. First, Bank efforts have maintained IBRD lending levels in real terms during the past five years despite a contraction in demand for sovereign external financing. Within the IBRD client group, Blend countries experienced increases in approvals. However, lending in real terms to core IBRD countries, which accounted for half of IBRD lending, has been stagnant with underutilized lending envelopes. It is recommended that Management explore opportunities to increase financial and non-financial assistance and services to the core IBRD countries. 4. Second, resource transfers to Fragile States increased significantly in FY7, the highest level since FY2. Fragile States accounted for about a fifth of IDA FY7 approvals, with post conflict countries receiving about a third of resource transfers to Fragile States. In addition, about 4 percent of Recipient Executed Trust Fund project approvals in FY7 went to Fragile States. However, only 61 percent of Fragile States projects exiting the portfolio during FY4-7 were rated by IEG as satisfactory. In addition, QSA7 found weaknesses in project design and readiness for implementation in Fragile States projects included in the sample. 5. Third, the infrastructure sectors have been the main source of growth in lending during the past five years. This reflects the large financing needs in infrastructure and the importance of infrastructure in improving access to services especially by the poor. The growth in lending is a successful response to the 23 Infastructure Action Plan which sought to reverse the decline at that time in the Bank s infrastructure lending. Almost half of the Bank s current portfolio is composed of infrastructure projects, with Transport accounting for half of the infrastructure portfolio. The challenges facing infkastructure lending include managing the interface with other sectors and responding to cross-border issues. The recent merger of the About half of FY7 exits have been evaluated by IEG.

8 Annual Renort on Portfolio Performance FY7 11 Infrastructure and Environmentally and Socially Sustainable (ESSD) networks is part of the response to these challenges. 6. Fourth, though simple and repeater operations have increased, Bank operations include many new approaches and innovations. The Bank s work in Middle Income Countries has been responsive to demand fiom sophisticated clients for new products and services as well as increased efficiency of delivery. Multi-country and regional programs and projects are increasing in recognition of the need for collective action to address cross-border issues and improve synergies of individual country programs. The Sub-National Development Program was created in 26 to address the lack of Bank instruments to support an important and growing client segment. There has been rapid growth of Additional Financing operations in investment lending and a major shift to programmatic approaches in Development Policy Operations. Carbon Finance operations have been growing, with Operations Policy and Country Services (OPCS) issuing guidelines in October 27 to improve design and risk assessment and mitigation. The quality assurance processes will have to keep pace with the introduction of new products and approaches. 7. Fifth, aggregate trends in lending during the past five years reflect how the Regions implemented the Bank strategy introduced in FYO1. Thematically, about a quarter of lending went to private sector development, a quarter went to human and social development, a fifth to public sector governance, and the remainder roughly equally divided among urban and rural development and environment. There has been a stronger emphasis in IDA countries on human and social development to support poverty reduction and Millennium Development Goals (MDG) implementation, while IBRD operations focused more on economic growth. It is recommended that Management review whether there should be a shift in focus of IBRD and IDA lending towards more emphasis on addressing inequality and environment in IBRD countries and more lending in sectors such as infrastructure in IDA countries as argued by some Bank papers. 8. As the Bank moves towards new strategic directions, there will be a shift in emphasis of existing programs and introduction of new approaches. Perhaps the major challenge will be operationalizing the global public goods initiative, though much can be learned fi-om the current work on multi-country approaches. Global Programs and Partnerships (GPPs) are important instruments for supporting global initiatives - an FY7 QAG review found improvement in quality-at-entry and oversight of Window Two programs compared to the FY6 assessment results, though Regions indicated room for greater integration of GPPs in Regional Strategies. Future ARPPs could review progress in lending, AAA, and other Bank programs towards the new strategic directions. BANK PERFORMANCE 9. Development outcomes of operations exiting the Bank portfolio continued to improve through FY6, though IEG evaluation of half of FY7 exits shows lower outcomes Strategic Framework Paper, January 21.

9 ... Annual ReDort on Portfolio Performance FY7 111 in terms of number of operations. On the basis of a three year moving average, about 8 percent by number of projects, and 87 percent by disbursement, of FY6 exits were rated by IEG as satisfactory. This represents sustained improvement in performance since FY9 when one of three projects was unsatisfactory. The aggregate project performance meets the 8 percent goal set by Management a decade ago. However, Management will have to continue to monitor recent performance. 1. There was deterioration in aggregate outcomes of AFR projects from 83 percent in FY4 to 65 percent in FY6. The FY7 performance thus far is slightly lower than that of FY6 based on evaluation of about half the FY7 exits. Within IDA, AFR performance is lower than the rest of the Bank. The performance of Fragile States projects in AFR is significantly worse than that in other Regions. However, QEA8 rated FY6-7 AFR approvals as satisfactory. The next QSA should include an assessment of the quality of older projects in the AFR portfolio to determine the trajectory of outcomes during the next few years. 11. Among Sector Boards, there were declines in project performance in Health, Nutrition and Population, Private Sector Development, and Public Sector Governance during FYO4-7 compared to FY1-3. A review of the IEG evaluations revealed that project design flaws were the main factors contributing to unsatisfactory outcomes in these Sector Boards. In a significant number of cases, the projects were approved with lack of government commitment. In many cases, the design flaws were rooted in unrealistic assumptions of absorptive capacity of the Borrower, and project supervision was not able to successfully address weaknesses in quality-at-entry. The Regions and relevant Sector Boards should review the current portfolio to correct design flaws in existing projects and ensure more rigorous quality assurance of new projects, focusing on government commitment and capacity. The next QSA should also assess the portfolio of the above Sector Boards and recommend appropriate courses of action. 12. The results of QEA8 indicate a greater than 9 percent satisfactory quality-at-entry of projects approved in FYO6-7. There is also a convergence of performance by Regions compared to previous years. Historically, QEA ratings are about 1 percentage points higher than actual project outcomes. Based on QEA results since FYOO, as well as satisfactory QSA outcomes, Bank performance should continue to be at above 8 percent satisfactory. 13. With respect to supervision, QAG reviews had some findings of significance. First, the Proactivity Index indicates that teams have been addressing in a satisfactory manner the underlying issues of problem projects. A QAG review found that the Index understates actual proactivity, but recommended no changes at this time to the measurement after assessing the cost-benefit of such a move. Second, the number of projects with delayed effectiveness has been increasing mainly due to lengthy client approval processes and the number of effectiveness conditions. Several Regions are now addressing the factors contributing to delayed effectiveness. Third, QSA7 found no significant difference in quality of supervision of HQ vs. field based TTLs - both showed satisfactory efforts overall. However, more could be done to complement local presence with appropriate use of HQ staff with global experience.

10 Annual Report on Portfolio Performance FY7 iv 14. Notwithstanding the satisfactory overall results from QAG assessments, there are several areas with scope for improvement. Moving to a higher level of performance in quality-at-entry requires the strengthening of three aspects: (a) more realistic results framework and objectives; (b) better risk assessment and mitigation; and (c) improved implementation readiness. Further improvements in the quality of supervision would require: (a) more effective management attention; (b) timely identification and action on threats to development objectives; and (c) stronger skills mix. Various QAG assessments have identified effectiveness of managerial oversight as a critical factor that differentiated fully or highly satisfactory from moderately satisfactory project performance. Some Regions have begun to address the issue of overextended sector managers. 15. Measuring and tracking portfolio performance have been ineffective due to the lack of candor in project reporting. The Realism Index continues to show significant underreporting of project risks, a problem that has been recognized for some time but has not been effectively addressed. A recent QAG review of quality of reporting covering the FY7 portfolio estimated the share of problem projects in the portfolio to be twice that reported in ISRs. The finding is the same as that of QSA7, which covered the FY6 portfolio. The realism problem is acute in IDA countries, in particular in Fragile States. The usefulness of the current system of portfolio reporting as a management tool is dependent on improving candor in reporting project performance and risks. OPCS will monitor the impact of the new Realism Index to see whether any further action on portfolio reporting was necessary. 16. One way forward towards improving portfolio performance measurement and reporting is to establish a stronger link between risks identified at entry and the reporting of project performance. The current system rates projects entering the portfolio at zero risk, and flags project risk only when problems begin to surface. The revision of the measurement and monitoring of project risk during implementation should build on the reforms initiated in July 27 to ensure a more systematic and comprehensive assessment of risks at appraisal. It is recommended that OPCS and QAG work towards reviewing and recommending appropriate changes to the current system of portfolio risk measurement and reporting. ANALYTIC AND ADVISORY ACTIVITIES 17. The importance of M A as an instrument of Bank support has been growing. This is reflected in the increasing share of AAA expenditures in the country services envelope - AAA expenditures in FY7 were higher than that of lending preparation and almost as high as that of supervision. The use of AAA - as measured by its share in country services expenditures - has grown in Investment Grade IBRD countries, Blend countries, and Fragile States. This growth is consistent with the various strategies on improving Bank relevance in IBRD countries and the recommendations from the 22 LICUS Task Force Report. The increased use of Trust Funds was a major factor contributing to AAA growth. 18. Several trends in the size and composition of AAA are worth noting. First, there has been a major shift in AAA towards Non-Lending Technical Assistance (NLTA) both in terms of number as well as value of tasks. In part, this is due to an increasing number of large NLTA

11 Annual Reuort on Portfolio Performance FY7 V activities costing more than $.5 million each in the Africa Region and the Social Development Sector Board with a large part of the financing coming from Trust Funds. Second, within ESW, there is a shift from core diagnostic reports to other types of reports. These developments reflect decisions at the country level on the appropriate mix of AAA instruments as envisioned in the AAA reforms in 23. Third, multi-country AAA is a fast growing segment, mainly in AFR where there have also been an increasing number of multi-country lending operations. Fourth, while post delivery expenditures have been increasing, the Country AAA (C-AAA) assessment found that Dialogue and Dissemination continues to be a weak aspect o f AAA. Finally, unit cost of ESW tasks increased by 45 percent in real terms during the past five years, mainly in Country Economic Memorandums and Country Advisory Reports. 19. QAG also conducted two assessments of AAA programs during lwo3-7 covering a total of 53 countries. The combined findings from the assessments show a mixed picture. On the positive side, the overall quality of AAA programs was rated as satisfactory with strong aspects in Internal Quality as well as Scope and Strategic Relevance. But several weaknesses were identified; the low rating for Likely Impact is a major concern-the Bank may be producing good quality tasks which, when viewed in the overall strategic country context, may not be addressing the most critical developmental constraints. Coherence and Integration of country AAA programs was the weakest aspect and with Dialogue and Dissemination contributed to lower ratings for Likely Impact. These are in turn explained by inadequate management attention to quality. 2. In response to the increase in the costs of dropped AAA tasks highlighted in last year s ARPP, QAG carried out a review of dropped AAA tasks as a basis for identifying systemic recommendations. Two key findings emerged from the review. First, Value for Money - effective use of resources in relation to benefits - was found to be good overall. Indeed, a large majority of the tasks in the sample were judged to have produced satisfactory results. Second, Quality of Process was found to be poor, often reflecting inadequate institutional processes that hamper monitoring and tracking of AAA. About a third of the tasks assessed by QAG were misclassified. The process problems were particularly acute for AAA tasks that are programmatic, multi-sectoral or global/regional in nature. COUNTRY PROGRAM PERFORMANCE 21. There is a disconnect between aggregate performance of lending and AAA projects and the performance of country programs, which are rated below 6 percent satisfactory based on completed Country Assistance Evaluations (CAEs) during the past ten years. A QAG review of CAEs with unsatisfactory ratings found that two factors stand out as primary contributors to low outcomes: (a) weak design of country programs; and (b) ineffective country dialogue. Country program design issues included significant omissions, lack of coherence, and failure to take into account borrower commitment and capacity. Individual project outcomes could be satisfactory but poor program design and ineffective country dialogue would cause the country program to be rated unsatisfactory.

12 Annual ReDort on Portfolio Performance FY7 vi 22. The 26 Annual Report on Development Effectiveness (ARDE) found major deviations between sector and project outcomes mainly in Private Sector Development, Public Sector Development, and Rural Development - the deviations are important because CAS objectives are articulated in terms of sector or thematic outcomes. The QAG review identified three main reasons for the divergence between sector and project outcomes. First, many projects were undertaken which did not address the key development constraints in the sector or had questionable priorities. Second, poor coordination led to low outcomes in sectors where cross sector inputs from other Bank projects and donors were critical. Finally, the results frameworks of both projects and CASs did not have strong linkages among project, sector, and CAS outcomes. 23. The above findings point towards a two track approach to improving program outcomes. First, the CAS design, i.e., quality-at-entry, should be improved by ensuring that the country program addresses key development constraints, is focused and coherent, and takes into account Borrower commitment and capacity. The introduction of the results based CASs beginning 25 should help address the weaknesses in CAS design through the introduction of a more rigorous results framework. A planned retrospective on the experience with results based CASs would assess progress to date towards improving CAS design. Second, quality assurance should focus on quality of implementation of CASs not only at the task level but also at the program level to ensure that the timing and choice of design of interventions as well as their implementation maximize impact on CAS outcomes. It is recommended that OPCS review as part of the CAS Retrospective the adequacy and effectiveness of current instruments in ensuring quality of CAS implementation. RECOMMENDATIONS 24. A review of recommendations in FYO2-6 ARPPs found that a majority of the recommendations have been completed, but there was less than satisfactory progress in three areas: (a) actions to address structural and incentive issues that constrain Sector Managers from effectively performing their quality assurance responsibilities; (b) actions to address staff and managerial incentives to improve realism in portfolio reporting; and (c) actions to improve management of AAA. The status of these recommendations is summarized in Table 1 below. The recommendations which consolidate the findings from the FY7 ARPP and the uncompleted actions from previous ARPPs are shown in Table 2.

13 Annual ReDort on Portfolio Performance FY7 vii Recommendations Strengthening Lend Business Processes and Portfolio Reporting: There were 12 recommendations on improving processes and portfolio measurement and reporting. These included introducing several guidelines, reforming the PSR system, revising the Realism Index, and strengthening the review and quality assurance processes. Staff and Management Incentives and Capacities: There were 4 recommendations: address impediments to Sector Management in ensuring quality; address issues in staff and management incentives to improve candor in portfolio reporting; upgrading training; and improving support to management. Other Quality Issues: There were 1 recommendations relating to other quality issues including those targeting certain groups such as Sector Boards and certain aspects such as adequacy of resources. Status P and Portfolio Management Most of the recommendations have been implemented. One recommendation - integrating RETF in portfolio - is currently under implementation. Rating: Satisfactory Some Regions have begun to address the issue of impediments to sector management in ensuring quality. However, there is lack of systemic response to the issue of candor in project performance reporting, with the Realism Index continuing to be below target. Rating: Moderately Unsatisfactory Most of the recommendations have been implemented with satisfactory results. There are lagging areas, such as outcomes of HNF' and Fragile States projects. Rating: Moderately Satisfactory There were 1 recommendations on improving AAA management. These include improving reporting, addressing delays in delivery, improving dialogue and dissemination and improving management oversight. Half of the recommendations have been implemented. Several recommendations dealing with management oversight and dialogue and dissemination have not yet resulted in satisfactory outcomes based on the latest C- AAA Assessment. Rating: Moderately Unsatisfactory There were 8 recommendations on the results framework. The follow-up to these recommendations will be reported in the upcoming OPCS Results Paper. Rating: Not Rated Overall Rating for Follow-up to Previous ARPP Recommendations: Moderately Satisfactory

14 Annual ReDort on Portfolio Performance FY7... Vlll Table 2: Consolidated ARPP Recommendations Recommendation AFR to sustain efforts to improve project outcomes, focusing on older projects and projects in Fragile States, HDN, PREM, and ESSD. Regions to work with HNP, PSD, and PSG Sector Boards to address relatively low project outcomes focusing on improving project design at entry and restructuring problem projects in current portfolio. Intermediate Actions Lending and Portfolio Quality AFR with relevant Sector Boards to review current portfolio and address threats to satisfactory outcomes. Next QSA to report on progress in AFR with respect to targeted portfolio segments. Sector Boards and Regions to agree on role of Sector Boards in the quality assurance process at entry and to review current portfolio and address threats to satisfactory outcomes. Next QSA and QEA to report on performance of HNF', PSD and PSG Sector Board projects. Results Narrowing of performance gaps between AFR and rest of the Bank with respect to outcomes of projects in Fragile States, HDN, PREM, and ESSD. Specific targets and timetable to be determined by Management based on outcomes of reviews and QSA. Improved outcomes for projects in the HNF', PSD, and PSG Sector Boards towards convergence with Bank-wide performance. Targets to be determined by Management based on outcomes of reviews. OPCS and QAG to reform measurement and reporting of project risks by linking risk assessments at entry to risk management during implementation. OPCS to review Regional efforts to address impediments to managerial effectiveness in quality enhancement and lack of candor in reporting project performance. Portfolio Measurement and Monitoring I OPCS and QAG to review and recommend changes to the current system of project risk measurement and monitoring. OPCS will monitor the impact of the new Realism Index to see whether any further action on portfolio monitoring was necessary. More accurate reporting of portfolio risk starting FY9 to be tracked by QAG. Attainment of 8 percent Realism Index by FY9. OPCS to restructure AAA definitions, guidelines and systems to improve results orientation, results measurement, and governance arrangements. These initiatives would help address recurring issues of lack of adequate management oversight and weak dialogue and dissemination. AAA OPCS to review AAA definitions, guidelines and systems to improve monitoring, measurement, results orientation and governance arrangements to address recurring issues of lack of management oversight. Next QAG AAA Assessment to determine whether progress has been made in addressing areas for improvement.

15 Annual Reuort on Portfolio Performance FY7 ix Recommendation Intermediate Actions Results Regions to improve country program design and ensure that results fkameworks of projects and AAA are strongly linked to sector and CAS outcomes. Country Programs OPCS to complete the CAS Retrospective. OPCS to review as part of the CAS Retrospective the adequacy of current instruments that track implementation of CAS programs. OPCS to define, as part of the CAS Retrospective, the criteria for quality of MIC programs. Results will be measured by improved IEG ratings of country programs. OPCS to define indicators and instruments for tracking effectiveness of implementation of CAS programs.

16

17 Annual Revort on Portfolio Performance FY7 1 OBJECTIVES AND APPROACH I. INTRODUCTION 1.1 The Annual Report on Portfolio Performance provides the Board and Senior Management with a strategic overview of the size, composition and quality of the Bank s portfolio and the Analytic and Advisory Activities (AAA) pr~gram.~ It also provides Senior Management with real time information to assess what is working well, or less well, together with recommendations on measures to sustain or improve the quality and effectiveness of the lending portfolio and of the AAA program--two key vehicles for delivering results to our clients. 1.2 The FY7 ARPP draws on materials that are prepared as part of regular portfolio monitoring fimctions carried out by the Regions and Networks, supplemented by projectlportfolio data in the Bank s management information systems. It also draws on various QAG assessments. In preparing the ARPP, extensive consultations were held with managers and staff from around the Bank. STRUCTURE AND COVERAGE 1.3 The report is organized into six Chapters. Chapter 2 reviews the recent trends in size and composition of the lending portfolio. It analyzes trends by Regions, Networks, Client Segments, Instrument, Sectors, and Themes. Chapter 3 assesses overall portfolio performance results as well as issues associated with measuring and reporting the risks of the portfolio of lending operations not achieving their development objectives. Chapter 4 takes stock of the Analytic and Advisory Activities. It focuses particularly on trends in the program size, deliveries, and quality of AAA, drawing on selected recent QAG assessments. Chapter 5 reviews the outcomes of country programs based on IEG Country Assistance Evaluations. It focuses on the factors contributing to the relatively low program outcomes compared to project results. Chapter 6 examines progress in implementing recommendations of the FY2-5 ARPPs, and presents a consolidated set of recommendations covering the findings from this year s ARPP as well as uncompleted recommendations from previous ARPPs. The Statistical Appendix contains a detailed set of supporting statistical material. As agreed with CODE, and in order to avoid duplication, this ARPP does not address directly the Results agenda, which is to be the subject of a separate report by OPCS. AAA product lines discussed in this report are ESW and TA. ESW and TA include fee-based and reimbursable tasks. Other AAA product lines not covered here include Donor and Aid Coordination, Research Services, World Development Report and Impact Evaluation.

18 Annual Reuort on Portfolio Performance FY APPROVALS AND PORTFOLIO SIZE AND COMPOSITION 2.1 This chapter focuses on trends in lending and changes in the portfolio. In analyzing longer term trends, this chapter uses FYOO as the starting point due to the relatively high lending levels in the late 199s in response to the crises during that period. Due to significant year to year volatility, lending is analyzed by aggregating approvals during FY5-7 and FY2-4 and where appropriate, FY-1. Section A describes aggregate trends and issues, while the succeeding Sections discuss specific topics. The last Section summarizes the recommendations from this Chapter. 2.2 There are five main messages from this Chapter: Aggregate trends in Bank operations during the past five years reflect implementation of the 21 Strategic Framework Paper. There has been a stronger emphasis in IDA countries on human and social development to support poverty reduction and MDG implementation, while IBRD operations focused more on economic growth. Some Bank papers4 argue for a shift in sectoral emphasis in IDA lending and a strengthening of IBRD engagement in the areas of inequality, governance, and environment. Concerted Bank efforts resulted in a slight increase in IBRD lending in real terms during FY5-7 compared to FY2-4 despite a contraction in demand for sovereign external financing. Within the IBRD client group, Investment Grade and Blend countries experienced increases in approvals. However, lending to core IBRD countrie~,~ which accounted for half of IBRD lending during FY2-7, declined in real terms with underutilized lending envelopes. Approvals of projects for Fragile States increased significantly in FY7, the highest level since FY2. In addition, about 4 percent of Recipient Executed Trust Funds (RETF) in FY7 went to Fragile States. However, only 65 percent of Fragile States projects exiting the portfolio during FY4-7 were rated by IEG as satisfactory. In addition, QSA7 found weaknesses in project design and readiness for implementation in Fragile States projects included in the sample. Though simple and repeater operations have increased, Bank operations include many new approaches and innovations. The Bank s work in Middle Income Countries has been responsive to demand from sophisticated clients for new products and services as well as increased efficiency of delivery. The number of multi-country and regional programs and projects is increasing in recognition of the need for collective action to address cross-border issues and improve synergies of individual country programs. The Demand for IDAIS Resources and the Strategy for Their Effective Use, FRM (27) and Development Results in Middle-Income Countries: An Evaluation of World Bank s Support, IEG (27). Defined as IBRD clients that are neither Investment Grade nor Blend countries.

19 Annual ReDort on Portfolio Performance FY7 3 The Sub-National Development Program was created in 26 to address the lack of Bank instruments to support an important and growing client segment. The infrastructure sectors have been the main source of growth in lending during the past five years. The growth in lending is a successful response to the 23 Infrastructure Action Plan which sought to reverse the decline at that time in the Bank s infrastructure lending. Almost half of the Bank s portfolio at end-fy7 is composed of infrastructure projects, with Transport accounting for half of the infrastructure portfolio. Approvals A. AGGREGATE TRENDS 2.3 Bank lending increased by four percent in FY7 driven by strong IDA performance, notably in AFR and SAR. IBRD approvals declined by ten percent in FY7, though this was mainly due to the moving forward of many operations in LCR to FY6 in anticipation of elections in large borrower countries such as Mexico and Brazil. From a longer-term perspective, Bank lending approvals in real terms have been increasing over the past five years (see Figure 2.1) despite a constrained lending environment in the Middle Income Countries (MICs). The Bank-wide lending trend since FYOO also compares favorably with those of other Multilateral Development Banks (see Figure 2.2). The medium-term prospects for lending indicate that Bank lending will continue to be at about the same level as during FY5-7, with annual project approvals projected at the $22 to 26 billion range during FY Figure 2.1: Trends in Approvals, Disbursements, and Portfolio (WOO-7) (In Real Terms) - 6 I FYOO FYOI FY2 FY3 FY4 FY5 FY6 FY7 [ +Approvals in FY +Disbursements in FY t Ne -@- Commitments at End FY I

20 Annual ReDort on Portfolio Performance FY7 4 Figure 2.2: Approval Trends in Multilateral Development Banks (2-7) (In Real Dollar Terms) h % EI C [ +AfDB +ADB +IDB +WB 1 Sources: Annual Reports of AfDB, ADB, and IDB; Bank BW. WB approvals are by fiscal year while others are by calendar year. Box 2.1: THE PORTFOLIO DYNAMICS The portfolio as defined in the ARPP is a stock concept. Lending by contrast is a flow. The Bank portfolio consists of the IBRD loans, IDA credits and grants, GEF grants, Montreal Protocol, and Special Financing operations (financed in part out of the Bank s net income). It only includes operations that are active at the end of the fiscal year. It excludes operations which are closed or fully disbursed during the year. It is recorded as the sum of individual operations commitments, net of cancellations, if any. The chart below illustrates those relationships for FY7 based on the Business Warehouse (BW) data. Opening Balance $95.2B Closing Balance* (1,468 $1.4B Operations) of which $15.4B (1,485 Exits during Operations) FY7 >6 (266-$18.8B Operations) $.3B etroactive Extension * Closing balance of FY7 includes approximately $.3 billion in projects closed in previous years, which were reopened in FY7.

21 Annual ReDort on Portfolio Performance FY7 5 Portfolio 2.4 The Bank portfolio mainly captures investment lending. As of end-fy7, the Bank portfolio consisted of 1,485 operations with net commitments of $1.4 billiony6 about 8 percent of end-fy2 portfolio in real terms. Investment lending in terms of commitments accounted for about 95 percent of the portfolio and 75 percent of approvals in FY7. Development Policy Operations (DPOs) are not commensurately represented in the portfolio because these are mainly quick disbursing loans/credits with an average project life of less than one year. The growth in quick disbursing DPOs is the major factor explaining the decline in the portfolio. 2.5 Theportfolio is less concentrated compared tofive years ago (see Figure 2.3). The end- FY7 portfolio included operations in 133 countries, with the ten borrowers accounting for about half the portfolio, compared to about 6 percent five years ago. Nigeria and Pakistan replaced the Russian Federation and Bangladesh in the top ten countries in the Bank s portfolio in FY7. About 8 percent of the smallest borrowers accounted for 1 percent of the portfolio in FY7, roughly the same proportion as in FY2. 1, Figure 2.3: Portfolio Concentration (FY2 vs. FY7) No. of Largest Borrowers FY2 E FY7 Disbursements 2.6 Gross disbursements fell slightly in FY7 mainly due to a decline in approvals of quick disbursing DPOs. However, FY7 disbursements ffom investment loans were at the same level as in FY6 and higher than in FY4-5. Disbursement ratios for investment loans have been improving over the past five years. 2.7 Net disbursements declined from $5. billion in FYO6 to $.5 billion in FY7 due to large prepayments in LCR (see Table 2.1). In FY7, LCR, EAP and ECA had negative net disbursements; however, before prepayments, only EAP had negative net disbursements. The main net resource transfers were occurring in AFR and SAR. For IBRD, gross disbursements The Bank Portfolio is composed of IBRD loans, IDA credits and grants, and grant finds fiom the Global Environment Facility (GEF), Montreal Protocol (MP), and Special Financing (SF). A description of portfolio dynamics is provided in Box 2.1.

22 Annual ReDort on Portfolio Performance FY7 6 were almost equal to scheduled repayments in FY6 and FY7; about $2. billion in prepayments in FY6 and $6.4 billion in prepayments in FY7 resulted in negative net disbursements. IDA had positive net disbursements of $6.8 billion in FY7, almost unchanged from FY6. Projected Bank-wide net disbursements in FY8 range from $2 billion to $5 billion, with IBRD at negative $1.2 billion to negative $3.3 billion. Table 2.1: Gross and Net Disbursements (FY5-7) (US$ Million) FY5 FY6 FY7 IBRD IDA Bank IBRD IDA Bank IBRD IDA Bank Gross Disbursements 9,722 8,95 18,672 11,833 8,91 2,743 11,55 8,579 19,635 Scheduled Repayments 12,17 2,8 14,116 11,57 2,134 13,641 1,852 1,891 12,743 Disbursements less Repayments -2,385 6,942 4, ,776 7, ,688 6,892 Prepayments 2,676 2,676 2,68 2,68 6,354 6,354 Net Disbursements -5,62 6,942 1,88-1,742 6,776 5,33-6,151 6, Investment Lending 2.8 Investment loans, accounting for about 75percent of FY7 approvals, remain the main instrument for deliveringjinancia1 and knowledge services to clients. There was a decline in approvals of Emergency Recovery Loans (ERLs) in FY7 compared to the previous two years. Stand-alone Technical Assistance Loans (TALs) were also utilized less in the face of declining demand in IBRD countries, increased utilization of Non-lending Technical Assistance (NLTA) and Recipient Executed Trust Fund (RETF) capacity building projects, and integration of TA components in other investment loans. However, the share of simple and repeater projects has increased to about 25 percent of number of approvals in FY7, compared to about ten percent in FY Based on exits during FY5-7, the average investment project had a life of 6.7 years, about the same as during FY2-4. The average age of active investment loans in the FY7 portfolio was 3.5 years, slightly lower than five years ago. However, the number of project exits has been declining, mainly in AFR. The decline in exits is partly explained by the increase in Additional Financing operations in FY7, which contributed to some increase in project extensions (see Para 2.32). Management would need to pay close attention to extensions of closing dates. 2.1 There has been a continuing decline in average preparation cost but a slight increase in the preparation time7 for investment loandcredits in FY7 (see Figure 2.4). Preparation cost per project has declined in real terms by about 25 percent since FY3, consistent with the increase in simple and repeater projects as well as Additional Financing operations. AFR was the only region with a preparation cost per project in excess of $4k in FY7, due mainly to a growing number of multi-country projects. The increase in average Bank-wide preparation time for investment lending in FY7 is partly explained by the increasing number of multi-country projects. This is measured as elapsed time from Project Concept Review to Board Approval.

23 Annual ReDort on Portfolio Performance FY OPCS is currently reviewing the Policy Framework for Investment Lending. The proposed approach is to modernize and streamline the policy framework towards: (a) creating a single principles based Investment Lending umbrella policy; (b) embedding a risk based model on internal controls governing Investment Lending; (c) rebalancing attention and resources between preparation, approval, and implementation stages; and (d) strengthening alignment with the current development and business model, borrower needs, and strategic directions. I Figure 2.4: Preparation Time and Cost per Investment Lending Operation (In Real Terms) = 45 v) lft Approval FY I 1 +Ave. Preparation Cost +A*. Elapsed Time (Concept to Board) 1 Development Policy Operations ; - ISZ * - 173; w, - IS$ U Development Policy Operations accounted for about a quarter of approvals in FY7. As part of the operational policy change in 24, the 25 percent ceiling on policy-based lending was removed, although Management committed to report annually to the Board on the anticipated share of policy based commitments. The projected level and share of DPOs for next year will be higher than in FY7 and closer to the average level of about one-third of total approvals during the past five years. In addition, as part of IDA 14 commitments, Management would seek guidance from IDA Executive Directors in the event the projected share of IDA DPO commitments was to exceed 3 percent. The share of DPOs in IDA lending in FY8 is projected to be below 3 percent IBRD accounted for two-thirds of DPO approvals in terms of lending commitments during thepastfive years. IBRD used DPOs to deliver 35 percent of its lending during the past five years, compared to about 25 percent for IDA. LCR processed almost half of DPOs during the past five years in terms of commitments, reflecting large demand of Middle Income Countries in LCR for utilization of this instrument to support second generation reform programs in a wide range of sectors and themes. AFR, ECA, and SAR have roughly the same share of DPO approvals by amount in their lending activities at about 15 percent each over the past five years. In terms of number, AFR delivered the most DPOs, mainly providing recurring support for the implementation of poverty reduction strategies. The amount of DPO approvals in SAR doubled since FY3, with increased use of this instrument in India, Pakistan, and Bangladesh.

24 Annual ReDort on Portfolio Performance FY Since their introduction in 1999, programmatic DPOs have become a widespread approach. They accounted for 9 percent of DPOs in terms of number and 85 percent in terms of commitments in FY7. Under programmatic operations, the Bank supports the implementation by borrowing countries of a medium-term program through a phased series of typically single tranche loans, each of which is disbursed on the basis of completed actions. The programmatic design is well suited for supporting complex institutional reforms requiring a flexible step-by-step approach. DPOs are increasingly rapid disbursing instruments--the average project life has declined to less than one year based on exits in FY5-7, compared to almost two years in FY2. IEG evaluates programmatic DPOs after the closing of the last DPO in a series, and to date there has not been a sufficient number of evaluations to assess the effectiveness of the programmatic approach Programmatic lending and the use of less complex designs reduced DPO preparation time and cost in FYU7 (see Figure 2.5). Average preparation cost in real terms has been declining since FY5, though the FY7 level is still about 1 percent higher compared to FY3. Average preparation time has declined to about seven months in FY7, compared to 8.8 months in FY6 and 8.1 months in FY3. In FY5, there was a restructuring of policies and processes guiding DPOs, including fast disbursing and programmatic operations, contributing to improved efficiency u) s ' 5 f 45 16v ~3 4 E- 35 a f Approval PI +Am. Elapsed Time (Concept to Board) 1 1 +Ave. Preparation Cost 1 a, 9 +-.E Q' s 6 $ 5a W E Guarantees 2.16 Because of their unique characteristics, the Guarantee amounts are not included in the Portfoliofigures. The Bank's Portfolio of 33 Guarantees is spread through all Regions, with the highest Bank exposure concentrated in EM and ECA. The Energy and Mining Sector accounted for 6 percent of Guarantee operations, followed by the Financial Sector with 16 percent. There are 15 new operations currently under preparation. Given the growing volume of Guarantees, and in response to Senior Management request, QAG carried out an assessment of Quality-at-Entry of Guarantees approved in FY5/6.

25 Annual ReDort on Portfolio Performance FY The QAG assessment of Quality-at-Entry of Guarantees in FY6 found the quality to be less than that of other Bank operations. The weaknesses identified were: (a) an unsatisfactory underlying policy and regulatory framework of the Borrower that had not been adequately addressed through credible actions; (b) generally no assessment of the risk of the guarantee being called; and (c) inadequate readiness for implementation. The Anchor Unit (FEU) is following up the QAG recommendations and has introduced simplification to the implementation processes. While Regions have full ownership of the Guarantee operations, FEU has clarified its role in the quality assurance process. Trust Funds 2.18 The FY6 ARPP recommended that Recipient Executed Trust Funds (RETFs) be recorded in the Bank s Portfolio. In June 26, the RETF product line was introduced as a first step towards a better recording of deliverables and cost relating to recipient executed projects carried out without any IBRDADA financing. Most of the entries in RETFs were converted from other product lines that had varying oversight; the result was that RETFs inherited some problems of data quality and consistency, which are now being addressed. All projects of $5 million or more, classified as RETF, will become part of the Bank s Portfolio definition during FY QAG review of RETF projects found weaker quality-at-entry than other Bank operations. RETF projects that were linked to ongoing or past operations were assessed as generally satisfactory - most of these RETF initiatives funded capacity building and had focused objectives with well targeted interventions. On the other hand, stand-alone RETF projects, especially in conflict countries, tended to have unrealistic objectives and designs that were not consistent with implementation capacity. The QAG review recommended: (a) greater realism in objectives and results framework; (b) more attention to institutional and implementation arrangements; (c) subjecting RETF projects to regular Bank processes and quality assurance mechanisms; and (d) ensuring adequate preparation and supervision budgets. B. FOCUS OF LENDING f 2.2 A breakdown of lending ap rovals by themes gives an indication of how the Bank implemented the two-pillar strategy during FY2-7 (see Table 2.2). About a quarter of the lending went to finance and private sector development, mainly in infrastructure, regulation and competition, and enterprise and banking restructuring. Slightly more than a quarter of lending went to human and social development, mainly in education and health. Slightly less than a fifth of the lending supported public sector governance, mainly in public expenditure and civil service reforms. The remaining 3 percent of the lending addressed rural development, urban development, and environment with each theme having roughly equal shares. The two-pillar strategy was introduced in 21. The two pillars are: (a) building the climate for investment, jobs, and sustainable growth; and (b) empowering poor people to participate in development and investing in them.

26 Annual Reuort on Portfolio Performance FY7 1 I Table 2.2: Approvals by Theme (FYOO-7) FYOO-1 M2-4 FYO5-7 Major Theme %of %of %of Bank- %of %of %of Bank- %of %of %of Bank- IBRD IDA wide IBRD IDA wide IBRD IDA wide Economic Management Public Sector Governance Finance and Private Sector Development Social Development Human Development Urban Development Rural Development Environment Total " Public Sector Governance includes Rule of Law. *' Finance and Private Sector Development include Trade and Integration. 3' Social Development includes Social Protection and Risk Management Economic growth has been a prominent theme in IBRD lending. In terms of themes, the largest share of IBRD lending approvals--about 3 percent--during the period FY5-7 went to Finance and Private Sector Development mainly to the infrastructure sectors; this share is more than twice as high as those of other themes. Nonetheless, IBRD lending has focused on countries where poverty remains prevalent - two-thirds of IBRD lending went to non-investment Grade countries. However, a recent IEG Evaluation' found less progress in three important issues included in many IBRD CASs: (a) addressing inequality; (b) fighting corruption; and (c) meeting environment challenges The distribution of IDA lending reflects a stronger emphasis than in IBRD on human and social development. Thematically, about one-third of IDA lending during FY5-7 went to human and social development," compared to a fifth in IBRD. Lending for human development support empowerment and MDGs. Finance and Private Sector Development" (FPD) accounted for about a fifth of IDA lending, compared to about one-third in IBRD. A main component of FPD thematic lending was support for infrastructure investments, which have high financing requirements and are critical to improving access to markets and services. Public sector governance also accounted for a fifth of lending, slightly higher than IBRD. Rural Development had a share of 15 percent in IDA, double that of IBRD. Urban and Environment together represented about 1 percent of IDA lending, half of the share in IBRD lending. A recent paper for IDA1512 has raised the issue of appropriateness of sectoral allocation of lending, suggesting that lending patterns should focus more on sectors (i.e., infrastructure) with decreasing shares in ODA than on those (ie., social sectors) where shares in ODA are increasing The planned OPCS Results Paper is an opportunity to systematically review whether the lending patterns are appropriate. While overall lending is the aggregation of decisions at lo l1 l2 Development Results in Middle-Income Countries: An Evaluation of the World Bank's Support, IEG 27. This is the aggregation of the following themes: (a) human development; (b) social development, gender, and inclusion; and (c) social protection and risk management. The trade and integration theme was included in this classification. The Demand for IDAIS Resources and the Strategy for their Effective Use, FFW 27.

27 Annual Revort on Portfolio Performance FY7 11 the country level, the relatively low effectiveness of country programs as measured by the CAEs raises the issue of whether the lending programs are addressing the key development constraints and priorities (see Chapter 5). In addition, with new strategic directions for the Bank, a robust results framework could be developed that would provide benchmarks for evaluating whether future lending and portfolio trends are moving in the right direction. Future ARPPs would utilize this results framework as a basis for evaluating lending trends. C. IDA AND FRAGILE STATES 2.24 FY7 IDA approvals of $11.7 billion were at an all time high, representing an almost 25 percent increase over the previous year. Average annual commitments in nominal terms increased from $6.4 billion under IDA12 and $8.3 billion under IDA13 to $1.6 billion thus far under IDA14. IDA approvals have grown faster than IBRD since FYOO. The IDA Portfolio of $43.8 billion in FY7 was also at a historic high, now accounting for 44 percent of the Portfolio compared to 36 percent five years ago. Average annual disbursements also showed an upward trend, increasing from $5.6 billion during IDA12 to $7.8 billion in IDA13 and $8.9 billion during the first year of IDA Lending to Fragile States increased by about 6 percent over FY6 to $2.1 billion in FY7 (see Figure 2.6). The share of Fragile States in total IDA approvals was 18 percent in FY7. In addition, about 4 percent of FY7 RETF approvals were in Fragile States. The Bank s strategy uses a segmentation of Fragile States into four groups based on client environment: deterioration, prolonged crisis or impasse, post conflict or political transition, and gradual improvement. A large share of the lending during FY4-7 went to post conflict or political transition states. The risks of lending to Fragile States are relatively high - about 4 percent of the projects exiting the portfolio during FY4-7 were unsatisfactory (see discussion in Chapter 3). QSA7 found both design and supervision issues that would have to be addressed to improve project outcomes The IDA Results Management System (MS) established in IDA13 has been enhanced considerably in IDA14. The Rh4S has become an integral part of the Bank s effort to enhance results orientation. The Rh4S indicates uneven progress across IDA countries, with Fragile States lagging. Several key human development indicators remain a cause of concern. The Bank is further refining the IDA15 RMS indicators and related signaling mechanisms to reinforce the focus on results.

28 Annual ReDort on Portfolio Performance FY ,5.- E 2, E tft 1,5 to 2 1, e L a 5 Figure 2.6: Approvals for Fragile States (FYO3-7)" (US$ Million in Real Terms) FY3 FY4 FY5 FY6 FY7 1 LTotal Lending +Lending to Post-conflict Countries I * Includes countries in the Fragile States Category during FY5-8 D. IBRD 2.27 The Bank has been systematically adapting to the changing environment in IBRD client countries. Over the past few years, IBRD lending has been facing a constrained environment with a contraction of real demand for sovereign external financing. There have been general client concerns, especially from MICs, about the attractiveness of traditional IBRD loans relative to alternatives from other sources, including financial markets. There has been an evolution in the way the Bank supports IBRD clients in meeting their development needs, with the latest paper-strengthening the World Bank's Engagement with IBRD Countries-defining the focus areas. In September 27, the Bank simplified and reduced loan charges to IBRD clients Bank efforts resulted in maintaining IBRD lending levels in real terms since FY2 (see Figure 2.7). Without the efforts to remain relevant with sophisticated clients that have wider financing options, IBRD lending might have fallen. Nonetheless, IBRD net commitments of $54. billion accounted for 54 percent of the FY7 Portfolio, compared to 65 percent five years ago, though part of the portfolio decline is attributed to increased use of quick disbursing DPLs. SAR and EAP were the only regions that posted increases in IBRD approvals in FY The ability to respond to client demand will be critical to lending to MICs. Many clients with ample foreign resources such as China and Brazil continue to borrow mainly for the embedded knowledge and learning services where the Bank is seen to have a comparative advantage. In a recent Client Survey of MICs carried out by IEG in the context of its paper on MICs, some 8 percent of the respondents reported that their country's development goals in the coming five years could be best serviced with Bank lending remaining the same or increasing.

29 Annual ReDort on Portfolio Performance FY m Figure 2.7: IBRD Approvals by Client Segment (3-Year Moving Average in Real Terms) I woo Mol no2 FY3 FY4 FY5 FY6 FY7 +Investment Grade +Core IBRD +BLEND (IBRD portion) +Total IBRD Note: IBRD Investment Grade Countries include China, and Blend Countries include India. 2.3 Lending to corei3 IBRD clients has been stagnant (see Figure 2.7). These countries do not have the same access to financial markets as the Investment Grade countries. In many cases, the core IBRD countries are those that need both financial and knowledge services from the Bank, hence there is an expectation that there would be more lending to this set of countries. Based on SFR analysis, about 75 percent of the CASs in core IBRD countries have programs that on average are 55 percent below the lending envelope, indicating substantial space for lending growth. Four out of 27 core IBRD countries accounted for about 7 percent of the lending - the challenge is how to expand engagement. It is recommended that Management explore opportunities to increase financial and non-financial assistance and services to core IBRD countries. E. INNOVATIONS IN LENDING 2.31 While scaling up and the use of simple and repeater projects have been increasing, lending has also been characterized by innovations in approaches and instruments. In MICs, new services are being provided in response to demand from sophisticated clients, including the use of fee based delivery of services. Programmatic approaches have become widespread in DPOs and are also being utilized in investment lending through APLs and SWAps. Carbon Finance operations are growing,14 and due diligence and processing guidelines were issued in October 27 to ensure appropriate methodology for assessing design and risks. In addition, the lending growth in the following areas is noteworthy The main growth area in investment lending has been the use of Additional Financing operations following the introduction of guidelines in June 25. Approvals for additional financing amounted to $1.8 billion in FY7, doubling that of the previous year. LCR and AFR accounted for more than half of Additional Financing in terms of amount; among networks, INF accounted for about half. Additional Financing approvals result in an increase in commitments l3 l4 This is defined as IBRD clients that are neither Investment Grade nor Blend countries. The Bank currently manages 11 Carbon Funds and Facilities worth more than $2 billion.

30 Annual ReDort on Portfolio Performance FY7 14 in existing projects, which typically involve extending the closing dates of projects. QEA8 found no differences in quality-at-entry of Additional Financing projects compared to other investment projects Regional projects and programs are growing as cross border dimensions of environment, health, infrastructure, and trade facilitation take on greater importance. About $1.9 billion of regional projects are recorded in the portfolio, though based on feedback from the Regions; the amount is understated and is closer to $3 billion. Regional projects in the portfolio are concentrated in AFR. QEA8 included a sample of regional projects and found several aspects which could be improved. QEA8 recommends: (a) improving efficiency and reducing the cost of preparing regional projects by addressing duplication of work in Bank processes, such as the multiplicity of fiduciary arrangements and staff involved with a regional program; and (b) ensuring adequacy of supervision budgets in light of the nature of regional programs. In addition, there should be a review of lending instruments for multi-country programs To respond to the unique requirements of sub-national borrowing for Infrastructure, the WB and IFC Boards approved the pilot Sub-National Development Program in 26. The program addresses the lack of Bank instruments to best support the growing sub-national client segment; the Bank s Articles of Agreement prevent the Bank from lending directly to subnational entities without a sovereign guarantee. A joint IBRD-IFC Sub-National Department (SND) was created in November 26 to implement the program. In FY7, four transactions under the program were approved for a total exposure of $156 million. SND has developed a pipeline of over 2 projects representing an investment value of over $5 million. The goals of the program extend beyond providing finance, to include improving the ability of sub-national governments to access financial markets more widely. To this end, SND helped create, in July 27, a multi-donor technical assistance facility (a window within the Public-Private Infrastructure Advisory Facility) that helps sub-national entities improve their institutional capacity and creditworthiness. F. INFRASTRUCTURE 2.35 Infrastructure lending has increased during FY5-7 in response to the Infrastructure Action Plan (see Table 2.3). The share of infrastructure in Bank-wide lending increased from 28 percent during FY2-4 to 35 percent during FY5-7, close to the share during FY-1. Infrastructure accounted for almost 4 percent of IBRD lending, and about 3 percent of IDA approvals during FY5-7. Transportation accounted for half of infrastructure lending in FY7, with Water, Sanitation and Flood Protection having a one-third share Project outcomes have been generally satisfactory. Transport Sector projects have been consistently high performers, with more than 9 percent satisfactory outcomes during FY4-716 in both IBRD as well as IDA countries. Performance of Water Supply and Sanitation Sector l5 l6 Many regional or multi-country projects are classified in the portfolio as country level operations. IEG has completed the evaluation of about half of FY7 exits.

31 Annual ReDort on Portfolio Performance FY7 15 projects has improved considerably during FY4-7 to 88 percent satisfactory compared to FYO1-3 exits when performance was below 7 percent. In IBRD countries, the performance of Urban Development Sector projects improved from 55 percent satisfactory during FYO1-3 to 99 percent during FY The challenges facing infrastructure lending include improving the interface with other sectors and responding to cross-border issues. This would introduce greater complexity to designing and supervising infrastructure operations. The integration of the INF and ESSD networks provides an organizational response to these challenges by enabling better cross sectoral coordination. The quality assurance process should keep pace with changes in focus and introduction of new approaches in infrastructure lending to ensure continued satisfactory performance of project outcomes. Table 2.3: Approvals by Sector (FYOO-7) Major Sector woo-1 M2-4 FYO5-7 % of Bank-wide Agriculture, Fishing and Forestry Public Administration, Law and Justice Infrastructure Human Development Finance, and Industry and Trade Bank-wide Notes: l' Infrastructure includes Energy and Mining, Transportation and Water, Sanitation and Flood Protection. Human Development includes Education and Health and other social services. 3' Finance, Industry, and Trade includes Information and Communications. G. RECOMMENDATIONS OPCS Results Paper to develop framework for assessing aggregate lending and portfolio trends in the context of new strategic directions; and Management to explore opportunities to increase financial and non-financial assistance and services to core IBRD countries.

32 Annual Reuort on Portfolio Performance FY PORTFOLIO PERFORMANCE 3.1 This Chapter reports on portfolio performance by reviewing IEG evaluations of latest exits and analyzing the existing portfolio based on risk indicators and QAG assessments. The analysis uses aggregate outcomes of exits covering three year periods except for the period FY4-7, which includes partial IEG evaluations of FY7 exits. The main messages from the Chapter are: Development outcomes of operations exiting the Bank portfolio through FY6 continued to improve. On the basis of three year moving average, about 8 percent in terms of number of projects, and 87 percent weighted by disbursement, of projects exiting the portfolio in FY6 had satisfactory development outcomes. However, Management will have to monitor recent performance which indicates a decline in terms of number of projects based on IEG evaluations of about half of the FY7 exits thus far. There was deterioration in performance of AFR projects with outcomes of FY6 exits at 65 percent satisfactory compared to 78 percent in FY5 and 83 percent in FY4. AFR had 61 percent satisfactory performance in FY7 based on about half of FY7 exits evaluated to date. Among Sector Boards,17 there were declines in project performance in Health, Nutrition and Population, Private Sector Development, and Public Sector Governance during FY4-7 compared to FYO1-3. A review of the IEG evaluations of projects in these Sector Boards revealed that project design flaws were the main factors contributing to unsatisfactory outcomes. The results of QEA8 indicate a greater than 9 percent satisfactory quality-at-entry of projects approved in FY6-7. There is also a convergence of performance by Regions compared to previous years. Historically, QEA ratings are about 1 percentage points higher than actual project outcomes. QAG is tightening its QEA methodology to reduce the disconnect between QEA ratings and project outcomes. With respect to supervision, there are three developments worth noting. First, the Proactivity Index indicates that teams have been addressing in a satisfactory manner the underlying issues of problem projects. Second, the number of projects with delayed effectiveness has been increasing, mainly due to lengthy client approval processes and, for some Regions, the relatively high number of effectiveness conditions. Third, QSA7 results show no significant difference in supervision quality of projects based on the location of TTL; however, panels find that more could be l7 The FY7 Sector Board classification was used for purposes of the analysis in this chapter. The Finance and Private Sector Development Boards were merged in FY8.

33 Annual ReDort on Portfolio Performance FY7 17 done to complement local presence with appropriate use of HQ staff with global experience. Lack of candor in project reporting has diminished the effectiveness of the current system of measuring and reporting portfolio performance. The Realism Index continues to show significant under-reporting of project risks, a problem that has been recognized for some time but has not been effectively addressed. A recent review of quality of reporting covering FY7 projects estimated the share of problem projects in the portfolio to be twice that reported in ISRs. The usefulness of the current system of portfolio reporting as a management tool is dependent on improving candor in reporting project performance and risks. One way forward towards improving portfolio performance measurement and reporting is to establish a stronger link between risks identified at entry and the reporting of project performance. The current system rates projects entering the portfolio at zero risk, and flags project risk only when problems begin to surface. The revision of the measurement and monitoring of project risk during implementation should build on the reforms initiated on July 1, 27 to ensure a more systematic and comprehensive assessment of risks at appraisal. It is recommended that OPCS and QAG work towards reviewing and recommending appropriate changes to the current system of portfolio risk measurement and reporting. A. DEVELOPMENT OUTCOMES OF PROJECTS 3.2 The Development Outcomes of operations exiting the Bank s Portfolio continued to improve through FY6 (see Figure 3.1). Satisfactory development outcomes based on FY6 exitsig were 82 percent by number of projects and 89 percent when weighted by disbursement. Because of significant year-to-year volatility, the development outcome trends are best analyzed using three year moving averages. On this basis, about 8 percent in terms of number of projects, and 87 percent weighted by disbursement, of operations exiting the portfolio in FY6 had satisfactory development outcomes. This performance represents sustained improvement since FY9. It also meets the 8 percent goal set by management a decade ago. However, this performance will have to be tempered by the following developments: IEG project assessments covering about half of FY7 exits thus far show lower outcomes Bank-wide in terms of number of projects compared to the FY6 exits.2 * IEG uses a six-point scale in rating project outcomes. The frst three ratings - Highly Satisfactory, Satisfactory, l9 2o and Moderately Satisfactory - indicate a satisfactory outcome while the other three ratings - Moderately Unsatisfactory, Unsatisfactory, and Highly Unsatisfactory - indicate an unsatisfactory outcome. Unless specified otherwise, the terms satisfactory and unsatisfactory used in this chapter follow the above definitions. About 85 percent of FY6 exits have been evaluated by IEG. With about half of the FY7 exits evaluated, the satisfactory performance of FY7 exits in terms of number of projects is 75 percent compared to 82 percent for FY6 exits. There is no significant change thus far in the performance of FY7 exits compared to FY6 exits in terms of disbursement.

34 Annual ReDort on Portfolio Performance FY7 18 Development outcomes for FY7 are projected to be lower than 8 percent by number of projects, which would reverse the upward trend since FY4. SAR, LCR, EAP, and AFR are projected to show declines. The Bank performance reported above is measured on the basis of projects that were evaluated by IEG to be Moderately Satisfactory or better. The number of projects with Moderately Satisfactory ratings during FY4-7 accounted for about 25 percent of total, indicating opportunities for further improvements in project quality. IEG evaluations of CAEs completed in FY97-7 show that aggregate performance of country programs is less than 6 percent satisfactory. This indicates that while outcomes of individual projects may be improving, the projects may not be addressing the key development constraints or priorities at the sector or country level (see Chapter 5). Figure 3.1: Proportion of Operations with Satisfactory Outcomes* (Based on Year of Project Exit) 5, l / I I I, I I l I I / / i I I I I i I I / / I I I FY8 FY83 FY86 FY89 FY92 FY95 FY98 FYO1 FY4 FY7* + 3 Year Moving Avg. (By No. of Projects) +3 Year Moving Avg. (Weighted by Disbursement) * FY7 Outcomes based on QAG Projections. Regional Performance 3.3 Aggregate project outcomes improved in all Regions during FY4-7 compared to previous periods, with a growing convergence of performance among four Regions. SAR and AFR performance lagged the other Regions in FY4-7, with SAR showing little improvement over FY1-3 compared to the rest of the Bank. However, SAR performance improved significantly in FY6 to 91 percent; the challenge is to maintain this level. On the other hand, AFR performance has been deteriorating from 83 percent in FY4 to 78 percent in FY5 and 65 percent in FY6, in part due to low ratings of first generation Multi-Country HIV-AIDS Program (MAP) projects. IEG evaluations of about half of AFR FY7 exits indicate a slightly lower performance compared to FY6 exits. Figure 3.2 presents the percentage of satisfactory project outcomes by Region, weighted by disbursement, for FY4-7 compared to FY1-3 and FY98-.

35 Annual ReDort on Portfolio Performance FY C EAP LCR MNA ECA P p! SAR AFR Ban k-w ide Figure 3.2: Project Outcomes by Region %Satisfactory Outcome by Disbursement (IEG Rating) 1 FY98- FYOI-3 UFY Further improvement in the performance of AFR projects is key to improving overall Bankperformance. There are several areas where the performance gap between AFR and the rest of the Bank is significant. Within IDA, aggregate satisfactory AFR project outcomes during FY4-7 were lower than those of non-afr projects. This is mainly due to the lower outcomes of projects in Fragile States in AFR compared to other Regions. Among networksy2l AFR performance is lower than non-afr in HDN, PREM, and SDN. A comparison within the FPD network was not included due to the relatively small number of IEG evaluations for AFR during FY4-7. Please see Table 3.1 for comparative performance between AFR and other Regions. Table 3.1: IEG Evaluations for Africa and Other Regions (FYO4-7) (Percent Satisfactory Outcome) AFR Other Regions Ban k-wide By No. of BY By No. of BY By No. of BY Projects Disbursement Projects Disbursement Projects Disbursement IDA Fragile States HDN PREM SDN ESSD INF QEAS rated the overall quality-at-entry of AFR projects approved in FY6-7 as satisfactory. In addition, AFR recently completed a review of FY7 projects and found no significant issues with respect to readiness for implementation. The AFR review also found that 75 percent of the operations approved in FY7 had satisfactory M&E arrangements using the IDA14 criteria. Nonetheless, AFR should review its portfolio focusing on older projects, i.e., those exiting during the next two years, and make a realistic assessment of the trajectory of project performance during the coming years. The next QSA should also provide Management with an assessment of performance of the AFR portfolio segmented by project age and network. 21 For analysis purposes, SDN is divided into INF and ESSD.

36 Annual ReDort on Portfolio Performance FY AFR has been doing an extensive review of quality assurance to improve performance of projects in the Region. The Region has also completed separate reviews of the results frameworks and M&E arrangements of earlier projects, focusing on Transport, Education, Health and Social Protection sectors. The reviews found the results frameworks to be satisfactory in majority of the projects, with some weaknesses in clarity and realism of PDOs and specificity of outcome indicators which are being addressed. To further strengthen the results focus in operational work, the Region has launched the Africa Results Monitoring System that was developed and piloted in FY7. Finally, there is a continuing effort to strengthen staff capacity. Sector Board Performance 3.7 There was significant variation in development outcomes among Sector Boards based on IEG evaluation of project exits. Several Sector Boards had improved outcomes during FY4-7 compared to FYOl-3, notably Environment, Economic Policy and Urban Development, which had increases of more than 3 percentage points. Three Sector Boards - HNP, PSG, and PSD - were below Bank-wide average during FY4-7 and showed declines from previous periods. Figures 3.3 and 3.4 show the major improvers and decliners in performance among Sector Boards. Figure 3.3: Major Improvers by Sector Board Urban Development Social Protection Water *icull :ure and Rural Development Economic Policy Environment Ban k-w ide %Satisfactory Outcome by Disbursement (IEG Rating) 1 I3 W98- WOI-3 OWO4-7 1

37 Annual ReDort on Portfolio Performance FY7 21 Figure 3.4: Major Decliners by Sector Board Public Sector Governance?i 2 Private Sector L Development Q I I I I $ Health, Nutrition and Population I [ DFY98- ID FYOI-3 OFyO4-7 1 %Satisfactory Outcome by Disbursement (IEG Rating) Health, Nutrition and Population 3.8 Since FYOl, HNP project outcomes have been lower than Bank-wide average.22 To identify the main factors that contributed to the lower performance of HNP Sector Board projects, QAG performed a review of the 75 HNP projects that exited during FY4-7 focusing on 25 projects that were rated by IEG as unsatisfactory. The review found that generally, the set of Project Development Objectives were appropriate and relevant, had good Borrower ownership, and were linked to national priorities and the CAS. However, almost 7 percent of the unsatisfactory projects had weak designs that eventually resulted in unsatisfactory outcomes. In addition, about half of the projects had implementation problems that contributed to unsatisfactory project performance. 3.9 The main design flaws in unsatisfactory HNP projects were rooted in unrealistic assumptions of absorptive capacity of the Borrower, resulting in projects that were not consistent with implementation capacity. In addition, the results frameworks were inadequate with ambiguous outcome indicators, absence of baseline data, and inadequate monitoring and evaluation systems. The review found that many of the projects did not have adequate sector work and were appraised and brought to the Board prematurely. Design flaws of HNP projects were not adequately addressed during implementation. During supervision, corrective actions to address quality-at-entry issues were either not undertaken or not effective in about half of the cases. The three major factors that underpin poor implementation and supervision performance were: (a) lack of Management attention; (b) inadequate reporting due to weak M&E systems; and (c) poor Borrower performance. 3.1 A review of the combined HNP results in QEA6-8 and QSA7-8 showed the same weaknesses as those identified by the review of unsatisfactory exits (see Table 3.2). This means that unless these weaknesses in the current HNP portfolio are corrected, HNP performance would continue to lag Bank-wide performance. The pressing challenge to HNP is to effectively 22 HNP project outcomes during FY4-7 were 63 percent satisfactory by number of projects and 7 percent by disbursement, compared to Bank-wide outcomes of 8 percent by number of projects and 87 percent by disbursement.

38 Annual ReDort on Portfolio Performance FY7 22 address design issues in projects in the current portfolio, in addition to ensuring quality of new projects. While the end-fy7 HNP commitments at risk show improvement from the previous year, from 23 percent to 16 percent, this should be viewed in the context of very low realism numbers for HNP. Table 3.2: Results of QAG Assessments for HNP Selected Questions (Percent Moderately Satisfactory or Better) I HNP Bank-wide Quality at Entry (combined QEA6-8) Appropriateness of project approach and complexity? Adequacy of attention to social development issues? 7 91 Adequacy of arrangements for mitigating adverse environmental impacts? Capacity to implement the project? Capacity to implement the project's poverty and social development agenda? Prospects for completing the project within the prescribed time-frame? Adequacy of arrangements for monitoring and evaluation Appropriateness of arrangements for evaluating impact and measuring outcomes, using relevant national, sectoral or project-level data? Appropriateness of arrangements for monitoring poverty and social aspects? Quality of design to mitigatelmanage risks? Value added from Management (Sector Management) Quality of Supervision (combined QSA6-7) Quality of PADS results framework (including intermediate performance indicators) and readiness of implementation arrangements for M&E Extent to which there are adequate arrangements linking project results framework to the borrower's monitoring and evaluation system 5 74 Quality of baseline data Performance monitoring Adequacy and speed of management attention and actions Quality of timeliness of data (including the intermediate outcome indicators) to support the key performance indicators Note: Combining the HNP results in QEA6-8 and QSA6-7 results in a robust sample with statistical significance of 9 percent and confidence interval of +I- 1 percent There is recognition in the HNP Sector Board and HD Network of a need to do more to address quality issues. The recent HNP Strategy Paper called for seeking the right balance between the Regions' responsibility for portfolio management and quality control, and the responsibility of the HNP Anchor and Sector Board for monitoring and assisting the Regions. HDN is awaiting the results of an ongoing IEG review of the HNP sector which will be an input to a program for quality improvement. Nonetheless, the QAG review recommends the following actions: Regions and HD Council to immediately commission a special review of projects for which appraisals have been completed but which have not yet gone to the Board;

39 Annual Report on Portfolio Performance FY7 23 Over the next few months, Regions and HNP Sector Board to review projects in the portfolio rated MS for DO or IP to look into the appropriateness of actions being undertaken to address project issues; HNP Sector Board to agree with Regions on how to provide support to Regional teams in the design of projects, including the use of quality enhancement reviews; and In the medium term, the HD Council will commission a review that will address critical strategic issues related to the HNP Strategy and the appropriateness of the operations in meeting the objectives of the Strategy. Private Sector Development and Public Sector Governance 3.12 Performance of PSD projects in FY4-7 was about 75 percent satisfactory in terms of disbursement and about 6 percent in terms of number of projects. Weaknesses in design dominate as the main factor for the unsatisfactory rating of FY4-7 PSD exits.23 Design is oneand-a-half times as frequently cited by IEG as the second most important factor, namely weaknesses in supervision. The two main reasons, cited by IEG, as underlying weaknesses in design that have contributed to unsatisfactory outcomes were: (a) weak government commitment or ownership; and (b) lack of relevance to, or misalignment with, country or local conditions The review of unsatisfactory PSG projectg4 found the same weaknesses as in PSD projects. The three factors most frequently cited in IEG s evaluation underlying operations with unsatisfactory outcomes were: (a) poor project design; (b) inadequate M&E (either at entry or during implementation); and (c) gaps in Bank supervision. Weakness in design was the main factor for 88 percent of the operations classified as unsatisfactory. Lack of government commitment, overambitious objectives, and failure to address key constraints were factors that contributed to weak design The abovefindings for PSD and PSGprojects suggest two areas of improvement. First, there is need to have a better understanding of country conditions, political economy of reforms, and counterpart capacity to implement projects. This could be achieved through stronger analytical work and improved dialogue with the government and other stakeholders utilizing ESW as the main instrument. Second, the Bank is missing important opportunities to correct design and other flaws during the course of supervision; more effort needs to be made to enhance the quality of supervision. An effective M&E system would provide the appropriate feedback in identifylng threats to the achievement of development objectives. In addition, as discussed in Chapter 5, PSD and PSG projects have the highest disconnect between project and sector outcomes which suggests a wider set of issues linked to the strategic approach to these sectors and the design or scope of the operations. The PSD and PSG Sector Boards may want to review the issues in the implementation of the sector strategies The review covered 19 unsatisfactory operations out of 46 exits evaluated by IEG. The review covered 25 unsatisfactory operations out of 75 exits evaluated by IEG. While the performance in terms of number of projects was about 65 percent, performance in terms of disbursement was about 8 percent.

40 Annual Report on Portfolio Performance FY7 24 Client Segments IBRD 3.15 Performance of IBRD projects improved significantly during the period FY4-7 with a 91 percent satisfactory rating compared to 77percent during FYOl-3 (see Figure 3.5). Of 17 Sector Boards whose IBRD projects were evaluated by IEG, 11 had satisfactory perf~rmance~~ of 9 percent or better during FY4-7; of these, five achieved improvements of 3 percentage points or more compared to FYO1-3. The Education, Finance, and Transport Sector Boards have been consistently high performers during the past ten years. The Sector Boards that were below IBRD average were: Health, Nutrition, and Population; Private Sector Development; Energy and Mining; and Public Sector Governance. In terms of instrument, IBRD DPLs had a 95 percent satisfactory performance with Investment Loans at 89 percent. Figure 3.5: Proportion of Operations with Satisfactory Outcomes (IBRD vs. IDA) (3 Year Moving Average Weighted by Disbursement) M9 M92 FY94 FY96 FY98 FYOO FY2 M4 FY6 IBRD +IDA 3.16 In a client survey conducted aspart of the IEG review of MICs, more than 8 percent of the respondents viewed the Bank s programs and services as moderately effective or better. About 45 percent of the respondents viewed the Bank as more effective than other multilateral development banks and bilateral programs; 21 percent of the respondents rated the Bank as less effective. While government officials had a positive view of the Bank s performance, civil society representatives were not as positive. The relevance and quality of the Bank s work were rated higher than responsiveness and ease of access. IDA 3.17 There is scope for improving performance of IDA projects. IDA performance was lower than that of IBRD during FY4-7, though this gap may be partly explained by the higher risk faced by IDA projects due to generally weaker institutions in IDA countries. Nonetheless, there are areas where IDA performance could improve. Among Sector Boards, HNP and PSD IDA projects were at 7 percent or lower whether in terms of number or projects or 2* Weighted by disbursement.

41 Annual Report on Portfolio Performance FY7 25 disbursement; RDV IDA projects had a 77 percent satisfactory rating in terms of disbursement which is much lower than the 98 percent performance of RDV in IBRD. Narrowing the performance gap of IDA and IBRD projects in these Sector Boards would help improve overall IDA performance. Table 3.3 shows the Sector Boards where there was a performance gap greater than 1 percentage points between IBRD and IDA projects based on exits during FY4-7. Table 3.3: FYO4-7 IBRD and IDA Performance Gap by Sector Board (Percent Satisfactory Outcome) IBRD By No. of BY By No. of BY Projects Disbursement Projects Disbursement Agriculture and Rural Development Education Energy and Mining Economic Policy Health, Nutrition and Population Public Sector Governance Urban DeveloDment Based on the FY6 Bank Client Suweg6 conducted by EXK stakeholders from IDA countries found Bank relevance to be high and had positive perceptions of the Bank s mission. IDA stakeholders also regarded poverty reduction as the most important priority in their countries, with agricultural development as the best way to achieve this goal - improving the outcomes of RDV is therefore critical. They considered the provision of financial resources to be the most important contribution of the Bank, in contrast to IBRD respondents who considered knowledge services to be the main value added of the Bank In Fragile States, the sizeable performance gap between AFR and the other regions indicates significant room for improving outcomes of Bank operations. Development effectiveness of operations in Fragile States exiting the portfolio during FY4-7 was 61 percent satisfactory, weighted by disbursement, compared to 67 percent during FYO1-3 and 59 percent during FY98-. During FY4-7, non-afr operations in Fragile States had 89 percent satisfactory rating, weighted by disbursement, compared to 49 percent for AFR operations in Fragile States. 3.2 QSA 7 found weaknesses in project design (66percent moderately satisfactory or better) and readiness for implementation at approval (63 percent moderately satisfactory or better)27 in Fragile States projects. Panels recommended that supervision improvements emphasize the following areas: (a) focus on sustainability, especially on institution building; (b) performance monitoring; (c) Management guidance; and (d) realism of reporting project performance. Improving Bank institution building capacity has become more critical with the expansion of the focus of Bank work in Fragile States to the more ambitious and complex objective of State IDA Client Survey 26 Review, The World Bank, These were part of the Context Section of the QSA7 Questionnaire.

42 Annual Report on Portfolio Performance FY7 26 building, which requires even more sophisticated skills. A recent IEG review of Fragile States also flagged the issue of inadequate managerial attention There are ongoing efforts to strengthen internal Bank support for Fragile States. A recent IEG report2* recommended: (a) ensuring adequate incentives for staff with relevant skills to work in these countries; and (b) streamlining the organizational structure for Fragile States and Conflict work. The streamlining has been implemented with the merger of the Fragile States Group and the Conflict Prevention and Reconstruction Unit in July 27. The restructuring of staff incentives was addressed in the March 27 paper Strengthening the Organization Response to Fragile States and will be included in the forthcoming StafJing Strategy Paper. Realism of Reporting Risks B. PORTFOLIO PERFORMANCE AND MEASUREMENT 3.22 Lack of candor in reporting project risks has diminished the effectiveness of the current system of indicators in tracking portfolio performance. In FY7, the measurement of the Realism Index was revised to make it a more meaningful measure of quality of portfolio reporting and to make it more robust. While the Realism Index shows improvement from 46 percent to 56 percent during FY7 using the revised measurement system, the measure still shows significant under-reporting of risks. This makes the current reported measures of portfolio risk unreliable A recent QAG review concluded that under-reporting of risks continues to be significant in FY7- the conclusion is consistent with the QSA 7findings which covered the end-fy6portfolio. A sample of 125 projects in the FY7 portfolio was reviewed by panelists focusing on the candor of risk reporting and the quality of explanation of the ratings. The review was conducted with the Quality Units of the Regions which nominated staff to review questionnaires where the findings of QAG panelists deviated significantly fiom the ISR. The review estimated the share of problem projects in the FY7 portfolio to be twice that reported in the ISRs. In addition, panelists found significant under-reporting of risks in the fiduciary, M&E, and project management areas. Table 3.4 gives the ISR and Panel ratings for DO, IP and risk factors. 28 Engaging with Fragile States: World Bank Support to Low Income Countries Under Stress, IEG, 26.

43 Annual Report on Portfolio Performance FY7 27 Table 3.4: FW7 Risk Ratings - QAG Review vs. ISR DO, IP, and Risk Factors FY7 Portfolio (N=1,479) ISRs % Unsat. QAG Panels % Unsat. % Diff. Development Objectives Implementation Progress Legal Covenant Safeguard M&E Financial Management Procurement Project Management Actual Problem Projects Potential Problem Projects The review found that there was a reluctance to downgrade project ratings until the problems became prolonged. There was also a tendency to upgrade project ratings prematurely, based on expectations of improvement rather than concrete results. Ratings for project management often did not reflect the shortcomings described in the ISR or in other documents such as the Aide Memoire or letters to government. The review suggested some reasons for the under-reporting: (a) many teams shared ISR ratings with the government; (b) managers were not consistent in identifylng rating inconsistencies with reports; and (c) teams perceived a link between project ratings and team performance The QAG review identiped several actions that would help improve realism in reporting project performance: Regions to ensure that ISRs (text and ratings) are kept internal as intended, to ensure the candor of Bank reporting; Regions and OPCS to develop short checklists to aid TTLs in determining ratings for discretionary ratings, especially project management, procurement and M&E, which showed the highest levels of disconnect; OPCS and Regions to provide more training on ISR standards, especially for newlyappointed TTLs; and Regions to clarify responsibilities for ISR quality and oversight between SMUs, CMUs and Quality Teams, and reinforce incentives for candid reporting. Portfolio Risk 3.26 The results of the latest 1 IEG evaluations provide a more reliable measure of portfolio risk than existing estimates. Table 3.5 below gives the percentage of projects at risk by number of projects and commitments categorized by client group using IEG evaluations which covered project exits during the past 3-4 years. The results show that portfolio risk measured by number of projects is substantially higher than risk measured in terms of commitments indicating

44 Annual Report on Portfolio Performance FY7 28 that many of the smaller projects may not be receiving appropriate attention during preparation and supervision. Fragile States and core IDA countries are the segments with the highest risk and with the biggest disparity between reported problem projects and actual performance. It should be noted, however, that using IEG results as a basis for measuring may not sufficiently take into account improvements taking place recently in some countries. Table 3.5: Reported Problem Projects vs. Actual Unsatisfactory Projects As of November 15,27 ISR Latest 1, Evaluations Client Group % Unsatisfactory % Realism % Problem Projects % Problem Projects % Unsatisfactory Projects by Index bv Number bv Commitments Proiects bv Number Commitments IBRD IDA Core IDA Fragile States Bank-wide The current portfolio monitoring system rates projects entering the portfolio at zero risk, though the project documents provide an assessment of project risk at appraisal. The current portfolio monitoring system flags project risk only when performance indicators show that problems are surfacing. There is merit to reporting riskiness of projects when these enter the portfolio to ensure that supervision strategies are appropriate to the nature of risks. This would focus both management and team attention on high risk projects even before problems arise. There is a case for a review of how risk is defined and reported in the context of monitoring project implementation On July 1, 27, the Guidance Note on Management Review of Investment Lending established new arrangements to foster a more systematic and comprehensive assessment of risks. The Note creates three management review tracks to ensure that the level of management review is aligned with the level of risks involved so that decisions are taken at an appropriate level. Improvements in the risk assessment tools are expected to have a positive impact on project design. However, this should be sustained by appropriate Management oversight of risks during implementation to eventually have an impact on project outcomes A standard risk identipcation worksheet facilitates the risk review process. The worksheet includes a checklist of risk categories to be considered, a description of the risks, the mitigation measures, and the ratings of risks before and after mitigation. The individual risks fall under two categories - country/sector level and operation-specific - with an overall risk rating based on relative importance of individual risks. The Project Concept Note Meeting decides whether a Risk Assessment Review prior to the Decision Meeting for Appraisal Authorization is required. Investment operations with substantial and high risk characteristics will be subject to ROC or OC review for Decision Meeting. 3.3 The measurement and monitoring of portfolio risks should be revised by building on the changes made in the risk assessment matrk at appraisal, First, the current practice of

45 Annual Report on Portfolio Performance FY7 29 classifylng projects as zero risk at the beginning of the supervision period should be discontinued. Second, the portfolio monitoring system should track the risks identified at appraisal, with risk ratings to be changed only when mitigation measures are implemented or events occur that impact the project s risk profile. Finally, there should be a process for systematic review of changes in risk ratings. Proactivity 3.31 The Bank-wide Proactivity Index of 79 percent indicates that teams have been addressing in a satisfactory manner the underlying issues of reported problem projects. Among Regions, SAR and EAP had proactivity ratings of below 7 percent in FY7. Among Sector Boards, Financial Sector, Social Protection, and Transport had lower than 7 percent proactivity ratings. Among client groupings, only Fragile States at 66 percent was below Bankwide average. Focusing on supervision of projects in the above segments would improve overall proactivity A QAG review found that even in cases ofproblem projects not classified as proactive, a majority of teams were adequately addressing the issues underlying the problem status of projects. QAG conducted a review of all problem projects which were not considered proactive during FY7 based on the current definition of what constitutes proactivity. The review found that, in about 6 percent of the cases, the teams were in fact exerting the appropriate efforts to address issues faced by the projects but these efforts were not captured in the Proactivity Index. If the Index were to capture all proactive actions, the Index would show 91 percent proactivity. However, these actions are diverse and not easily measurable and monitorable, though they are expected to eventually lead to one of defined proactivity actions Though the reporting of problem projects showed instances of premature upgrading, teams had been exerting the appropriate efforts. The Realism Review described earlier included a sample of problem projects at end-fy6 that were upgraded during FY7. The review found that in about 7 percent of cases, the upgrading was appropriate. In 3 percent of cases, the upgrading was premature, such as when the upgrading is based on agreements to implement action plans but no concrete actions have yet been taken by the Borrower. Nonetheless, even in cases of premature upgrading, appropriate efforts were being expended to address problems, and at some point, the upgrading would be appropriate. What is critical is that problem projects be identified as early as possible to enable teams to address underlying problems and for Management to provide adequate attention and resources in a timely manner At this stage, the value added of making changes to the Proactivity Index does not seem to justih the cost. Several options were considered to improve the precision of the Proactivity Index, including adding to the list of proactive actions and extending the proactivity period from 12 to 18 months. The feedback from the Regions was that the gains from a change in the measurement did not justifl the possible added complexity as well as the risk of certain perverse behavior arising from extending the proactivity period. The overarching conclusion from the 29 The management target is Proactivity Index at 8 percent.

46 Annual ReDort on Portfolio Performance FY7 3 reviews was that projects rated as problems in the ISRs are likely to receive adequate attention, resources and efforts towards addressing the underlying issues. The focus should be on ensuring that teams identify problem projects in a timely manner and report these in the ISR, i.e., improving the Realism Index. Effectiveness Delays 3.35 As of end-fyo7, 41 projects were awaiting effectiveness beyond the benchmark number of months from Board appr~val.~' This number is double that of a year ago. QAG performed a desk review of the 41 projects to identify the reasons behind the effectiveness delays. The review classified the reasons into three groups: (a) delays in country approvals, by both the Government as well Parliament; (b) number of special effectiveness conditions; and (c) country conditions beyond the control of Borrower and the Bank. About 6 percent of the delays were attributed to client approval delays, and about 16 percent each to number of effectiveness conditions and to country conditions. The average number of special effectiveness conditions was about two, although this varied by region with the AFR sample having 3.4 and LCR having less than one Two actions could help reduce Delayed Effectiveness. First, the number of special conditions of effectiveness could be reduced by only taking operations to the Board for approval when they are ready for implementation; some Regions, such as LCR, are already doing this as a matter of practice. Second, Regions--with the help of Legal--could find ways of addressing the problem of lengthy Parliamentary ratifications. MNA has had some success in the case of Egypt. LCR is currently doing a review to identify ways to accelerate Client approvals. Quality-at-Entry 3.37 The results of QEAS show that Quality-at-Entry of Bank projects approved in FY6 and FY7 meets the 9 percent benchmark This represents sustained performance over a ten year period. The QEA8 results continue to show convergence of overall regional and network performance. The analysis of the S+ ratings point to three dimensions where there is scope for improvement (see Figure 3.6): (a) Implementation Arrangements, specifically readiness for implementation and M&E arrangements; (b) Risk Assessment, especially quality of risk mitigation measures; and (c) Bank Inputs and Processes, focusing on management oversight and peer review process QEAS results indicate that generally operations paid attention to governance and accountability issues. QEA8 rated governance and accountability at 98 percent Moderately Satisfactory or better. These results indicate a good start to the implementation of the Bank's Governance and Anti-corruption Strategy. However, a quarter of the sample had scope for improvement, especially on mitigation measures to address identified governance risks, pointing towards scope for further improvement. In addition, supervision should pay close attention to governance risks. 3 The benchmarks are three months for Ems, six months for DPLs, and nine months for Investment Loans.

47 Annual ReDort on Portfolio Performance FY7 31 Figure 3.6: QEA8 Results by Quality Dimension Rl RI=Strategic Relevance and Approach R2 =Technical, Financial and Economic Aspects R3 = Poverty, Gender and Social Development R4 = Environmental Aspects R5= FiduciaryAspects R6 =Policyand Institutional Aspects R7= knplementation Arrangements R8 = Risk Assessment R9 = Bank Inputs and Processes % Moderately Satisfactory or Better %Satisfactory or Better 3.39 QAGperformed a study to determine the predictive value of QEA results with respect to eventual project outcomes. The review found that when QEA had rated a project satisfactory at entry, its chances of being unsatisfactory at outcome had been on average about 2 percent, compared to twice or 4 percent for a project rated by QEA as unsatisfactory at entry. 3.4 QAG is tightening current QEA methodology to reduce the disconnect between QAG ratings andproject outcomes. A recent QAG review found two main reasons for the disconnect. First, there was an underestimation of risks by the panels. This was partly ascribed to the absence, until lately, of internal Bank standards in the area of risk management. Second, there were some inconsistencies between written comments and ratings in specific questions. In addition, there was lack of clarity as to how different aspects of quality affect overall ratings. The review recommended sharpening the focus of the QEA process on whether development objectives would be achieved and strengthening several aspects of the questionnaire including in the areas of risk assessment, results fkamework, and realism of objectives Most of the 31 recommendations from QEA4-7 have been acted upon. The specific recommendations with unambiguous accountability for action were generally implemented in a satisfactory manner. OPCS related actions have focused on clarifying and improving processes, policies, and systems including revisions to the PAD format, guidelines for emergency operations, and clarifying DPO policies. QAG actions were mainly modifications to assessment methodologies, such as refining questionnaire for DPOs. Where the QEAs identified Regionspecific actions such as the QEA6 recommendation to LCR on improving quality of adjustment loans, Regions have responded favorably. The same is true for Sector Boards--the more targeted the recommendation, the greater probability of action being undertaken Actions on three recommendations from QEA4-7 have not yet produced satisfactory results. QEA5 suggested a review of impediments to the ability of Sector Managers to assume a greater role in quality enhancement. QEA8 rated the follow-up to this recommendation as Moderately Satisfactory.

48 Annual ReDort on Portfolio Performance FY7 32 QEA6 asked Senior Management to address managerial effectiveness and incentives to ensure greater realism in setting Development Objectives and greater candor in assessing project risks. QEA8 rated progress in the area concerning greater candor in assessing project risks as Unsatisfactory. QEA5 recommended that Management address the issue of lack of adequate implementation readiness of IDA projects. Both QEA8 (see Para 3.37) and QSA73' found implementation readiness to be a continuing issue. Quality of Supervision 3.43 Quality of Bank supervision has been sustained at 9 percent satisfactory or better since FYOO. Unlike in previous QSAs when there were differences in performance among Regions and Networks, the QSA7 results showed a convergence of performance. Nonetheless, about 2 percent of the QSA7 sample had an unsatisfactory rating in at least one of the four dimensions, mainly in Candor and Quality of ISR. In addition, there was a significant increase in the Moderately Satisfactory category compared to QSA6, indicative of growing missed opportunities. Finally, about 2 percent of the sample were rated as having inadequate supervision budgets QSA7 found that generally, adequate resources were provided for supervision but found that there may be scope for improved allocation. Reported problem projects receive about 3 percent more resources than Bank-wide average. Nonetheless, there may be scope for improving the allocation mechanisms to ensure that potential problem and risky projects receive appropriate funding. There are different practices among Regions in the allocation of supervision resources. AFR sets aside escrow funding for supervision and restructuring of problem operations as reported in the ISRs, with funds disbursed based on an agreed set of actions to be implemented. LCR has a Risk Review process which identifies the projects at risk, and based on this review, additional resources are provided to the Country Unit. Other Regions leave the allocation decision to the Country Units. It is unclear which practice achieves the best results There was no significant difference in supervision quality between projects managed from HQ and those managed from the field - both were rated highly. Panels noted the need for better integration of decentralized or field-based fiduciary staff in supervision teams. They also indicated that more could be done to complement field-based TTLs with appropriate use of HQ staff with global experience. Finally, there are challenges to Management oversight where the Sector Manager is in HQ and the TTL is in the field OPCS, Regions, and Sector Boards have been acting on specific recommendations in QSA4-7. The OPCS actions were in the areas of facilitating project restructuring, building staff capacity for M&E, and training on results framework and general project supervision. QAG 31 QSA7 rated Readiness for Implementation at Approval at 7 percent MS+ and 37 percent S+, much lower than the overall QSA7 rating of 95 percent MS+ and 52 percent S+.

49 Annual ReDort on Portfolio Performance FY7 33 restructured the Realism Index and tightened the QEA8 methodology to give greater importance to certain quality at entry issues such as results framework and readiness for implementation. Regional action has generally been satisfactory. The relevant Sector Boards have implemented recommendations to improve fiduciary and safeguard compliance Improving candor of reporting was a recurring recommendation in QSA4-7 and has yet to yield satisfactory results. Candor of reporting was the weakest dimension in QSA5-7 though there was a slight improvement in QSA7. Realism of project performance ratings continues to be poor. The underlying problems of lack of candor in reporting have not been addressed in a way that produces results. C. RECOMMENDATIONS AFR to sustain efforts to improve project outcomes, focusing on projects in Fragile States and in the HD, PREM and ESSD Networks. A review of older projects in the portfolio would provide a realistic assessment of the projected outcomes of exits during the next two to three years. The next QSA should assess the quality of AFR portfolio segmented by project age and network. HNP, PSD, and PSG Sector Boards, in coordination with the Regions, to address relatively lower project outcomes focusing on improving project design and addressing development effectiveness issues in a timely manner during supervision. Future QAG assessments should also focus on these three Sector Boards to review progress. QAG and OPCS to review framework for measuring and monitoring risk in the portfolio by establishing a stronger link with risk assessment at appraisal. Portfolio monitoring should report on projects at risk both in terms of number of projects and commitments. Management to address: (a) impediments to greater effectiveness of Sector Managers in quality enhancement; (b) issue of continued lack of realism in reporting project performance; and (c) inadequate implementation readiness of projects.

50 Annual ReDort on Portfolio Performance FY7 34 IV. ANALYTIC AND ADVISORY ACTIVITIES 4.1 Analytic and Advisory Activities (AAA) include both Economic and Sector Work (ESW)32 and Non-lending Technical Assistance (NLTA) and are a key component of the Bank s tool kit for promoting economic development and reducing poverty among its client countries. ESW underpins the Bank s policy dialogue with clients, the development of country assistance strategies, and the design of lending programs. NLTA is an important instrument for helping clients implement reforms and strengthen institutions. This Chapter reports on the trends in the Bank s AAA work and the quality of both AAA tasks and country AAA programs, and makes recommendations for further improvement. It also draws from the key findings of two recently completed QAG assessments: Quality of Country AAA and Managing AAA Droppage. The main messages from this Chapter are: The importance of AAA as an instrument of Bank support has been growing. This is reflected in the increasing share of AAA expenditures in the country services envelope - AAA expenditures in FY7 were 25 percent higher than that of lending preparation and about 95 percent of supervision costs. Several trends in the size and composition of AAA are worth noting. First, there has been a major shift in AAA towards NLTA both in terms o f number as well as value of tasks. Second, within ESW, there is a shift from core diagnostic reports to other types of reports. Third, multi-country AAA is a growing segment; AFR and LCR accounted for more than half of regional AAA delivered in terms of cost in FY7. There has been deterioration in the efficiency of delivery of AAA tasks. Average preparation cost in real terms and preparation time for ESW tasks have increased by 45 and 35 percent, respectively, during the past five years. For NLTA, the increases in average preparation cost and time have been due mainly to the growing number of multi-country tasks; the efficiency of delivery of country level tasks has been stable over the past five years. QAG also conducted two assessments of AAA programs during FY3-7 covering a total of 53 countries. The combined findings from the assessments show a mixed picture. On the positive side, the overall quality of AAA programs was rated as satisfactory with strong aspects in Internal Quality as well as Scope and Strategic Relevance. But several weaknesses were identified. In particular, the low rating for Likely Impact is a major concern - the Bank may be producing good quality tasks which, when viewed in the overall strategic country context, may not be generating sufficient value added. In addition, management oversight was found to be a critical factor that distinguished high and low quality AAA products. 32 ESW is defined as an activity that: (a) involves analytic effort; (b) is undertaken with the intent of influencing an external client s policies andor programs; and (c) is owned by a specific Bank unit. OPCS provided in July 24 a decision tree to help task managers in determining whether a task meets the ESW criteria and in properly coding and recording ESW.

51 Annual Report on Portfolio Performance FY7 35 QAG carried out a review of dropped AAA tasks as a basis for identifying systemic recommendations. Two key findings emerged from the review. First, institutional mechanisms for planning, monitoring, and tracking AAA tasks remain problematic with many dropped tasks suffering from classification errors and lack of managerial attention. The process problems are particularly acute for AAA tasks that are programmatic, multi-sectoral or globalhegional in nature. Second, the fact that a task was not delivered to the client does not mean that it had no valued added - either in itselfor indirectly for other tasks orprojects. Indeed, a large majority of the tasks in the sample appear to have produced acceptable results. There were, however, missed opportunities; with better management of such tasks, the outcomes could have been significantly better. Growth of AAA A. TRENDS IN EXPENDITURES AND DELIVERIES MO2 Roject FY7 1 I, 4.3 There has been a major shijl in the composition of AM towards NTLA activities (see Table 4.1). Whether viewed in terms of number of tasks delivered, cost of tasks delivered, annual expenditures, or number and cost of tasks in progress, the share of NLTA in total AAA has been increasing over the past five years as country units increased their support to clients in the areas of policy implementation and capacity building. In FY7, the share of NLTA in AAA

52 Annual Report on Portfolio Performance FY7 36 expenditures and cost of tasks delivered was about 45 percent, up from 35 percent in FY3. The share of NLTA in cost of tasks in progress was about 6 percent at end-fy7 indicating continued growth of NLTA activities in FY8. A. Tasks in Progress at the Begining of the Year Table 4.1: AAA Programs by Number and Cost (FYO3-7) AAA Program By No. of Tasks (Bank-wide) FY3 FY4 FY5 FY6 FY7 ESW TA Total ESW TA Total ESW TA Total ESW TA Total ESW TA Total , , , , ,41 B. Tasks Initiated in FY , , , , ,8E D. Tasks in Progress at the I End of the Year (A+B-C) C. Tasks Delivered in FY , , , A. Tasks in Progress at the Begining of the Year , , , , ,111 AAA Program By Cost (Bank-wide) (US$ Million in Real Terms) FY3 FY4 FY5 FY6 FY7 ESW TA Total ESW TA Total ESW TA Total ESW TA Total ESW TA Total B. Total Expenses in FY C. Tasks Delivered in FY I D.ExpensesofTasks Delivered in Previous FYs E. Tasks in Progress at the End I of the Year (A+B-C-D) I Note: Cost of tasks delivered includes previous years expenditures. Efficiency 4.4 Unit cost of ESW delivered has increased by about 45 percent in real terms over the past fzve years (see Figure 4.2). One explanation is the consolidation of certain tasks, though this was mainly happening with respect to fiduciary assessments with the integration of the CFAA and CPAR into the IFA; there have been efficiency gains in the delivery of fiduciary assessments. However, unit cost of non-fiduciary ESW has been increasing; in particular, unit costs of Country Economic Memorandums (CEM) and Country Advisory Reports have increased by more than 6 percent in real terms during the past five years. With respect to NLTA, unit cost of tasks has increased by about 33 percent in real terms during the past five years. This is explained by the increases in multi-country TA programs in AFR and large SDV initiatives in Indonesia.

53 Annual Report on Portfolio Performance FY7 37 Figure 4.2: Unit Cost in Real Terms of Dropped AAA Tasks (FyO3-7) 3 25 P tt) U.- U C 3 I"", FY3 FY4 FY5 FY6 FY7 I +ESW +TA +Total I 4.5 As percent of cost of task delivered, post delivery expenditures increased from 8percent to 1 percent during the pastfive years. In the case of ESW, the increase was from 7 percent to 12 percent. Post delivery expenditures find various activities occurring after the delivery of a task to client, e.g., output finalization, translation of documents, dissemination of findings, and in some instances, field visits. In part, the increases in post delivery expenditures are in response to the recommendations of various QAG assessments for improved dialogue and dissemination. Nonetheless, given the growing size of post delivery expenditures, there needs to be better monitoring of their uses as well as evaluation of likely impact and effectiveness. 4.6 Preparation time for AAA tasks increased during the pastfive years (see Figure 4.3). In the case of ESW, preparation time increased from 12 months to 18 months during the past five years. About a quarter of the ESW tasks delivered in FY7 took more than two years to prepare, compared to five percent in FY3. In the case of NLTA, preparation time saw a major increase in FY7, during which a quarter of tasks delivered had at least two years of preparation time. This is mainly explained by the rising number of multi-country and other large NLTA tasks that required longer preparation time. Eighty-five tasks delivered with per unit cost above $3k took between 3-48 months to complete; these tasks accounted for about a fifth of FY7 deliveries. Increases in preparation time for both ESW and NLTA require greater management attention.

54 Annual Report on Portfolio Performance FY7 38 E 2 i= la s- -2 c 2 16 m s Figure 4.3: Preparation Time of AAA Deliveries (FYO3-7) kg Q Instruments ESW 4.7 In FYOl, the Bank launched a major effort to increase coverage of core diagnostics with a target of full coverage33 of active clients. By end-fy4, most Bank clients had up-todate core diagnostic products, with full coverage in all Regions except AFR (due to gaps in PAS and CEMs in Fragile States) and MNA (due to gaps mainly in Fiduciary Assessments). Starting FY5, frequency of diagnostic reports is programmed on a country by country basis, depending on the types and level of Bank engagement and partner country priorities and circumstances, and the availability of relevant knowledge from development partners. As of end-fy7, core diagnostic coverage has improved further, especially with respect to CEMs and PAS (see Table 4.2). Region Table 4.2: Core Diagnostic Reports Coverage (NO4 vs. FY7) (Number of Countries) CEMlDPR Fiduciary PER PA Total FY4 FY7 FY4 FY7 FY4 FY7 FY4 FY7 FY4 FY7 AFR EAP ECA LCR M NA SAR Bank-wide 71 a7 91 9a 7a a CEM= Country Economic Memorandum; DPR=Development Policy Review; PER=Public Expenditure Review; PA= Poverty Assessment. 4.8 The number of Core Diagnostic deliveries has been declining in recent years, from 122 in FY4 to 68 in FY7 (see Table 4.3). The number of CEMs delivered in FY7 is less than half of average annual deliveries during FY3-6 and is now at the same level as during FY A country is classified as filly covered if all core diagnostic products are five years old or less.

55 Annual ReDort on Portfolio Performance FY7 39 This development is expected from the FY4 ESW reforms which left to country management the determination of the number and frequency of core diagnostics. In addition, there has been a consolidation of CFAA and CPAR instruments into a single Integrated Fiduciary Assessment (IFA) - 14 IFAs were delivered in FY7 with only four CFAAs and CPARs completed. In the case of PAS and PERs, the number of tasks delivered has remained at about the same level for the past five years. Table 4.3: Number and Unit Cost of ESW Deliveries by Output Type Unit Cost (US$ ') A M Deliveries Deliveries (#) (In Real Terms) FY3 FY4 FY5 FY6 FY7 FY3 FY4 FY5 FY6 FY7 Core Diagnostic Reports PA CEMIDPR PER CFM 3 23 IO CPAR I FA INon-core Diagnostic Reports Advisory Reports Policy Notes lall ESW Products The combined share of Non-core Diagnostic Reports and Country Advisory Reports has been increasing during the pastfive years and now accounts for 65percent in number and cost of ESW deliveries. Unit cost of delivering country advisory reports has been increasing and is now higher than that of non-core diagnostic reports. Further analysis is needed to determine why this is so. During the recently completed C-AAA Assessment, Non-core Diagnostic Reports and Country Advisory Reports scored much lower than Core Diagnostic Reports. QAG is preparing a special review of Non-core Diagnostic Reports and Country Advisory Reports to understand better the reasons for the lower quality ratings for these two product lines. 4.1 while the number and unit cost in real terms of Policy Notes have declined since FY5, preparation time has increased. Policy Notes are meant to be quick response, focused, and short pieces. The number of Policy Notes delivered more than doubled from 75 in FYOO to a high of 193 in FY5. During this period, preparation time and cost increased and became comparable with those of Non-core Diagnostic Reports and Country Advisory Reports, hence negating the intent of Policy Notes to provide just-in-time advice. Since FY5, there has been a decline in the number and per unit cost in real terms of Policy Notes delivered, though per unit cost in FY7 was about 35 percent higher in real terms than five years ago. However, preparation time has lengthened from 12 months in FY5 to 15 months in FY7. There may be need to review the efficiency and impact that Policy Notes have had on client countries.

56 Annual Report on Portfolio Performance FY7 4 Non-Len ding Technical Assistance FY7 NLTA deliveries were at an all time high in terms of cost, almost doubling the level in FY6. About 45 percent of the cost of FY7 NLTA deliveries were for multi-country or regional initiatives. Trust Fund spending doubled in FY7, mainly to deliver high cost NLTA. For example, an AFR regional transport NLTA cost almost $14 million, of which $1 1 million were from Trust Funds. In Indonesia, the total cost of tasks delivered increased from $1.2 million in FY6 to over $13 million in FY7 mainly due to large SDV tasks financed from Trust Funds The recent QAG Country AAA Assessment found good value for money in cases where NTLA resources were used to implement ESW recommendations. To further improve effectiveness of NTLA, the assessment recommends: (a) budgetary support for NLTA and staff flexibility should become a standard feature of the future structure of AAA programs in order to help follow-through ESW recommendations; and (b) expert assistance in short-term technical cooperation for just-in-time advice should be elevated to the status of ESW in terms of staff incentives. Global Programs and Partnerships 4.13 Global Programs and Partnerships (GPPs) are now a prominent part of the Bank s work The Development Grant Facility (DGF), the Bank s primary source of financial support to GPPs, is a key instrument for the Bank to engage partners on high priority initiatives that complement Bank country programs. In FY7, the Bank was involved in about 17 GPPs to varying degrees-56 were funded out of a DGF budget of $172 million. DGF disbursements have been at about $17 million per year since FY4. The two window funding approach was introduced in FY2 to distinguish between programs tackling long term development challenges (Window One) and those meriting support for up to three years (Window Two). In FY7, there were 17 programs under Window One and 35 under Window 4.14 A QAG review (GPP-2) of six out of eight new Window Two programs approved for FY7 indicates improved quality-at-entry compared to the results of the review of FY6 Window Two programs (GPP-I). GPP-2 found the FY7 initiatives to be highly relevant, reflecting international consensus with strong alignment with the Bank s mission and sector strategies. The Bank utilized its convening power, knowledge base, and country experience to bring together partners and stakeholders to own and participate in programs and partnerships that promote collective actions on important issues. Advocacy and dissemination of best practices were major activities of the GPPs reviewed. The GPPs evaluated supported directly or indirectly the MDGs, and contributed to global public goods GPP-2 identi$ed results framework, risk assessment, disengagement stratem, and linkage to regional strategies as areas to be strengthened. Most of the programs reviewed 34 These do not include the Special Programs (Partnership for African Capacity Building) and the Institutional Grant Programs (Post Conflict Program, Small Grants Program, and Institutional Development Fund) which are also financed from DGF.

57 Annual Reuort on Portfolio Performance FY7 41 lacked an acceptable results framework including appropriate M&E arrangements. This has led to overly ambitious objectives for the GPPs. Panels noted that GPPs generally took on risks that could not be undertaken by typical Bank lending operations, such as piloting innovative approaches - this was viewed to be a major value added of GPPs. However, panels recommended a more systematic assessment of risks and rewards, including the establishment of clear criteria for evaluating risks. The quality of assessment of the impact on sustainability of the GPPs of completion of Bank funding was uneven. Finally, better alignment with regional strategies would help maximize the benefits of GPPs. C-AAA Assessment B. QUALITYOFAAA 4.16 Since FY3, QAG has been utilizing a country approach in AAA assessments where both the country AAA program and a sample of tasks within the country program are assessed. The Country AAA assessments in FY5 and FY7 covered a total of 53 countries, 36 in Phase I and 17 in Phase 11. The two assessments combined covered 485 sampled tasks -- representative from all six regions, IBRD and IDA borrowers, and different country settings. The analysis of the combined sample is statistically robust The combined analysis from the two phases presents a decidedly mixed picture of quality of AAA. On the positive side, although only 83 percent of C-AAA programs are rated Moderately Satisfactory or better, the weighted average by expenditure is 88 percent, which is close to the Bank goal of 9 percent. All this provides reassurance that the Bank is doing satisfactory quality work under the C-AAA programs for most of the expenditures But there are also significant issues and weaknesses. One in six countries reviewed has an unsatisfactory performance. Bank-wide averages for quality fall short of goals in 3 out of 6 quality dimensions: Dialogue and Dissemination, Coherence and Integration, and Bank Inputs and Processes. Lower ratings for Dialogue and Dissemination reflect that planning and provision for broader AAA dialogue and dissemination are too often neglected and AAA is not generally disseminated beyond Bank counterparts. A shortfall in Coherence and Integration arises because Country Teams are not pursuing adequately inter-linkages among tasks and seeking to take advantage of synergies that may arise. Lower ratings for Bank Inputs and Processes reflect insufficient managerial attention to quality, inadequate monitoring and evaluation, and poor quality of information in BW, SAP, and ACS. Weak ratings in these three quality dimensions are contributory factors to the low ratings of the Likely Impact dimension There are also variations in quality in both Regions and Networks35 (see Table 4.4). There are considerable regional variations around the average ratings - performance in AFR and MNA are of particular concern. Both regions also have relatively lower ratings on the three quality dimensions that are low Bank-wide. There are also some variations at the network level 35 The definition of network was based on the structure at the time the assessments were performed.

58 Annual Report on Portfolio Performance FY7 42 with FSD, INF, OPCS, and PREM above the Bank s goal and HDN, ESSD, and PSDN at the lower end, with the last three networks having relatively lower ratings on Likely Impact. Region Table 4.4: Quality of Country AAA Programs (Percent Moderately Satisfactory or Better) Total Countries OA R1 R2 R3 R4 R5 R6 AFR EAP ECA LCR MNA SAR Bank-wide OA=Overall Assessment; R1= Strategic Relevance; ==Internal Quality; IU=Dialogue and Dissemination; R4=Coherence and Integration; RS=Likely Impact; R6=Bank Inputs and Processes. 4.2 Many programs may not be generating sufficient value added. A major concern from the C-AAA assessment is the low rating - 75 percent MS+ - for Likely Impact, which falls well short of the Bank s goal. Thirteen of the 53 programs are assessed to have Likely Impact rated Moderately Unsatisfactorily or below. Lower ratings for Dialogue and Dissemination and Coherence and Integration are closely linked to lower ratings for Likely Impact on the client and are in turn explained largely by inadequate management attention to quality as captured by Bank Inputs and Processes. Apparently, the Bank is producing good products that are not necessarily finding resonance in client countries, raising questions about the value of a significant part of the program. The inclusion of a clear results framework for AAA tasks and programs would help define outcomes and impact expected from AAA Strong management oversight from program inception through completion is the single most important distinguishing factor between C-AAA programs rated high and those rated low or average quality. This finding is consistent throughout all assessments of Phases I and 11. Management oversight is weak in regions rated lower in quality. Low CPIA countries appear to also fare poorly in management oversight. Management oversight is better at entry and lower during implementation Generally low quality of information and weak Bank institutional memory exacerbate managerialproblems. In particular, the quality of information in BW, SAP, and ACS is low. Recording still leaves much to be desired. Despite significant cumulative investment in AAA in virtually all countries, a significant number of country teams do not consider country knowledge to be adequate The combined Phase I and 11 C-AAA Assessment identified Dialogue and Dissemination as the other key area that needs the most improvement. Ratings for Dialogue and Dissemination, which reflect the effectiveness of the effort by the Bank to reach consensus around the recommendations of AAA work, are also low for most of the regions. The key

59 Annual ReDort on Portfolio Performance FY7 43 weaknesses identified in country program rated below the line were a lack of a well articulated strategy for broad dissemination and insufficient dissemination efforts of the findings, a less than clear formulation of conclusions and recommendations, inadequate consultation with the government and particularly with other stakeholders, and not having the documents translated into local languages. As noted earlier, quality of Dialogue and Dissemination has a direct bearing on the Likely Impact of the taswprogram Within ESW, there were some quality differences among outputs (see Figure 4.4). Core diagnostic reports (CEM, DPR, CFAA, CPAR, IFA, PA, PER) were on the whole rated higher than other tasks. Policy Notes, Non-core Diagnostic Reports, and Advisory Reports were rated lower than Core Diagnostic Reports. Advisory Reports, in particular, were rated lower both in Overall Quality and Likely Impact. Figure 4.4: Quality of ESW Product Types (Percent Moderately Satisfactory or Better) Core Consultations1 Policy Notes Other Advisory Total ESW Diagnostic Country Diagnostic Reports Reports Dialogue Reports 4.25 The overall quality of NLTA at 95 percent MS+ was rated higher than ESW. NLTA scored above the Bank s goal in all quality dimensions, including 1 percent in Dialogue and Dissemination. These results follow the findings of an FY4 QAG assessment of NLTA which found that 99 percent were Satisfactory with a high number of Highly Satisfactory ratings. The report found that close client involvement and solid technical inputs resulted in both high quality content generation and significant knowledge transfer The C-AAA Assessment found significant scope for improving quality of AAA, specifically: Ensuring that the various tasks add up to a coherent strategic agenda for the country and requiring stocktaking of previous work by the Bank and others as a first step in any Bank task; Providing greater receptivity to clients views on priority issues at the time of AAA program formulation, together with encouraging greater client participation at the Concept Note and implementation stages;

60 Annual Report on Portfolio Performance FY7 44 Having a well articulated dissemination strategy up-fkont, with a specific budget allocation for this purpose in the context of broader dissemination beyond government circles; the strategy should include improving ex-ante incentives for more effective dissemination and building coalitions for change through effective dissemination; Holding Country Directors accountable for making strategic AAA choices and Sector DirectorslManagers for task implementation; Conducting annual C-AAA portfolio reviews, which may be done as part of the CPPR process, with a view to making mid-course corrections and increasing attention to AAA in low CPIA countries; and Strengthening the current information systems for improved recording and reporting of AAA tasks. Review of Dropped Tasks 4.27 Cost of dropped tasks increased compared tofive years ago (Figure 4.5). In FY7, there was a clean-up of AAA tasks which resulted in a large number of inactive tasks being dropped during the fiscal year. About 8 percent of cost of dropped tasks in FY7 were for inactive tasks where the last expenditure was incurred prior to FY7. Figure 4.5: Total Cost in Real Terms of Dropped AAA Tasks (FYO3-7) FYO 3 FY4 FYO 5 PI 6 FY7 I +TA +EsW +Total In response to the increase in the costs of dropped AAA tasks highlighted in last year s ARPP, QAG carried out a review as a basis for identihing systemic recommendations. The review covered 6 dropped AAA tasks from the FY6-7 cohort and focused on Value for Money and the Quality of Process. Two key findings emerged from the review. First, Value for Money - effective use of resources in relation to benefits - was found to be good overall. Indeed, a large majority of the tasks in the sample are judged to have produced satisfactory results. Second, Quality of Process was found to be poor, often reflecting inadequate institutional processes that hamper monitoring and tracking of AAA. About a third of the tasks assessed by

61 Annual Report on Portfolio Performance FY7 45 QAG were misclassified. The process problems are particularly acute for AAA tasks that are programmatic, multi-sectoral or global/regional in nature Performance among Regions and Networks was found to be uneven. AFR and SDN were at the higher end in terms of Quality of Process. Regions and Networks need to give priority attention to Quality of Process, particularly with regard to the management of AAA at initiation, implementation, and droppage of tasks. The highest scores in Value for Money were recorded for EAP, SAR and AFR; the lowest ones by ECA and LCR. The highest scores for Value for Money in Networks were recorded by SDN and HDN; the lowest one by FPD. Regions and Networks need to also give priority attention to increasing the value added of such tasks. 4.3 The review makes three main recommendations. First, assign a high priority to improving the current state of AAA documentation. Second, provide adequate monitoring to ensure that AAA tasks that are no longer needed would be dropped without delay. Third, review and modify the current AAA architecture, including the Bank s data system, to accommodate the changes that are needed, and formulate guidelines to systematically address the issues identified in dropped tasks. Retrospective of Previous AAA Assessment Recommendations In recent years, three reports have been issued by QAG dealing with the quality of AAA: the 26 Report on the Assessment of Global and Regional AAA (GRAAA), 25 Assessment of Quality of C-AAA, and 25 Assessment of Quality of Non-Lending Technical Assistance. Table 4.5 examines and summarizes progress so far in implementing the key recommendations of all three reports. These key recommendations are grouped under two major headings: (a) program management; and (b) dialogue and dissemination. Table 4.5: Status of Recommendations from AAA Assessments Recommendations 1. Program Management AAA Monitoring and Evaluation Avoid task under-funding. Improve AAA guidelines. Reduce miscoding of AAA tasks. Implementation Progress AAA data quality remains poor and SAP, ACS, and BW oversight continue to require increased attention. Substantial progress has been made by Regions and Networks. Issued new guidelines for coding certain tasks; launched user-friendly ESW and TA portals; simplified the presentation of the ESW and TA decision tree; However, overall architecture is still too complex and not user fhendly. Initial steps have been taken. Unit coordinators are appointed in Regions and Networks to monitor coding. However, coding problems seem to have continued. Coding for Global AAA remains particularly problematic. Rating MU U S MS MU

62 Annual ReDort on Portfolio Performance FY7 46 Recommendations Adoption of CPPR-like process to improve planning and monitoring of AAA program. 2. Dissemination Plan and fimd dissemination as an integral part of AAA. Implementation Progress AAA portfolio reviews only introduced in SAR so far. Funding for dissemination has increased its impact, but remains uncertain. Rating MU MS MU Make available reports in local languages. Engage in broader country consultation and dissemination. Some, but not all reports going to the client are translated into local languages. Receiving increased Managerial attention, but progress is constrained by in-country political sensitivities. MS MU Use of in-country expertise. Overall Good progress achieved by Regions and Networks. S MU C. RECOMMENDATIONS 4.32 It is recommended that OPCS review AAA definitions, guidelines and systems to improve monitoring, measurement, results orientation and governance arrangements to address recurring issues of lack of adequate management oversight.

63 Annual Report on Portfolio Performance FY7 47 V. DEVELOPMENT OUTCOMES OF COUNTRY PROGRAMS 5.1 Notwithstanding gains in aggregate project outcomes and generally satisfactory quality of AAA tasks, the performance of country programs based on completed CAEs during the past ten years is below 6 percent satisfactory. There is an expectation that improving quality of Bank performance during project preparation and supervision and in AAA would result in successful country program outcomes. This chapter analyzes the factors contributing to the relatively low country program outcomes and identifies areas of improvement. The main messages from this Chapter are: A QAG review of CAEs with unsatisfactory ratings found that two factors stand out as primary contributors to low outcomes: (a) weak design of country programs, including poor quality of lending programs; and (b) ineffective country dialogue. Country program design issues include significant omissions, lack of coherence, and failure to take into account borrower commitment and capacity. The QAG review identified three main reasons for the divergence between sector and project outcomes. First, many projects were undertaken which did not address the key development constraints or priorities in the sector. Second, poor coordination led to low outcomes in sectors where cross sector inputs from other Bank projects and donors were critical. Finally, the results frameworks of both projects and CASs did not have strong linkages among project, sector, and CAS outcomes. The above findings point towards a two track approach to improving program outcomes. First, the CAS design, i.e., quality-at-entry, should be improved by ensuring that the program addresses key development constraints, is focused and coherent, and takes into account Borrower commitment and capacity. Second, quality assurance should focus on quality of implementation of CASs not only at the task level but also at the program level to ensure that the timing and choice of interventions maximize impact on CAS outcomes. A. TRENDS IN PROGRAM AND PROJECT OUTCOMES 5.2 The development outcomes at the country program level do not match the results at the project level discussed in Chapter 3. Based on sixty-three CAES~~ completed during FY97-7, IEG rated 35 percent of the Bank country assistance programs unsatisfactory for the entire evaluation period and rated another 2 percent of the programs unsatisfactory for part of the evaluation period. The CAEs covered 62 countries37 representing 75 percent of the end-fy7 portfolio; they also included the top 25 borrowers There were several Country Assistance Notes and Country Assistance Reviews completed, mainly prior to FY97, but these did not have ratings. These were excluded from the discussion in this chapter. One country had two CAEs.

64 Annual Report on Portfolio Performance FY The CAE outcome ratings deviate from the aggregate ofproject outcomes (Figure 5.1). Aggregate project performance is measured using the IEG ratings of project exits in countries covered by CAEs during the evaluation period. Modifying the measurement to include outcomes of projects which were active during the evaluation period3* does not change the overall result. Figure 5.1 : Performance of Country Programs and Related Projects (3 Year Moving Average) P --- E! 9 J! 8.- v) v) 7 E 6 Q) 2 5 al A FY9 FY92 FY94 FY96 FY98 FYOO FY2 FY4 FY6 1 +Country Level Evaluation +Project Level Evaluation I Note: To compute the country level outcome by fiscal year, the appropriate CAE and CASCR ratings were applied to each year covered by the evaluations. The project level outcomes were based on exits of projects during the CAE evaluation period in the countries covered by the CAEs. 5.4 The C-AAA Assessment rated country AAA programs lower than the aggregate of individual tasks ratings. As discussed in Chapter 4, the quality of individual tasks was found to be satisfactory but Coherence and Integration of country AAA programs was rated low at 72 percent MS+ and 43 percent S+. There may be a similar relationship between lending projects and country lending programs. Unlike in the case of AAA, QAG does not follow the country approach to assessments of lending operations. 5.5 Half of the CAEs with unsatisfactory ratings had aggregate project performance of 7 percent satisfactory or better (see Table 5.1).39 The divergence also goes the other direction - about 2 percent of satisfactory CAEs had aggregate project performance of less than 7 percent satisfactory. Overall, about one of three CAEs had a divergence between country program and aggregate project performance using 7 percent or better as the norm for satisfactory project performance IEG evaluations of projects that were active during the evaluation period were used if these projects have exited; otherwise, the latest DO ratings of the ISRs in FY7 were used to measure outcomes. The ARDE 26 used aggregate project performance of 5 percent or better as the comparator in analyzing divergence of programs and project performance. About 8 percent of CAEs with unsatisfactory ratings had aggregate project performance of 5 percent or better.

65 Annual Report on Portfolio Performance FY7 49 Project Portfolio (Aggregate Portfolio Rating) 9% or better 8-89% 849% (Cont d) 7O-79% 649% 549% Less than 5% Table 5.1: Country Program vs. Project Portfolio Performance HS El Salvador (89-) China (93-2) Uruguay (87-99) Vietnam (88-1) Chile (85-) Armenia (93-2) Bosnia (96-3) Maldives (8-98)+ Romania (-4)** Argentina (91-) Tunisia (9-3) Bulgaria (98-1)** Mexico (97-OO)** Brazil (9-2) Croatia (2-3)+** Peru (9-96)** Lithuania (91-2) Ethiopia (9-) Guatemala (9-1)** Mexico (89-91)** Y~TI~II (96-98)** S Ghana (95-99) Russia (99-1)** Uganda (87-99) WBG (93-) 7- CAE Ratin 1 Mongolia (91-1) Albania (98-4) Mexico (92-94)** Eritrea (92-OO)+ Kazakhstan (9-99) Senegal (94-4) Madagascar (94-6) Turkey (93-4) Pakistan (94-3) Bhutan (93-3)+ Yemen (99-5) Burkina Faso (89-99) Egypt (91-) Indonesia (9-98) KyrgyZ (93-) Mexico (95-96)** Dom. Republic (85-2) Bolivia (98-4) Sri Lanka (89-98) Yemen (9-95)** Bolivia (85-96) Lesotho (9-99) Morocco (97-) Pacific Islands (92-2) Cameroon (95-OO)** Rwanda (95-1)** Costa Rica (9-) 1Ecuador (94-98) Moldova (93-3) Peru (97-)** Ukraine (93-98) Croatia (94-1)** Zimbabwe (9-2) Mauritania (92-3) Russia (92-98)+** Zambia (96-1) Paraguay (9-) Guatemala (85-89)+** Haiti (86-1) Papua NG (9-99) + Less than 1 projects evaluated by IEG. 5.6 Effective January I, 25, all CAS S are required to be results based and to submit a CAS Completion Report (CASCR) to be reviewed by IEG. The central innovation of the results based CAS is the design of a framework that specifies explicit linkages between the Bank s interventions and long-term development goals. Such a framework would enable better selectivity and design of Bank activities with a strong focus on results on the ground. The framework would also help Management monitor progress, evaluate success or failure, and make corresponding adjustments in the Bank s lending and non-lending programs. Results based CASs include measurable indicators to monitor program implementation that could capture concrete progress toward the country s and the Bank s core development objectives.

66 Annual Report on Portfolio Performance FY IEG desk reviews of 43 CAS Completion Reports (CASCR) during the period FY2-7 indicate some improvement, IEG rated about 6 percent of program outcomes in the CASCRs as Moderately Satisfactory or better. The impact of the CASCR ratings is an improvement in the country program outcomes starting FY3 as shown in Figure both the CAE and CASCR ratings were included in estimating country level performance. It is unclear whether future CAEs reviewing new CASs would result in a different rating than CASCRs review QAG commissioned a review focusing on 4 CAEs that included all of the 32 CAEs with unsatisfactory ratings for all or part of the evaluation periods. The objective of the review was to identify the factors that contributed to the relatively low ratings of CAEs. In cases where there was a divergence between country program and project outcomes, the review examined the reasons for such divergence. The findings are discussed below. B. FACTORS CONTRIBUTING TO LOW PROGRAM OUTCOMES 5.9 Two factors stood out as primary reasons for the unsatisfactory ratings of the country programs: weak design of country programs including poor quality of lending programs, and lack of effectiveness of country dialogue (Figure 5.2). These are all factors that are within the Bank's control. Other factors-- relevance of the CAS, quality of AAA, quality of donor harmonization and coordination, and impact of exogenous events - did not stand out as major reasons for the unsatisfactory ratings of country programs. 1 Figure 5.2: Factors Contributing to Unsatisfactory CAEs o) 8 c C m 6 m I+ u) E 4 $jg E 2 2; ZL $ f! P Quality of Approp. of Bfec. of Relev. of Quality of Impact of Quality of 8 Lending Country Country CAS AAAProg. Bog. Donor a Prog. Prog. Dialogue Objectives Events Harmonir. S 8 Coord. Importance Factor Rating 6.- Weak Design of Country Program 5.1 There were several weaknesses in the design of country programs in the unsatisfactory CAEs. First, the programs failed to take into account borrower commitment and capacitythis occurred in almost all cases. As a result, the program had unrealistic objectives. For the 4 In the case of IEG project evaluations, Project Performance Assessment Report (PPARs) conducted with field missions about three years after project closing result in lower ratings than Evaluation Summaries based on desk reviews within six months from project closing.

67 Annual ReDort on Portfolio Performance FY7 51 most part, both lack of commitment and poor capacity were present, though there were cases where there was Government commitment but the program was not aligned with Borrower capacity. Several CAEs concluded that the amount of lending was not justified by the level and commitment to reforms. The roots of the problem were the poor quality of the assessment of political economy of reform and the overestimation of Borrower capacity Second, in about half of the unsatisfactory CAEs, there were significant omissions in the program. Several CAEs pointed out that while the CAS objectives were relevant, the program failed to focus on the key development constraints. For example, the Yemen CAE criticized the Bank program during FY9-98 for neglecting issues relating to gender, water resource management, and g~vernance.~~ In some cases, the omission was the lack of emphasis on AAA, especially in countries where reform commitment was weak and lending was constrained. This was the main reason for the unsatisfactory rating of the Russia ( ) and Bulgaria ( ) programs during the initial years of the period covered by the CAEs Third, lack of coherence and focus of country programs contributed to the unsatisfactory ratings. The Honduras program ( ), for example, covered a large number of sectors; greater depth of involvement in critical sectors would have been more appropriate and effective. In the case of Guatemala ( ), the program had limited relevance and weak links to the assistance strategy. The CAE found that the Rwanda program (199-94) was not selective Fourth, many lending programs suffered from weak analytical underpinning and lack of a coherent strategy. This has resulted in project choices that failed to address key development constraints in the sector, and contributed to the lack of sustainability of projects undertaken. In the case of Romania ( ), Private Sector Development was one of the three pillars in the CAS to be supported by significant lending. However, the lending strategy failed to tackle the problem of financial discipline caused by state owned enterprises which the CAE identified as a critical element to enterprise reform. In the case of Ecuador ( ), the societal risks of vested interests and opposition from Congress were underestimated in designing the lending program Fifth, some lending programs failed to establish strong linkages among different projects, and to ensure that projects were undertaken with the appropriate policy framework. This contributed to weak sector outcomes and poor links to CAS objectives (see discussion on sector outcomes below). Lending in the Transport Sector in Croatia during was criticized for moving forward despite policy distortions, with the CAE concluding that Bank lending in the sector exacerbated the already difficult fiscal and external debt problem. A major lesson from several of the satisfactory CAEs reviewed by QAG is the need for strong complementarity among different interventions, such as the appropriate mix of policy based and investment loans in Bulgaria during the period ' The FY3-5 Yemen CAS addressed these issues.

68 Annual ReDort on Portfolio Performance FY Sixth, the design of many individual projects - especially those with major impact on the CAS objectives - was unsatisfactory. For example, in Madagascar (1994-6), the flagship PRSCs were problematic and encountered numerous delays due to weak ministries. In most cases, the main problem with project design was the lack of consistency with Borrower capacity. The Angola CAE (1991-6) noted that the projects were overly complex and failed to address institutional weaknesses, though the CAE also noted that supervision did a creditable job of addressing many of these design problems. The Lesotho CAE (199-99) found that while the projects in some sectors4* had the correct diagnosis and relevant objectives, the project designs overestimated Borrower capacity and failed to take into account political conditions. Effectiveness of Country Dialogue 5.16 Based on the unsatisfactory CAEs reviewed, QA G panelists rated effectiveness of country dialogue unsatisfactory in 8 percent of the cases. There were cases where Bank dialogue was strong in some areas but not in others resulting in uneven performance. For example, in Lesotho (199-99), the Government was a constructive partner in Bank efforts to improve macroeconomic policy and the formulation of post-apartheid economic strategies and programs. However, there were bitter differences on critical agricultural reforms with little progress in rural development and employment creation in the sector Several CAEs noted that effectiveness of country dialogue could have been improved by expanding outreach beyond a limited number of people in government. In Ecuador ( ), country dialogue focused on and was effective with a small elite surrounding each president, but dialogue with the broader civil society was insufficient and this turned out to be an important omission. In Zimbabwe (1 99-2), the Bank had generally good rapport with technocrats but not with politicians and decision makers; only late in the period did the Bank increase its emphasis on engaging a broader group of stakeholders High quality ESW and an empowered field office were factors that would have contributed to effective dialogue. The Russia CAE argued that the Bank should have emphasized ESW in its FY92-98 program, and found that only a small number of Government officials had a good grasp of the Bank s views on reform. In Nepal (199-99), country dialogue improved and Bank program became more aligned with Borrower commitment and capacity when a Country Director was placed in the field. In Yemen, country dialogue improved significantly after the opening of the Bank s Country Office. Sector Outcomes c. DIVERGENCE BETWEEN PROGRAM AND PROJECT OUTCOMES 5.19 Poor sector outcomes explain some of the divergence between program and project outcomes. The overall impact of the Bank s lending program is achieved through individual interventions in various sectors, and these interventions should result in improvements in these 42 The Education sector, whose projects performed well, was a notable exception.

69 Annual Reoort on Portfolio Performance FY7 53 sectors which in turn contribute to the outcomes articulated in the CASs. To achieve the program outcomes which are typically articulated in the CASs in terms of sector or thematic results, there should be a strong link between project and sector outcomes. 5.2 Figure 5.3 provides the sector ratings based on CAEs completed during FYOl-7, and compares these with the aggregate project ratings based on exits during the same period. The sector ratings for FY1-6 are based on IEG studies and the FY7 ratings are based on the QAG review of CAES~~. The results show that the lowest sector outcomes were in Private Sector Development, Public Sector Governance, and Rural Development. These are also the sectors with the greatest divergence between sector and project outcomes. Figure 5.3: Sector and Aggregate Project Ratings (FY1-7 CAEs) c g 8 w-.- v) r;i 5 6 m z 5: 4 c) to 2 ' i2 s ED INF FSE HDN PSD PSG RDV Sector Q Sector Rating Aggregate Project Rating 5.21 There are several reasons for the divergence between sector and aggregate project outcomes in these three sectors. First, as mentioned earlier, many projects did not address the key development constraints in the sector, e.g., the agricultural projects in Yemen. There is a wide range of choices for Bank intervention in these sectors, which makes selectivity not only important but also difficult. Improving the choice of projects and ensuring appropriate design require a strong analytical underpinning to identify key constraints and understand political economy of reforms. This should lead to a clear sector assistance strategy that identifies where the Bank would have maximum impact Second, the above sectors require a high degree of cross sectoral inputs from other Bank projects and coordination with donors. Private sector development and public sector governance are viewed to be two sides of the same coin--bank efforts in these two areas need to be coordinated, e.g., effective regulatory systems to support enhanced competition. The Governance and Anti-Corruption Agenda acknowledges the need to work with donors towards a harmonized approach and effective coordination based on the respective mandates and strengths of different institutions. Rural development has linkages with almost all sectors in the Bank, especially infrastructure and environment. 43 There were five CAEs completed in FY7; all of them were unsatisfactory.

70 Annual Report on Portfolio Performance FY Third, many of the CASs covered by the CAEs reviewed by QAG had a weak results chain linking country and sector outcomes with Bank operations. This weakness has resulted in several CASs adopting unrealistic objectives, but more important it has failed to provide a framework for optimal choice of intervention and project design. The current CASs are required to articulate the results chain linking Bank interventions and donor activities with intermediate outcomes and development impact. It is unclear whether the project reviews and quality assurance perform a rigorous analysis of the project-sector-country causal chain and address the question of whether the project and its design choices maximize contribution to sector outcomes. In the case of MICs, successful projects may not add up to broad sectoral impact attributable to the Bank - the appropriate definition of quality at the program level in MICs may require a fresh look. Methodology as Source of Divergence 5.24 Some divergence between the results of evaluations of country programs and projects is expected since the evaluation methodologies have significant differences. These methodological differences that partly explain the divergence include: Timeframe: project evaluations are based on results at a point in time while the CAE covers a longer period. In several of the CAEs reviewed, projects that were originally evaluated by IEG as satisfactory could not sustain the project outcomes over a longer time period. Coverage: a CAE reviews the coherence of all aspects of Bank work, including lending, AAA, country dialogue, and donor coordination; project evaluation has a much narrower coverage. Omissions: there may be major gaps in the country assistance strategy and program resulting in an unsatisfactory CAE rating (see discussion above); in most cases, these omissions are beyond the control of project management Nonetheless, the results-based CASs create the expectation of a narrowing of divergence between project and program outcomes. Two things need to happen for this to occur: (a) improvements in the CAS design; and (b) strengthening the project-sector-country outcome links. The CAS reforms over the past few years have been addressing the weaknesses of the CAS design by introducing a results framework linking Bank and donor inputs to outcomes. Separately, there have also been initiatives to improve the results framework of projects. The current challenge is to harmonize the two initiatives by ensuring that results frameworks of projects are linked to those of CASs where outcomes are articulated at the thematic or sectoral level.

71 Annual Report on Portfolio Performance FY7 55 D. RECOMMENDATIONS OPCS to review as part of the CAS Retrospective the experience to date with results based CASs focusing on realism of CAS objectives and quality of program design, and the adequacy or effectiveness of current instruments to track implementation of CAS programs. PSD, PSG, and RDV Sector Boards to address the divergence between sector and project outcomes. Regions to review whether CPPRs are being utilized to monitor consistency between the portfolio and CAS objectives. OPCS to define, as part of the CAS Retrospective, the criteria for quality of MIC programs.

72 Annual ReDort on Portfolio Performance FY7 56 VI. RECOMMENDATIONS 6.1 This chapter reviews the QAG recommendations during the past five years in the ARPPs to determine progress in implementing these recommendations. This chapter will identify the actions that are of a longer term nature which will be monitored in future ARPPs. 6.2 The FY2-6 ARPPs contain 44 recommendations, most of which have been implemented There were 26 recommendations on strengthening lending and portfolio managements, ten recommendations to improve AAA, and eight recommendations relating to results. Table 6.1 gives the status of these recommendations as well as performance ratings. As agreed with OPCS, the recommendations on results will be included in the upcoming Results Paper. Table 6.1: Status of FYO2-6 ARPP Recommendations Recommendations I Status Business Processes and Portfolio Reporting: There were 12 recommendations on improving processes and portfolio measurement and reporting. These included introducing several guidelines, reforming the PSR system, revising the Realism Index, and strengthening the review and quality assurance processes. Stafland Management Incentives and Capacities: There were 4 recommendations: address impediments to Sector Management in ensuring quality; address issues in staff and management incentives to improve candor in portfolio reporting; upgrading training; and improving support to management. Other Quality Issues: There were 1 recommendations relating to other quality issues including those targeting certain groups such as Sector Boards and certain aspects such as adequacy of resources. Most of the recommendations have been implemented. One recommendation - integrating RETF in portfolio - is currently under implementation. Rating: Satisfactory Some Regions have begun to address the issue of impediments to sector management in ensuring quality. However, there is lack of systemic response to the issue of candor in project performance reporting, with the Realism Index continuing to be below target. Rating: Moderately Unsatisfactory Most of the recommendations have been implemented with satisfactory results. There are lagging areas, such as outcomes of HNP and Fragile States projects. Rating: Moderately Satisfactory There were 1 recommendations on improving AAA management. These include improving reporting, addressing delays in delivery, improving dialogue and dissemination and improving management oversight. Half of the recommendations have been implemented. Several recommendations dealing with management oversight and dialogue and dissemination have not yet resulted in satisfactory outcomes based on the latest C- AAA Assessment. Rating: Moderately Unsatisfactory

73 Annual Report on Portfolio Performance FY7 57 Table 6.1: Status of FYO2-6 ARPP Recommendations (Cont'd) I Recommendations I Status I There were 8 recommendations on the results framework. Improving the Results Framework I The follow-up to these recommendations will be reported in the upcoming OPCS Results Paper. Rating: Not Rated Overall Rating for Follow-up to Previous ARPP Recommendations: Moderately Satisfactory 6.3 The recommendations from the FY7 ARPP and the uncompleted actions from previous AMPs are consolidated in Table 6.2 below. The pending actions addressing AAA issues are rolled into the FY7 ARPP recommendation on AAA. QAG should provide Senior Management with an interim status of these recommendations in July 28, and report on actions taken and progress towards results in the FY8 ARPP. Recommendation AFR to sustain efforts to improve project outcomes, focusing on older projects and projects in Fragile States, HDN, PREM, and ESSD. Regions to work with HNP, PSD, and PSG Sector Boards to address relatively low project outcomes focusing on improving project design at entry and restructuring problem projects in current portfolio. OPCS and QAG to reform measurement and reporting of project risks by linking risk assessments at entry to risk management during implementation. Intermediate Actions Lending and Portfolio Quality AFR with relevant Sector Boards to review current portfolio and address threats to satisfactory outcomes. Next QSA to report on progress in AFR with respect to targeted portfolio segments. Sector Boards and Regions to agree on role of Sector Boards in the quality assurance process at entry and to review current portfolio and address threats to satisfactory outcomes. Next QSA and QEA to report on performance of HNP, PSD and PSG Sector Board projects. 'ortfolio Measurement and Monitoring OPCS and QAG to review and recommend changes to the current system of project risk measurement and monitoring. Results Narrowing of performance gaps between AFR and rest of the Bank with respect to outcomes of projects in Fragile States, HDN, PREM, and ESSD. Specific targets and timetable to be determined by Management based on outcomes of reviews and QSA. Improved outcomes for projects in the HNP, PSD, and PSG Sector Boards towards convergence with Bank-wide performance. Targets to be determined by Management based on outcomes of reviews. More accurate reporting of portfolio risk starting FY9 to be tracked by QAG.

74 Annual Reoort on Portfolio Performance FY7 58 Recommendation Intermediate Actions Results OPCS to review Regional efforts to address impediments to managerial effectiveness in quality enhancement and lack of candor in reporting project performance. OPCS will monitor the impact of the new Realism Index to see whether any further action on portfolio monitoring was necessary. Attainment of 8 percent Realism Index by FY9. OPCS to restructure AAA definitions, guidelines and systems to improve results orientation, results measurement, and governance arrangements. These initiatives would help address recurring issues of lack of adequate management oversight and weak dialogue and dissemination. OPCS to review AAA definitions, guidelines and systems to improve monitoring, measurement, results orientation and governance arrangements to address recurring issues of lack of management oversight. Next QAG AAA Assessment to determine whether progress has been made in addressing areas for improvement. Regions to improve country program design and ensure that results frameworks of projects and AAA are strongly linked to sector and CAS outcomes. Country Programs OPCS to complete the CAS Retrospective. OPCS to review as part of the CAS Retrospective the adequacy of current instruments that track implementation of CAS programs. OPCS to defme, as part of the CAS Retrospective, the criteria for quality of MIC programs. Results will be measured by improved IEG ratings of country programs. OPCS to define indicators and instruments for tracking effectiveness of implementation of CAS programs.

75 Annual Reuort on Portfolio Performance FY7 59 THE PORTFOLIO - AN OVERVIEW TABLE Annex 1 lpening Balance IBRD IDA TF pprovals in FY IBRD IDA TF Fiscal Year :ancellations in FY IBRD IDA TF xits IBRD IDA TF rrors in reconciliation b1 ORTFOLIO: end-year balance!ea/ c/ lpenlng Balance IBRD dl IDA TF pprovals in FY IBRD d' IDA TF xits IBRD dl IDA TF rrors in reconciliation nd-year Balance '' FY2 FY3 FY4 FYO5 FY6 FY7 Net Commitments ($ M) 1,261 14,577 96,93 94,73 95,479 95,194 69,295 64,741 57,336 52,791 54,39 53,111 37,346 37,86 37,436 39,763 38,92 39,778 1,62 1,976 2,157 2,149 2,267 2,35 19,789 18,729 2,353 22,221 23,94 24,874 11,452 11,231 11,45 13,334 14,135 12,784 8,68 7,283 9,35 8,559 9,446 11, ,881 3,258 1,792 2,92 1,132 1,211 1,557 2,89 1,437 1, , ,682 24,242 2,721 2,81 23,827 18,813 14,32 16,499 14,56 1,734 14,419 1,979 7,252 7,532 6,333 9,21 9,194 7, , ,577 96,93 94,73 95,479 95,194 1, ,5 16,957 97,758 98,559 96,717 1,357 Number of Projects 1,561 1,543 1,516 1,466 1,451 1, ,543 1,516 1,466 1,451 1,468 1,485 Cancellations represent partial reduction in commitments but do not include commitments for projects that exit They therefore reduce commitment amounts but not the number of projects in the portfolio. ' End-year balance may not equal Opening balance plus approvals minus cancellations and exits due to synchronization errors between systems. Real term figures are based on Manufacturer Unit Value (MW) Index. ' The Number of Projects in Business Warehouse for IBRD Source of Funds includes Blend operations.

76 Annual ReDort on Portfolio Performance FY7 6 BASIC PORTFOLIO DEFINITIONS AND DATA SOURCES PORTFOLIO DEFINITIONS Annex 2 1. The portfolio covered by the FY7 ARPP includes all IBRD, IDA, GEF, Montreal Protocol, and Special Financing operations approved through FY7, and excludes those that were completely cancelled and/or closed during the fiscal year. All dollar figures are in nominal terms unless otherwise stated. Manufacturer Unit Value (MUV) Index varied by 19 percent between FY2 and FY7. Terms used in reference to the portfolio include: Portfolio. All loans approved through FY7 excluding those which were closed or completely cancelled prior to the end of the fiscal year. The portfolio includes GEF, IBRD, IDA, Montreal Protocol, and Special Financing operations. The portfolio only includes operations that are active at the end of the fiscal year; Actual Problem Projects. Projects for which Implementation Progress is rated unsatisfactory and/or the Development Objectives are rated as unsatisfactory; Country Client Groupings. Countries are grouped according to the level of their income, size, risk and performance for purposes of portfolio trend analysis. IBRD Investment Grade Countries include countries that have high credit ratings. There are presently 27 countries in this group. The Fragile States country group includes 34 countries with low CPIA ratings. China and India, with populations over one billion each, are in individual categories because of their size. The other three groups are core IBRD, core IDA, and Blend. They are categorized according to IDMIBRD eligibility criteria. Country groupings are mutually exclusive. Therefore, the core IBRD group excludes Investment Grade countries and China. The Blend group excludes India, and the core IDA group excludes Fragile States; Commitments at Risk Commitments at risk of not meeting their development objectives. This includes commitments associated with both actual and potential problem projects; Country Policy and Institutional Assessment (CPIA). The Country Policy and Institutional Assessment is an annual exercise in which country teams provide input to OPCS in order to assess the quality of each borrower's policies and institutions in the areas generally considered to be relevant to economic growth and poverty reduction and effective aid use; Deflator. Where so indicated nominal net commitments have been converted to real terms by using Manufacturers Unit Value (MUV) Index Deflator converted to 27

77 Annual ReDort on Portfolio Performance FY7 61 US$ by using an index of.84 for FY2,.91 for FY3,.97 for FY4,.97 for FY5, and.98 for FY6; Development Objectives (DO). The rating of an operation s DO is based on the likelihood of attaining the development objectives set in the Project Appraisal Document or as formally revised during Implementation. This rating may be satisfactory or unsatisfactory and is the responsibility of the Task Team Leader, who must report on it, at least annually, in the Implementation Status and Results Report. The DO rating takes into account not only implementation progress, but also other factors such as inappropriate design, unforeseeable adverse economic and financial developments, price fluctuations of project outputs, and changes in government policy; Disbursement Ratio. The ratio of disbursements during the fiscal year to the undisbursed balance at the beginning of the fiscal year, investment operations only; Implementation Progress (IP). The IP rating is based on an overall judgment of implementation performance in relation to the benchmarks in the Project Appraisal Document or as formally revised during implementation. The rating is the responsibility of the Task Team Leader, who reports it generally at least once a year in the ISR; Net Commitments. Total commitments net of cancellations for all projects in the portfolio; Net Disconnect. The difference between the percentage of projects rated as unsatisfactory by IEG and the percentage rated by the Regions in the final ISR as unsatisfactory for achieving their development objectives; Potential Problem Projects. Projects which are rated satisfactory on IP and DO but have other risk factors historically associated with unsatisfactory outcomes. The criteria to consider projects as potential problem projects are described below in the Section on Measuring Portfolio Performance; Proactivity Index. The proportion of projects rated as actual problem projects 12 months earlier that have been upgraded, restructured, suspended, closed, or partially (2% plus of commitments) or fully canceled; Projects-at-Risk. Projects at risk of not meeting their development objectives. Projects at risk is the sum of actual problem projects and potential problem projects; Quality-at-Entry Assessment (QEA). A periodic exercise conducted by QAG to measure the Quality-at-Entry o f projects shortly after they are approved by the Board. Quality-at-Entry is a prime determinant of successful development outcomes, and

78 Annual Report on Portfolio Performance FY7 62 deficiencies in design are difficult to correct during Implementation. The foundations of a project are laid during Preparation, before it enters the portfolio. QEAS is the most recent Quality-at-Entry exercise and covered all projects approved by the Board in FY6-FY7; Quality of Supervision Assessment (QSA). A periodic exercise conducted by QAG to measure the quality of supervision for projects, during a specific period. The Quality of Supervision Assessments are real time reviews of overall supervision performance for the previous two years. The assessment focuses on the quality of the supervision of Bank projects and not on the quality of the projects per se. The last QSA7 exercise, covered the end-fy7 portfolio; and Realism Index The ratio of the percentage of Actual Problem Projects in the Portfolio and the percentage of Unsatisfactory IEG Outcomes for the most recent project completions (1, evaluations), on a rolling basis. MEASUFUNG PORTFOLIO PERFORMANCE 2. Experience shows that IP and DO ratings have tended to be over-optimistic when compared to the outcomes ratings that projects are given by IEG upon completion. To address this deficiency, the FY96 ARPP introduced the concept of projects at risk as the basic measure of portfolio performance. 3. Projects at risk include both actual and potential problem projects. Potential problem projects are those that, although rated as satisfactory for both IP and DO, are affected by factors likely to bring about an eventual unsatisfactory outcome. These projects are identified by criteria ( flags ) that take into account not only various aspects of actual implementation experience, but also other relevant factors such as economic management and past portfolio performance in the country. Specifically, potential problem projects are identified as projects exhibiting three or more of the following twelve risk flags for investment projects: Legal Covenants. Any of the Critical Legal Covenants rated Not Complied with in the last ISR; Safeguards. Ratings of MU, U or HU on any Applicable Safeguard Policy in the last ISR; Counterpart Funds. Counterpart Funding rated MU, U or HU in the last ISR (formerly the Financial Performance Flag); Monitoring and Evaluation (M&E). Monitoring and Evaluation rated MU, U or HU in the last ISR; Financial Management. Financial Management rated MU, U or HU in the last ISR;

79 Annual Report on Portfolio Performance FY7 63 Procurement. Procurement rated MU, U or HU in the last ISR; Project Management. Project Management rated MU, U or HU in the last ISR; Long-Term Risk Project with IP or DO rated MU, U or HU for any 24 months cumulative during the life of the project. This flag is removed when the project has been rated MS, S, or HS for IP and DO for the previous 24 months; Effectiveness Delay. Elapsed time between Board approval and effectiveness of more than nine months for investment and more than three months for emergency operations. This flag is turned off three years after Board approval; Disbursement Delay. Disbursement delay of 24 months or more for investment and 6 months or more for emergency operations. Delay is calculated based on the initial or formally revised disbursement schedule for the project; Country Environment. Located in a country with weak economic management (CPIA rating of less than 3. on a scale of 1 to 6). Once "flagged," the CPIA rating must exceed 3.5 for the flag to be removed. This flag also includes countries which are in a conflict or post-conflict environment; and Country Record. Located in a country with a net disconnect of 2 percent or more, or where net commitments associated with unsatisfactory projects (as rated by IEG) represent more than 4 percent of commitments for completed projects over the previous five years. In cases where the sample of IEG evaluations is too small, ICR data, data on mature projects, and experience of other donors is used to arrive at a robust conclusion. This flag also captures countries with less than Moderately Satisfactory Country Assistance Evaluation (CAE) ratings by IEG in previous five fiscal years. 4. For Development Policy Lending operations, potential problem projects are identified as projects with two or more of the following seven flags (at least one project specific): Monitoring and Evaluation. Monitoring and Evaluation rated MU, U or HU in the last ISR; Project Management. Project Management rated MU, U or HU in the last ISR; Long-term Risk Project with IP or DO rated MU, U or HU for any 24 months cumulative during the life of the project. This flag is removed when the project has been rated MS, S or, HS for IP and DO for the previous 24 months;

80 Annual Reuort on Portfolio Performance FY7 64 Effectiveness Delay. Elapsed time between Board approval and effectiveness of more than six months for policy-based lending. This flag is turned off three years after Board approval; Disbursement Delay. Disbursement delay of 6 months or more for policy-based lending. Delay is calculated based on the initial or formally revised disbursement schedule for the project; Country Environment. Located in a country with weak economic management (CPIA rating of less than 3. on a scale of 1 to 6). Once flagged, the CPIA must exceed 3.5 for the flag to be removed. This flag also includes countries which are in a conflict or post-conflict environment; and Country Record. Located in a country with a net disconnect of 2 percent or more, or where net commitments associated with unsatisfactory projects (as rated by IEG) represent more than 4 percent of commitments for completed projects over the previous five years. In cases where the sample of IEG evaluations is too small, ICR data, data on mature projects and experience of other donors is used to arrive at a robust conclusion. This flag also captures countries with less than Moderately Satisfactory CAE ratings by IEG in previous five fiscal years. 5. The at-risk ratings provide a better picture of the current state of the portfolio than IP/DO ratings taken in isolation, because they are more comprehensive and provide an early warning of potential failures and their causes. 6. Golden Flag. The projects at risk concept, however, is not perfect. It has been noted that some operations that get flagged as risky are subsequently evaluated as Satisfactory because risks have been addressed, and others that are evaluated as unsatisfactory were not captured by the system. To correct for this, the Regions can override the at-risk rating with a thirteenth flag first introduced in FY97--the Golden Flag. In each of the fiscal years FY2, FY3, FY4 and FY6, approximately one percent of the portfolio had the golden flag. A Golden Flag for a project is turned off if the project becomes unsatisfactory for IP or DO, or the total number of at risk flags for that project goes below three for investment and below two for policy-based lending operations. If the project subsequently gets three or more at-risk flags for investment and two or more for policy-based lending operations, a new request and justification for a Golden Flag is required. DATA SOURCES 7. Data for the ARPP Report and Statistical Tables are taken from the Bank s Business Warehouse. The ISR ratings used in the ARPP were frozen by ISG as of June 3, 27. The IEG evaluation ratings from the Bank s Business Warehouse are provided as of January 22, 28. Other data sources include the Loan Accounting System for data on disbursements and cancellations.

81 Annual Report on Portfolio Performance FY Blend operations include both IDA and IBRD. In the ARPP Statistical Tables, number of projects, portfolio status indicators, IEG outcomes and net disconnect for blend operations are included under IBRD. Commitment amounts, however, are included under IDA and IBRD, respectively. 9. All costs related to AAA in the ARPP Report and Statistical Tables include both Bank Budget (BB) and Trust Fund (TF). 1. Fragile States country category in the ARPP Report and Statistical Tables is based on the list of Fragile States countries as of July 3,27 from Fragile States Web site. Portfolio classification 11. The portfolio is classified in the ARPP by region, networkhector board, sector, theme and lending instrument The Projects (No.) column in the Statistical Appendix, Tables 3.11 to 3.18 includes only those projects that are rated by IEG. 44 These classifications are assigned by Task Team Leaders during project preparation. While the classification by Regions is reliable, there are ambiguities and overlaps in the classification by sectors and lending instruments, e.g., projects which belong to the Urban Development sector board may be misclassified by the task team to other sector boards.

82

83 ANNUAL REPORT ON PORTFOLIO PERFORMANCE Fiscal Year 27 February 15,28 (STATISTICAL APPENDIX) QUALITY ASSURANCE GROUP

84

85 FY7 ARPP: STATISTICAL ANNEX Contents PORTFOLIO SIZE AND COMPOSITION ~ Table 2.1 Table 2.2 Table 2.3 Table 2.4 Table 2.5 Table 2.6 Table 2.7 Table 2.8 Table 2.9 Table 2.1 Table 2.11 Table 2.12 Table 2.13 Table 2.14 Table 2.15 Table 2.16 Table 2.17 Table 2.18 Table 2.19 Table 2.2 Table 2.21 Table 2.22 Table 2.23 Portfolio Distribution by Region Portfolio Distribution by RegionlCountry Portfolio Distribution by NetwoMSedor Board Portfolio Distribution by Instrument Portfolio Distribution by Source of Funds Portfolio Distribution by Theme Portfolio Distribution by Sector Portfolio Distribution by Country Category Grouping Portfolio Concentration by Country (FY7) Approvals by Region Approvals by NetworWSector Board Approvals by Instrument Approvals by Theme Approvals by Sector Approvals by Country Category Grouping Entries and Exits by Region Entries and Exits by Source of Funds Entries and Exits by NetworWSector Board Entries and Exits by Instrument Entries and Exits by Theme Entries and Exits by Sector Entries and Exits by Country Category Grouping Number of Overage Projects by RegionlNetwork PORTFOLIO PERFORMANCE Table 3.1 Portfolio Status Indicators by Region Table 3.2 Portfolio Status Indicators by NetwoMSector Board Table 3.3 Portfolio Status Indicators by Sector Table 3.4 Portfolio Status indicators by Theme Table 3.5 Portfolio Status Indicators by Instrument Table 3.6 Portfolio Status Indicators by Source of Funds Table 3.7 Portfolio Status Indicators and IEG Outcomes by Region for IDA Projects Table 3.8 Portfolio Status Indicators by Country Category Grouping Table 3.9 Portfolio Risk Status Ordered by Country (FY7) Table 3.1 Performance of Projects Exiting the Portfolio by Region Table 3.11 Net Disconnect by RegionlCountry Table 3.12 Net Disconnect by Instrument Table 3.13 Net Disconnect by Source of Funds Table 3.14 Net Disconnect by RegionlExit Year Table 3.15 Net Disconnect by NetworklExit Year Table 3.16 Net Disconnect by ThemdExit Year Table 3.17 Net Disconnect by SectorlExit Year Table 3.18 Net Disconnect by Country Category Grouping/Exit Year Table 3.19 (a) Changes in Outcomes Ratings between ICRR and PPAR by Exit Year (FY9-5) Table 3.19 (b) Summary of Changes in Outcomes Ratings between ICRR and PPAR Table 3.2 Changes in Outcomes Ratings between ICRR and PPAR by Region (FY9-5) Table 3.21 Changes in Outcomes Ratings between ICRR and PPAR by Network (FY9-5) Table 3.22 Changes in Outcomes Ratings between ICRR and PPAR by Source of Funds (FY9-5)

86 PORTFOLIO PERFORMANCE (Cont'd) Table 3.23 Table 3.24 Table 3.25 Table 3.28 Table 3.27 Table 3.28 Table 3.29 Table 3.3 Table 3.31 Table 3.32 Table 3.33 Table 3.34 Table 3.35 Table 3.36 Table 3.37 Table 3.38 Table 3.39 Net Changes in Outcomes Ratings between ICRR and PPAR by Elapsed Time between icrr and PPAR (FY9-5) Net Change in Outcomes Ratings between ICRR and PPAR by ICR Quality Portfolio Risk Factors Portfolio Risk Factors by Region Disbursement Ratio by Region and Country Category Grouping Cancellations by RegionlCountry Cancellations by NetworWSector Board Quality at Entry by Region Quality at Entry by Network Quality at Entry by Country Category Grouping Quality at Entry by Source of Funds Quality at Entry by Giobal & Regional, Additional Financing and MultiSector Quality at Entry For Guarantees Quality of Global Programs and Partnerships Trends in Quality of AAA Quality of AAA by Region Quality of AAA by Network AAA Table 4.1 Table 4.2 Table 4.3 Table 4.4 Table 4.5 Table 4.8 Table 4.7 Table 4.8 Table 4.9 Table 4.1 Table Table 4.12 Table 4.13 Table 4.14 AAA Deliveries and Costs by RegionlCountry AAA Deliveries and Costs by NetworWSector Board AAA Deliveries and Costs by Country Category Grouping AAA Deliveries (NO. OF TASKS) by Region and NetworWSector Board, FY3-7 AAA Deliveries (US$ ') by Region and NetworWSector Board, FY3-7 AAA Concentration: Top Ten Countries by Number of Deliveries, FY3-7 AAA Concentration: Top Ten Countries by Cost of Deliveries, FY3-7 AAA Deliveries and Costs by Output Type ESW Deliveries and Costs by Report Type AAA Size Variations by Cost RangdMajor Output Type Timeliness of AAA Reports by Region and by NetworklSector Timeliness of AAA Deliveries by Output Type Country AAA Intensity by RegionlBorrower AAA Products by Major Sector and Theme

87 m co *-

88 TABLE 2.2: PORTFOLIO DISTRIBUTION BY REGlONlCOUNTRY RegionlCountry Projects (No.) Net Commitments (US$ M illion) FY2 FY6 FY7 FY2 FY6 FY7 AFR Africa Anaola I Benin Burkina Faso Burundi w Cameroon Cape Verde Central African Republic 3 3 Chad Comoros Congo, Democratic Republic of Congo, Republic of Cote d'lvoire 12 E ritrea Ethiooia Gabon Gambia, The Ghana Guinea Guinea-Bissau Kenya Lesotho Liberia 1 6 Madagascar Malawi Mali Mauritania Mauritius 3 1 Mozambique Namibia 2 3 Niger Nigeria Rwanda Sao Tome and Principe Senegal Sierra Leone South Africa Swaziland I Tanzania Toao I- 4 Uganda Western Africa 1 Zambia I _ I _ Zimbabwe 1 65 Sub Total ,331 18,569 21,93 5 Of 83

89 TABLE 2.2: RegionlCountry EAP Cambodia China East Asia and Pacific Indonesia Kiribati Korea. Keoubllc at Lao People's Democratic Republic Malaysia Mongolia Pacific Islands Papua New Guinea Philippines Solomon Islands Thailand Timor-Leste Tonga Vanuatu Vietnam Sub Total - PORTFOLIO DISTRIBUTION BY REGlONlCOUNTRY Projects (No.) Net Commitments (US$ Milli on) FY2 FY6 FY7 FY2 FY6 FY I I n A,-. A U 4 L YS I I I I ECA Albania I 272 Aral Sea 1 12 Armenia Azerbaijan Belarus Bosnia and Herzegovina Bulgaria Central Asia 1 1 I Croatia Estonia I 25 Europe and Central Asia I 1 I Georgia Hungary Kazakhstan Kosovo Kyrgyz Republic Latvia Lithuania Macedonia, former Yugoslav Republic of Moldova Montenegro 34 Poland Romania Russian Federation Serbia Slovak Republic Slovenia 2 25 Tajikistan Turkey Turkmenistan 2 45 Ukraine 1 12 I Uzbekistan Sub Total ,21 16,514 16,687 6 of 83

90 TABLE 2.2: PORTFOLIO 1 DISTRIBUTION BY REGlONlCOUNTRY RegionlCountry LCR Argentina Barbados Belize Bolivia Bra7il Caribbean Central America Chile Colombia Costa Rica Dominica Dominican Republic Ecuador El Salvador Grenada Guatemala Guyana Haiti Honduras Jamaica Latin America Mexico Nicaragua OECS Countries Panama Paraguay Peru St. Kitts and Nevis St. Vincent and the Grenadines Trinidad and Tobago Uruguay Venezuela, Republica Bolivariana de Sub Total MNA Algeria Djibouti tgypt, Arab Kepublic of Iran, Islamic Kepublic of Iraq Jordan Lebanon Morocco Red Sea and Gulf of Aden West Bank and Gaza Yemen. ReDublic of Sub Total Prolects (No.) Net Commitments (US$ M illionl FY2 FY6 FY7 FY2 FY6 FY ,28 3,492 3,927 I ,225 4,49 4,386 n la ,652 16,628 16, ,12 1,795 1, ,355 1, , ,356 6,621 6,118 7 Of 83

91 TABLE 2.2: PORTFOLIO DISTRIBUTION BY REGlONlCOUNTRY SAR RegionlCountty Projects (No.) Net Commitments (US$ Mill ion) FY2 FY6 FY7 FY2 FY6 FY7 OTH Sub Total Total 1,543 1,468 1,485 14,577 95,194 1,357 a of a3

92

93 m co L 2 c 4

94 TABLE 2.5: PORTFOLIO DISTRIBUTION BY SOURCE OF FUNDS Source Of Funds Projects (No.) Net Commitments (US$ Billion) FY2 FY6 FY7 FY2 FY6 FY7 IBRD IDA GEF MONT SPF Total 1,543 1,468 1, GEF MONT SPF Global Environment Facility Montreal Protocol Special Fund

95 -- TABLE 2.6: PORTFOLIO DISTRIBUTION BY THEME Theme Projects (No.) FY2 FY6 FY7 Economlc management Macroeconomic management Sub Total Environment and natural resources management Environmental policies and institutions Pollution management and environmental health Water resource management Sub Total Financial and private sector development Infrastructure services for private sector development Other financial and private sector development Regulation and competition policy State enterprise/bank restructuring and privatization Sub Total Human development Education for all Health system performance Sub Total Public sector governance Administrative and civil service reform Decentralization Sub Total Rule of law Rural development Rural services and infrastructure Sub Total Soclal developmentlgender/lncluslon Participation and civic engagement Sub Total Social protection and risk management Improving labor markets 32 2 I9 Sub Total Trade and integration Export development and competitiveness Sub Total Urban development Access to urban services and housing ' Municipal governance and institution building Other urban development Sub Total Net Commitments (US$ Milllon) FY2 FY6 FY , _, , ~ 5,36 4,15 4,314 3,36 2,545 3,38 15,419 11,428 12,177 4,717 6,959 8,71 4,748 2,27 1,812 4,642 3,199 2,686 3,851 2,75 1,982 2,329 16,878 17,15 3,151 2,749 2,476 2,68 1,985 1,925 11,22 12,598 13,396 1,994 2,541 2,967 2,68 2,197 1,993 9,249 8,194 8, ,877 1,948 1,58 9,81 8,68 8,957 14,454 13,1 14,232 4,139 3,991 3,977 8,249 7,656 7,616 1,939 1,374 1,424 6,284 6,795 6, ,359 1,45 2,914 4,799 5,616 A 5?d A 5n MR ,23 3,547 4,274 12,971 11,121 12,324 Total 1.m 1.m l.ae5 Notes: 1. This table shows subthemes where the no. of projects or commitments exceeds 2.5% of the portfolio, or the largest sub-theme if no sub-theme exceeds 2.5%. 2. The number of projects or commitments in a theme is the sum of the individual fractional parts attributed to each theme within a project. 12 of 83

96 TABLE 2.7 PORTFOLIO DISTRIBUTION BY SECTOR Sector Agrlculture, fishing, and forestry General agnculture. fishing and forestry sector Irrigation and drainage Sub Total Projects (No.) Commltments (US$ Million) FY2 FY6 FY7 FY2 FY6 FY ,83 2,21 2,232 3,663 3,63 4,147 9,673 8,845 9,386 Educatlon Primary education Sub Total Energy and mining Power Sub Total Ill ,41 3,61 2,652 9,283 8,222 7,875 9,465 6,924 6,573 12,38 1,8 9,59 Finance Banking Health and other social services Health Other social services Sub Total 3,347 1,567 1,725 8,536 4,374 4,462,42 6,128 Industry and trade General industry and trade sector Sub Tatal lnformatlon and co--llni-+i-- communications Telecomr Telecommunications Sub Total ,911 1,15 1,24 5,815 4,87 4, Public Administratlon, Law, and Justice Central government administration Sub-national government administration ti A2 -- Sub Total ,31 8,377 8,532 3,45 4,554 5,372 17,6 16,354 17,334 Transportation General transportation sector Roads and highways Sub Total Water, sanltation and flood protection Sewerage Water supply Sub Tatal _. -.- _._ IA ,14 15,276 16,769 19,683 2,175 22,584 2,463 2,683 3,11 3,427 3,857 4,376 1,29 1,37 12,377 _(Hirtoric)Environment (Historic)Other environment Sub Total 1 n n ",l /l L U v ,543 1,468 1,485 14,577 95,194 1,357 Notes: 1. This table shows sub-sectors where the no. of projects or commitments exceeds 2.5% of the portfolio, or the largest sub-sector exceeds 2.5%. 2. The number of projects or commitments in a sector is the sum of the individual fractional parts attributed to each sector within a project. 13 of 83

97 TABLE 2.8: PORTFOLIO DISTRIBUTION BY COUNTRY CATEGORY GROUPING Country Category Grouping Total Projects (No.) FY2 FY6 FY7 IBRD Investment Grade a2 171 China IBRD Only (Others) India Blend IDA Only Licus Multi-Country Total 1,543 I,468 1,485 Commltments (US$ Billion) IBRD Investment Grade China IBRD Only (Others) India Blend IDA Only Licus Multi-Country Total

98 19 Arranged by Countries with Largest Number of Projects TABLE 2.9: PORTFOLIO CONCENTRATION BY COUNTRY (FY7) Country Commitments Cummuiatlve Projects Yo of Total Projects China 74 9,98 5 India _ 9 Brazil 52 4, Vietnam 3a 3, Argentina 29 3, Africa 28 1,77 19 Indonesia 26 2, Bangladesh 24 1, Nigena 24 2, Turkey 24 5, Philippines 23 1, Tanzania 23 1,94 29 Mexico 22 2, Afghanistan 21 1,14 32 Ethiopia 21 1, Romania 21 1,76 35 Colombia 2 1, Kyrgyz Republic Pakistan 2 2,47 39 Russian Federation Georgia Peru Yemen, Republic of Albania ia Croatia la 1,19 46 Uganda 18 1, Armenia Madagascar 17 1,16 5 Senegal Tunisia Azerbaijan Bosnia and Herzegovina Honduras 16 lifil 55 Kenya Riirkina Facn I5 5AI 5A ~ Egypt, Arab Republic of 15 ~ 1,322 ~~ 59 Ghana Moldova Mozambique Morocco 14 a 1 63 Mali Mongolia Tajikistan Lao People's Democratic Republic Sri Lanka Nepal Macedonia, former Yugoslav Republic of 12 1 a2 69 West -- Bank -- and _ - Gaza -_-_ Cambodia Malawi Nicaragua Rwanda I Ukraine Haiti Kazakhstan Mauritania Uruouav I _. 76 Serbia Republic of 1 1, Bulgaria Benin Arranged by Countries with Largest Commltments Country Cummulative Projects Commitments Yo of (US$ Million) commitments India 67 14, China ~ Turkey 24 5,675 3 Brazil 52 4, Vietnam 38 3, Argentina 29 3, Indonesia 26 2, Nigeria 24 2, Mexico 22 2,322 5 Pakistan 2 2,47 52 Ethiopia 21 1, Bangladesh 24 1, Colombia 2 1, Tanzania 23 1,94 59 Russian Federation 2 1, Africa 28 1,77 63 Romania 21 1,76 65 Congo, Democratic Reoublic of _r- _.- Iran. Islamic Republic of 9 1, Egypt, Arab Republic of 15 1, Uganda 18 1,313 7 Philippines 23 1, Croatia Madagascar 17 1,16 74 Afghanistan 21 1,14 75 Ukraine PeN Kenva Ghana Morocco Sri Lanka Azerbaijan Poland -- -? _- Mozambique a2 Senegal YemeyRepublic of ~ 712 a4 Tunisia Mali a5 Kazakhstan Burkina Faso Nepal Bulgaria Iraq a7 Serbia _. _ Guatemala aa Uruguay Malawi Honduras - _--_ _ Lebanon Zambia Georgia Burundi Ecuador Rwanda Bosnia and Herzegovina Niger Mauntania Dominican Republic Benin Armenia Angola of 83

99 TABLE 2.9: PORTFOLIO CONCENTRATION BY COUNTRY (FYO7) Arranged by Countrles wlth Largest Number of Projects Cummulatlve Commitments % of Total Country Projects W S Million) Pro,ects Ecuador Guinea Guatemala Iran. Islamic Republic of 9 1, Jordan Lebanon Nioer " a3 Sierra Leone Zambia Burundi Chile Cameroon Lesotho Poland Timor-Leste Dominican Republic Chad Angola Bhutan Djibouti Eritrea St - Lucia Liberia Uzbekistan Kosovo Bolivia Congo, Republic of Guinea-Bissau Jamaica South Africa 'Montenegro East Asia and Pacific Cape Verde Grenada Gambia, The Ira Panama Paraguay El Salvador Central America Latin America OECS Countries Caribbean Central African Republic Costa Rica Aloeria " Guyana Namibia Slovak Republic Samoa Belarus Gabon Hungary St. Kitts and Nevis Lithuania 'Maldives Papua New Guinea Sao Tome and Principe Thailand Tonga St. Vincent and the World Pacific islands Arranged by Countries with Largest Commitments Country Cummulatlve Projects Commitments % of (USS Miliion) commitments Albania Jordan Kyrgyz Republic Cameroon Uzbekistan Sierra Leone Cambodia Chile Nicaragua Eritrea Mongolia Guinea El Salvador Rnlivia 5 i qn 97 Macedonia, former Yugoslav Republic of Moldova Tajikistan Lao People's Democratic Republic Chad Panama Haiti Thailand Paraguay Congo, Republic of Central African Republic Lesotho West Bank and Gaza Jamaica Papua New Guinea Costa Rica Liberia Belarus Diihoiiti _ Guinea-Bissau Bhutan Cape Verde Gambia, The Oabnn 7 Fin 99 Algeria Hungary South Africa._ Lithuania St Lucia Timor-Leste Kosovo East Asia and Pacific Montenegro Samoa Grenada Guyana Maldives Central Asia Malaysia Central America Latin America Slovak Republic OECS Countries Trinidad and Tobago Namibia Comoros 'Venezuela. Republica Caribbean Barbados of 83

100 TABLE 2.9: PORTFOLIO CONCENTRATION BY COUNTRY (FYO7) Arranged by Countries with Largest Number of Projects Arranged by Countries with Largest Commltments Cummulatlve Commitments Cummulative Projects Commltments Country % of Country Projects (US$ Mllllon) ;:i 2;' M1lllon) Commitments St. Vincent and the Central Asia Grenadines 2 13 I Europe and Central Asia Tonga Barbados 1 15 I Sao Tome and Principe Dominica - I 1 inn -- Pacific - Islands - I 9 I._ Kiribati I 2 I St Kitts and Nevis Comoros I 18 1 Europe and Central Asia I 5 1 Malaysia World 1 3 I Trinidad and Tobago Kiribati I 2 I Bolivariana de I 17 1 nn Dnminira Latvia 1 Latvia 1 Total 1,485 1,357 1 Total 1,485 1, of 83

101 TABLE 2.1: APPROVALS BY REGION Region AFR EAP FCA LCR MNA SAR OTH Total TABLE 2.11: APPROVALS BY NETWORKSECTOR BOARD NetworWSector Board FPD FSE Financial Sector FSE Total PSD Private Sector Development PSD Total FPD Total Projects (No.) FY3 FY4 FY5 FY6 FY la I ,214 1,279 1,195 1,734 1,112 1,214 1,279 1,195 1,734 1, , , , , ,7 2,56 1,82 3,175 1,99 Education Health, Nutrition and Population Social Protection HDN Total ,158 1,534 1,288 1,81 2,32 1,193 1,761 1,11 1,15 1,918 I., I., min , mi ,699 4,955 4,414 3,816 4,756 Financial Management OPCS Total n n n n I SDV ESSD Environment Rural Sector Social Development ESSD Total INF Energy and Mining Global InformationlCommunications Transport Urban Development Water Supply and Sanitation INF Total SDV Total R ,185 1,76 1,495 2,894 1, ,61 2,299 1,446 1,118 1, ,935 3,752 3,358 4,749 4, ,93 2,9 2,679 2,752 3, ,877 2,547 4,116 3,768 3, ,55 1,636 2,788 1,45 Total ,729 2,353 22,221 23,94 24,874 Page 18 of 83

102 C a: 2 3 d a: N a 6 C C IC IC t- * C N C C $ 2 m N z Ln co r- N N r r t Q N VI cy 1 m cy cy m s 8 E C m

103 m co c cv

104 o co c

105 lc lc lc In w co co m m co L (v (v c C 2 * 3 CI.- 91 E n h

106 TABLE2.16: ENTRIES AND EXITS BY REGION Region AFR EAP ECA LCR MNA SAR OTH Total Projects Approved (No.) FY5 FY6 FY fir; fi I Projects Exiting (No.) FYOS FY6 FY A7 7. fia " Approvals - Exits (No.) FY I A7 Region AFR EAP ECA LCR MNA SAR OTH Total Projects Approved (US$ Million) FYO5 FY6 FY7 3,849 4,838 5,745 2,926 3,462 4,85 4,71 4,71 3,781 5,2 6,24 4,629 1,339 1, ,33 3,85 5, ,221 23,94 24,74 Projects Exiting (US$ Million) FY5 FY6 FY7 3,999 3,158 3, ,81 23,27 18,13 Approvals - Exlts (US$ Million) FY5-7 A.,-.- n7r -I.,_. P7R - 2,873-1,11 1,236 3,192-22,279 TABLE2.17: ENTRIES AND EXITS BY SOURCE OF FUNDS Source Projects Approved (No.) Projects Exitlng (No.) Approvals - Exits (No.) offunds FY5 FY6 FY7 FY5 FY6 FY7 FY5-7 IBRD - _ Ill I. RR i in IDA TF Total Source Projects Approved (US$ Million) Projects Exltlng (US$ Mlllion) Approvals - Exits (US$ Million) offunds FYO5 FY6 FY7 FY5 FY6 FY7 FY5-7 IBRD 13,334 14,135 12,784 1,734 14,419 1,979 4,121 IDA 8,559 9,446 I 1,752 9,21 9,194 7,676 3,867 TF Total 22,221 23,94 24,874 2,81 23,27 18,813 8, of 83

107 mm 7- I

108 .- 5 C. 5.E h E wag 3s *.g ucl, y o a tuvi E c z W I E z k c? 3 s B E z W.. Q) T N! m U c oc m zc m 2s m 3 c w

109 s E' F-J aj c i; F: Y

110 C al.- c c e ' E U C m C m C i.- U CI.- a 81 SE el to 9 Q) L $2' e- (3 co I- (V c C C U E m 5 I-.- E e E w

111 mob) owl- -. s h! ( V r Q O co b x Q, Q) 2 m co \c co (u

112 b E z: -(o SO o * +u o m N r N E $'o o m oc o m o r r r C o m r C O b r r h l r C r r r C o m o r m co c Q) cu a m a" co SQ] a= Y Y a z u nu IL

113 TABLE 3.1: PORTFOLIO STATUS INDICATORS BY REGION Region Active Projects (No.) FY2 FY6 FY7 AFR EAP ECA LCR MNA SAR OTH Total 1,543 1,468 1, Region Actual Problem Projects (No.) Potential Problem Projects (No.) FY2 FY6 FY7 FYOZ FY6 FY7 AFR EAP 23 -_ ECA LCR MNA SAR OTH Total Region Projects at Risk (No.) FY2 FY6 FY7 AFR EAP 35 -_ ECA LCR MNA ~ SAR OTH n Total Commitments at Risk (US$ Billion) FYOZ FY6 FY Region %at Risk % Commitments at Risk FY2 FY6 FY7 FY2 FY6 FY7 AFR -_ I9 EAP ECA LCR MNA SAR OTH Total Region YO Realism YO Proactivity FY2 FY6 FY7 FY2 FY6 FY7 AFR EAP ECA LCR MNA SAR OTH NA NA NA NA NA NA Total Ad ai 79.-

114

115 r r r; m 3 c (v m.. (*,

116 TABLE 3.5: PORTFOLIO STATUS INDICATORS BY INSTRUMENT instrument Dev. Policy DPL PRC PSL SAD SAL Sub Total % Projects at Risk %Commitments at Risk % Realism % Proactivity FYOZ FY6 FY7 FYOZ FY6 FY7 FY2 FY6 FY7 FYOZ FY6 FY NA NA NA NA NA 1 NA NA NA NA NA NA NA NA 5 55 NA NA NA NA NA NA NA NA ia io ia investment APL ERL FIL LIL SIL SIM TAL Sub Total NA I NA NA NA 8 NA ia a a5 a 7a Total a4 ai 79 Investment Dev. Policy APL Adaptable Program Loan DPL Development Policy Lending ERL Emergency Recovery Loan ECO Expanded Cofinancing Operation FIL Financial Intermediary Loan PRC Poverty Reduction Support Credit LiL Learning and Innovation Loan PSL Programmatic Structural Adjustment Loan SIL Specific Investment Loan SAD Sector Adjustment SiM Sectoral investment and Maintenance SAL Structural Adjustment Lending TAL Technical Assistance Loan TABLE 3.6 PORTFOLIO STATUS INDICATORS BY SOURCE OF FUNDS Source of % Projects at Risk % Commitments at Risk % Realism % Proactivlty Funds FY2 FY6 FY7 FYOZ FY6 FY7 FY2 FY6 FY7 FY2 FY6 FY7 IBRD IDA GEF MONT NA NA NA NA NA NA SPF NA NA Total a4 ai 79 GEF MONT SPF Global Environment Facility Montreal Protocol Special Fund 33 of 83

117 N N (D a 8 N ln N 3 m w w N h (D IC 1 IC ( (D h m k 7 z r z e E m h N (D f f h g d N ( m 1 N ln s1 m m d a s %- ln 2 5 ln h ln r- d z 8 8 h z h C s C U 2 ln d m 7 f h z h 3 c wc wc b b C c c E n,@ e m P

118 U z" ae v) N VI N m m s v) m VI P m cv N F m 1 4 E N 9 k m P m tn.- 6 U - Q) E B a CI a a ae U m a a B C ae r z li 9 6 IC m 3 m a2 In m c p1 3 ) m ( a k h 3 n u)

119 a s F L 3 3 a e C m m m a m $ P a z f r P 9 P 9 a z P F lo In N I N m 8 E m m F; 5 m m m cu m? f 9 T r N s! m N $ m r- a A m N 8 r N m 3 m P r N SI ln ln z ln F N s! 6 m 7 F ( N 2 % s 9 P a z ln N N ) P 8 ". 7 s f 8 r m $ 3 N 3 s OD r- r- 2

120 TABLE 3.8: PORTFOLIO STATUS INDICATORS BY COUNTRY CATEGORY GROUPING Country Category Grouping % at Risk FY2 FY6 FY7 IBRD Investment Grade China IBRD Only (Others) India Blend IDA Only Licus Multi-Country 8 13 Total Country Category Grouping IBRD Investment Grade China IBRD Only (Others) India Blend Yo Commitments at Risk FY2 FY6 FY IR IDA Only Licus Multi-Country 3 19 Total Country Category Grouping YO Realism FY2 FY6 FY7 IBRD Investment Grade China IBRD Only (Others) India Blend IDA Only Licus Multi-Country NA NA NA Total Country Category Grouping YO Proactivity FY2 FY6 FY7 IBRD Investment Grade China IBRD Only (Others) India Blend IDA Only Licus Multi-Country Total of 83

121 TABLE 3.9: PORTFOLIO RISK STATUS ORDERED BY COUNTRY (FY7) YO Projects No. of Country at Risk Projects Honduras 5 16 Argentina Peru Nigeria Bangladesh Senegal Africa Mozambique 2 15 Kenya Brazil Indonesia Rnrnania.._ ~ Egypt, Arab Republic of Ghana Philippines Azerbaijan India Madagascar Albania Uganda Vietnam Yemen, Republic of Kyrgyz Republic 1 2 Afghanistan 1 21 Ethiopia 1 21 Mexico 9 22 Turkey 8 24 Burkina Faso 7 15 Bosnia and Herzegovina 6 16 Georgia 5 19 Pakistan 5 2 Tanzania 4 23 China 4 74 Colombia 2 Russian-Federation 2 Croatia 18 Armenia 17 Tunisia 17 Moldova 15 Note: Only countries representing 1% or more of Bank commitments or projects are shown.

122 .- 3 n W VI E.- UI c n W b z T rs (r: cv T T m co c Q) T T m + R

123 RegionlCountry AFR Africa Angola Benin Botswana Burkina Faso Burundi Cameroon Cape Verde Central African Chad Comoros Congo, Democrat Congo, Republic Cote d'lvoire Djibouti Eastern Africa Equatorial Guin Eritrnn Ethiopia Gabon Gambia, The Ghana Guinea Guinea-Bissau Kenva Lesotho Liberia Madagascar Malawi Mali Mauritania Mauritius Mozambique Niger Nigeria Rwanda Sao Tome and Pr Senegal Seychelles Sierra Leone Somalia South Africa Sudan Swaziland Tanzania Togo Uganda Western Africa Zambia Zimbabwe Sub Total TABLE 3.11: NET DISCONNECT BY REGIONICOUNTRY ~~n.n~ Projects % Unsat % Unsat %Net (No.) DO Outcome Disc ia 36 ia _ n a a aa a I 1 1 a a a a ia ia AI in R 15 R ia in 8 RO n A I1 ia 7 _ , ia FYnAn7 Projects % Unsat % Unsat % Net (No.) DO Outcome Disc n n n NA NA NA 7 Id.. IA.. n A n - n n " i no 67 _. 6 a3 a3 NA NA NA NA NA NA NA NA NA a a NA NA NA n L U A U - U in.- n in.- in n n n L U U U IA.. IA.. IA.. n NA NA NA n NA NA NA 2 NA NA NA 1 ia 6 6 n NA... NA... NA NA NA NA NA NA NA Page 4 of 83

124 RegionlCountry TABLE 3.11: NET DISCONNECT BY REGlONlCOUNTRY FYQO Projects YO Unsat YO Unrat % Net (No.) DO Outcome Disc FY4-7. _. Projects %Unrat %Unsat #Net (No.) DO Outcome DISC EAP Cambodia China A A5 -- n 4 4 NA NA NA n n n a a ea 12 2a n AA Q Id , U U " n n 4 NA NA NA I A n 1 n 1 n NA NA NA n NA NA NA =CIA Alhnnin A I6 A ia 6 Aral Sea Armenia Azerbaijan Belarus Bosnia and Hem Bulgaria Central Asia Croatia Cyprus Czech Republic Estonia Georgia Hungary Kazakhstan Kosovo Kyrgyz Republic Latvia Lithuania Macedonia, form Moldova Montenegro Poland Portugal Romania Russian Federat Turkey Turkmenistan Ukraine a a AI 7 17 in a 11 la I9 n a n n n a ia 3a A 74 1A ia a a ia n n n _ 3 in._ ia a 13 3a 25 11,. 9., In I" NA NA NA in n n n I".,., NA NA NA I3 A 1.5 A n NA NA NA NA NA NA NA NA NA f. " n " n., n " n IA la NA NA NA 17 la 22 2a n n n in n Q 79 n NA NA NA Page 41 of a3

125 TABLE 3.11: NET DISCONNECT BY REGlONlCOUNTRY RegionlCountry LCR Argentina Bahamas, The Barbados Belize Bolivia Brazil Caribbean Central America Chile Colombia Costa Rica Dominica Dominican Repub Ecuador El Salvador Grenada Guatemala Guyana Haiti Honduras Jamaica Mexico Nicaragua OECS Countries Panama Paraguay Peru St. Kitts and N St. Lucia St. Vincent and Trinidad and To Uruguay Venezuela, Repu Sub Total MNA Lebanon Morocco M9-7 M4-7 Projects % Unsat % Unsat % Net Projects % Unsat % Unsat % Net (No.) DO Outcome Disc (No.) DO Outcome Disc a la NA NA NA NA NA NA a I a *6 5 5 NA NA NA NA NA NA a 4 I la ia ia a a a 9 a NA NA NA n n n ia a 17 a 5 NA NA NA 1 I 7 17 in 7n in SAR Afghanistan 5 5 Bangladesh ai a Bhutan India Maldives 6 I Nepal Pakistan a a Sri Lanka Sub Total Total 4, Page 42 of 83

126 TABLE 3.12: NET DISCONNECT BY INSTRUMENT FY9-7 Instrument Projects YO Unsat YO Unsat YO Net (No.) DO Outcome Disc Dev. Policy DPL DRL PRC PSL RIL SAD SAL SSL 2 Sub Total FY4-7 Projects % Unsat % Unsat YO Net (No.) DO Outcome Disc NA NA NA n " n " n Investment APL ERL FIL LIL SIL SIM TAL Sub Total Total , , , I1 2 9 Investment APL ERL FIL LIL SIL SIM TAL Dev. Policy Adaptable Program Loan DPL Development Policy Lending Emergency Recovery Loan DRL Debt and Debt Service Reduction Loan Financial Intermediary Loan PRC Poverty Reduction Support Credit Learning and Innovation Loan PSL Programmatic Structural Adjustment Loan Specific Investment Loan RiL Rehabilitation Loan Sectoral Investment and Maintenance SAD Sector Adjustment Technical Assistance Loan SAL Structural Adjustment Lending SSL Special Structural Adjustment Lending TABLE3.13: NET DISCONNECT BY SOURCE OF FUNDS FY9-7 FY4-7 Source of Projects YO Unsat YO Unsat ~o Net Disc Projects % Unsat % Unsat % Net Disc Funds (No.) DO Outcome (No.) DO Outcome IBRD 2, IDA 2, GEF MONT 1 4 SPF Total 4, GEF MONT SPF Global Environment Facility Montreal Protocol Special Fund 43 of 83

127

128 ~ TABLE 3.15: NET DISCONNECT BY NETWOWEXIT YEAR NetworklExlt Year Environment Rural Sector Social Development Sub Total FPD Financial and Private Sector Development Sub Total HDN Education Health, Nutrition and Population Social Protection Sub Total INF Energy and Mining Global InfwmatiodCommunications Transport Urban Development Water Supply and Sanitation Sub Total PREM Economic Policy Gender and Development Poverty Reduction Public Sector Governance Sub Total Tnkl no4 MO5 MW Project8 K Unsat 'h Unsat 'h Net Project8 K Unaat K Unaat 'h Net Project8 K Unsat K Unaat K Net (No.) DO Outcome Disc (No.) DO Outcome Disc (No.) DO Outcome Disc I IO ? n n n 1 n n n 7 n n n IC 4 7 I7 n +a c1 a n 4, 9c.)E. n I" I" I" I" I& L-. & I n n n pl7 Total NetworklCloslng Year Project8 K Unsat K Unsat K Net Projectl K Unsat Ye Unsat 41 Net (No.) DO Outcome DISC (No.) DO Outcome DISC ESSD Environment Rural Sector Soclal Development Sub Total FPD Financial and Pnvate Sector Development Sub Total HDN Education Health. Nutntion and Population Social Protection Sub Total ,.IC IRr Energy and Mining Global InfwmationICommunicatims Transport Urban Development Water Supply and Sanitation Sub Total PREM Economic Policy Gender and Development Poverty Reduction Public Sectw Governance Sub Total Tn-I n n n A n n n 7 1 1" c I " n n IS I D 45 of 83

129 L

130 v)q) ss ZQ) U Q O z.y $ a C m ~3 B t Q: yc P) N : m W c b d % U m a 2 a: F: v: w a P c tn p! e U K m m C E l a c a.- L 2 m E c E U C m % F u '1; n

131 r zb zz O W 518 W c co d

132 h 8 c Q, P - rr, 'm I a r m g NOn??*. 2

133

134 TABLE 3.23: NET CHANGES IN OUTCOMES RATINGS BETWEEN ICRR AND PPAR BY ELAPSED TIME BETWEEN ICRR AND PPAR (FY9-5) Elapsed Time between ICRR and corresponding PPAR No. of IEG Evaluations No. of Net Changes % Net Change <=2 Year 'a >2 & c= 5 Years >5 Years Total 1, a! About 38 projects with PPAR evaluation date less than their corresponding ICRR evaluation date in the system have been eliminated from the above analysis. 51 of 83

135 N r 22 m co L cv v)

136 TABLE 3.25: PORTFOLIO RISK FACTORS (As a Percentage of the Active Portfolio) Risk Factor FY6 FY7 Country Environment* 2 2 Country Record* 29 3 Counterpart Funds 4 4 Effectiveness Delay* 7 1 Financial Management 5 7 Legal Covenants 6 7 Long Term Risk* 4 4 Project Management* 6 7 Monitoring & Evaluation' 6 7 Procurement Problems 6 8 Safeguards 3 3 Slow Disbursing* 11 1 Golden Flag 1 * Risk-flags applicable to Development Policy operations as well. 53 of 83

137 I- C C C C C a r r C C Is: C C C v

138 ~ TABLE 3.27: DISBURSEMENT RATIO' BY REGION AND COUNTRY CATEGORY GROUPING (In Percent) Region FY2 FY6 FY7 ~ AFR 2 24 EAP ECA LCR MNA OTH SAR Total Country Category Grouping FY2 FY6 FY7 IBRD Investment Grade China IBRD Only (Others) India Blend IDA Only Licus Multi-Country Total ' Disbursement Ratio is the ratio between "IBRD/IDA Disbursements in the Fiscal Year" and "Opening Undisbursed Amount at the beginning of the Fiscal Year," and is restricted only to Investment projects. 55 of 83

139 TABLE 3.28: CANCELLATIONS BY REGlONlCOUNTRY (US$ Million, IBRDllDA and SPF Only) RegionlCountry FY3 FY4 FY5 FY6 FY7 AFR Cote d'lvoire Kenya Niaeria Tanzania Zimbabwe Sub Total EAP China Indonesia Vietnam Sub Total ECA Bosnia and Hetzegovina Kazakhstan Poland Russian Federation Turkev Ukraine Sub Total LCR Argentina Brazil Colombia Mexico El Salvador Uruguay Sub Total MNA Algeria Egypt, Arab Republic of Tunisia Sub Total SAR Bangladesh India Sub Total I , I Total 3,928 2,377 2,53 1,45 1,615 Notes: I. The table includes projects that are either partially or fully cancelled, while Annex 2 includes partial cancellations for projects that are either active or have exited the pottfolio. 2. The table shows individual countries with cancellations exceeding 2.5% of the Bank-wide total. 56 of 83

140 TABLE 3.29: CANCELLATIONS BY NETWORWSECTOR BOARD (US$ Million, IBRDllDA and SPF Only) NetworWSector Board FY3 FY4 FY5 FY6 FY7 FPn..- FSE Financial Sector FSE Total PSD Private Sector Development PSD Total FPD Total 1, , , Education Health, Nutrition and Population Social Protection HDN Total PREM Economic Policy Public Sector Governance PREM Total Ill SDV Environment Rural Sector ESSD Total INF Energy and Mining Transport Urban Development Water Supply and Sanitation INF Total SDV Total , , ,15 1,393 I, ,68 Total 3,928 2,377 2,53 1,45 1,615 Notes: 1. The table includes projects that are either partially or fully cancelled, while Annex 2 includes partial cancellations for projects that are either active or have exited the portfolio. 2. The table shows individual Sector Boards with cancellations exceeding 2.5% of the Bank-wide total. Page 57 of 83

141

142

143 m a) c (D

144

145

146 TABLE 3.35: QUALITY AT ENTRY FOR GUARANTEES Rated (No.) S+ MS+ OA Overall Rating R1 Strategic Relevance and Approach R2 Technical, Financial and Economical Aspects R3 Compliance with Safeguard Policies R4 Implementation Readiness and Arrangements R5 Risk Assessment and Mitigation R6 Bank Inputs and Processes to Approval of 83

147

148 cr) Q3 c In ( - C ([I b t C

149 TABLE3.38: QUALITY OF AAA BY REGION (YO Moderately Satisfactory or Better) Region No. of Tasks OA R1 R2 FY98-99 FYO5-6 FY98-99 FY5-6 FY98-99 FYO5-6 FY98-99 FY5-6 AF R EAP ECA LCR MNA SAR OTH~ 21 NA 86 NA 95 NA 95 Total Region No. of Tasks R3 R4 R5 FY98-99 FYO5-6 FY98-99 R3 FY98-99 FYO5-6 FY98-99 FY5-6 AFR EAP ECA LCR MNA SAR OTH~ 21 NA 86 NA 86 NA 86 Total Includes two Active tasks with Client Delivery in FY7 and FY8. Tasks denote Global tasks. OA = Overall Assessment R1 = Strategic Relevance R2 = Internal Quality R3 = Dialogue and Dissemination R4 = Likely Impact R5 = Bank Inputs and Processes 66 of 83

150 ~ TABLE3.39: QUALITY OF M A BY NETWORK (% Moderately Satisfactory or Better) Network No. of Tasks OA RI._. R2 FY98-99 FY5-6 FY98-99 FY5-6 FY98-99 FYO5-6 FY98-99 FY5-6 FPD FSE PSD I.. I 91 FPD Total HDN OPCS 1 NA 9 NA 1 NA 9 PREM SDV ESSD 2 24 INF SDV Total uzz NA 5 NA NA 5 Total Network FPD FSE PSD FPD Total HDN OPCS PREM SDV ESSD INF SDV Total uzz R3 R4 R5 FY98-99 FY5-6 FY98-99 FYO5-6 FY98-99 FYO NA 1 NA 8 NA NA 1 NA 5 NA 5 Total Includes two Active tasks with Client Delivery in FY7 and FY8. *Tasks that have not been assigned a Sector Board. OA = Overall Assessment R1 = Strategic Relevance R2 = Internal Quality R3 = Dialogue and Dissemination R4 = Likely Impact R5 = Bank Inputs and Processes Page 67 of 83

151 TABLE 4.1: M A DELIVERIES AND COSTS BY REGIONCOUNTRY A=m Africa Eastern Africa Southern Africa Central Africa Western Africa Angola Burkina Faso RegiodCountry Benin Botswana Central African kepublic Congo, Republic of Cote d'lvoire Cameroon Cape Verde Eritrea Ethiopia Gabon Ghana Gambia, The Guinea Equatorial Guinea Guinea-Bissau Kenya Comoros Liberia Lesotho Madagascar Mali Mauritania Mauritius.... Malawi Mozambique Namihia Niger Nigeria Rwanda Seychelles Sudan Sierra Leone Senegal Somalia Sao Tome and Principe Swaziland Chad Togo Tanzania Uganda Sniiih Afrira Zambia Congo, Democratic Republic of Zimbabwe Sub Total Deliveries (No.) Initiation to Completion Costs (US$ ') FYOS FY4 FYOS FY8 FY7 FY3 FY4 FYOS FYO8 FY n n R 1 17 A in in A R a A A a in ,647 15,27 2,988 15,711 3, n v n v n v , , , In n fia RRO ,382 2,81 2, ,267 1, , n n n AQ n , , , , , ,33 n 443?n a? 7R I_- "" , ,231 1,734 2, , , , , ,2 1,17 2, ,166 1, , ,998 1,459 1,381 1,79 1, ,585 3,871 51,52 45,81 58, of 83

152 TABLE4.1: AAA DELIVERIES AND COSTS BY REGlONlCOUNTRY RegionlCountry EAP East Asia and Pacific Mekong Pacific Islands Aiirtralia Fiji China. Hong Kong SAR Indonesia Japan Cambodia Korea, Republic of Lao People's Democratic Republic Mongolia Malaysia Papua New Guinea Philippines Palau Solomon Islands Singapore Thailand Timor-Leste Vietnam Vanuatu Samoa Sub Total Deikerles (No.) initiation to Completion Costa (US$') FY3 FY4 FY5 FY6 FY7 FY3 FY4 FY5 FY6 FY ,432 3,41 4,185 3,979 6, , n n n 1 n n n R1 -. n ,56 4,372 5,257 6,543 11, ,946 6, , ,16 1,82 1, ,38 A 1 n n n AAA 1-. 7A.- n n n ,293 1, ,559 1, , ,193 1,346 3,336 2, , I n I 1 n t , , , , ,196 2,271 2,23 1,935 1,864 2, ,162 23,854 27,728 29,85 51,66 ECA EU Accession Countries South Eastern Europe and Balkans Central Asia Europe and Central Asia Caucasus Albania Armenia Aiirtria Azerbaijan Bosnia and Herzegovina Bulgaria BelaNS CYPNS Czech Republic Estonia United Kingdom Georgia Croatia Hungary Italy Kyrgyz Republic Kazakhstan Moldova Montenegro Macedonia, former Yugoslav Republic of Poland Romania Russian Federation Slovenia Slovak Republic Tajikistan Turkmenistan Turkey Ukraine Uzbekistan Serbia Sub Total ?? A A A 7 a R a I ,928 1, , , ,512 2,9 6,394 3,56 6, ,42 A51 Q7A A5R?n 1 5A5 36 1,399 1, , n InR n n n 1, , , , ,514 2,766 2,239 2, ,3 56 1, , , , , ,763 4,931 2,845 3,16 5, ,63 1, , , ,889 1,435 3,571 2, ,651 1,276 1, , ,622 1, , ,639 29,98 29,193 26,144 37,124 6g of 83

153 TABLE 4.1: AAA DELIVERIES AND COSTS BY REGlONlCOUNTRY. -- RegIonlCountry Argentina Barbados Bolivia Brazil Chile Colombia Costa Rica Dominica Dominican Republic Ecuador Grenada Guatemala Guyana Honduras Jamaica St. Kitts and Nevis St. Lucia Mexico Nicaragua Panama Pan1. Paraguay El Salvador Trinidad and Tobago Uruguay St. Vincent and the Grenadines Venezuela, Republica Boiivariana de. SUB TOMI Deliverlea (No.) Initiation to Completion Costs (US$ ') FY3 FY4 FYO5 MOB FY7 FY3 FY4 FYOS M8 FY ',4 l2.. n R A , ,883 2,41 2,72 3, n aq 77a 179 M7 95 1, ,618 1, ,888 1,453 2,324 2,451 3, , , i ins i QRA t ma I AAn QRO , ,525 1,81 1,816 1, ,295 3, A7 776 I IA *RA ,443 15,93 16,533 22,564 17,767 MNA Gulf Cooperation Council Middle East and North Africa United Arab Emirates Bahrain Djibouti Algeria Egypt. Arab Republic of West Bank and Gaza Iraq Iran. Islamic Republic of Jordan Kuwait Lebamn Libya Morneeo Malta Yemen, Republic of Saudi Arabia Syrian Arab Republic Tunisia Sub Total Ill a q 1 ir 9 7 R 7 R ,643 3,138 2,381 2,378 2, , , ,411 2,693 2, ,327 1,228 1,595 1,84 4, ,243 1, , , ,97 1, , ,75 1,235 1,79 2,36 2, ,9 1,44 1,671 1, ,337 3,51 1,581 3, , , ,594 17,969 22,716 16,729 19,79 7 of 83

154 --. CAD ReglonlCountry TABLE4.1: A M DELNERIES AND COSTS BY REGlONlCOUNTRY Delivrrles (No.) lnltlatlon to Completion Costs (US$ ') FY3 FY4 FY5 MOO FY7 M3 M4 FYO5 FY6 FYO7 South Asia ,197 2,444 Afghanistan ,779 3,444 1,277 1,34 Bangladesh , , ,26 Bhutan India ,574 8,449 8,338 1,761 8,839 Srl Lanka , , Maldives Nepal ,145 1, ,584 Pakistan ,42 2,1 2,178 3,25 2,752 Sub Total I8 S ,123 16,636 19,498 21,871 19,51 OTH -... World ,989 1,158 14,254 5,97 8,16 Asia 1 22 Sub Total ,989 1,156 14,254 5,992 8,16 Total 1,65 1,37 1,45 SO8 S61 141, , ,425 17, ,939 Notes: 1. The table includes ESW Reports, Other ESW and TA products. 2. The Deliveries (No.) and the Initiation to Completion Costs (US$ ') includes supplemental deliveries. 3. ERective July 1,24, "ConsuItationsCountry Dialogue" and "ConferenceNorkshop" output types are no longer valid for the ESW product line 4. Initiation to Completion Costs include postdelivery costs. 71 of83

155 ._I. 1,65 ~ ~~ ~~ ~ ~~ ~ TABLE42 A M DELIVERIES AND COSTS BY NETWORWSECTOR BOARD NetworklSector Board ACS Administrative and Client Support Sub Tats1 ESSD Environment Rural Sector Social Development Sub Total FSE Financial Sector Sub Total HDN Education Health, Nutrition and Population Social Protection Sub Total INF Energy and Mining Global Information/ Communications Tech nology Transport Urban Development Water Supply and Sanitation Sub Total ISN Professional Development Sub Total OPCS Financial Management Operational Services Procurement Sub Total PREM Economic Policy Gender and Development Poverty Reduction Public Sector Governance Sub Total PSD Private Sector Development Sub Tntal RMN - Resource Management Sub Total Deliveries (No.) lnltlation to Completlon Costs (US$ ') FY3 FY4 FY5 FY6 FYO7 FY3 FY4 FY5 FY6 FY ,243 I I 83 2, ,67 6,43 12,84 9,645 11, ,33 9,18 15,483 17,137 12, , ,613 2, I ,25 23,491 37,64 31,595 44, ,755 16,288 13,196 14,458 11, ,755 16,288 13,196 14,458 11, ,774 9,159 8,58 9,1 9, ,5 5,927 1,281 9,562 11, ,991 5,747 6, , ,77 2,833 25,13 27,856 31, ,525 9,649 4,89 6,996 8, , ,641 1, ,35 3,445 4,236 2,612 17, ,4 5,241 8, , ,94 1,522 5,579 3,64 2, I ,23 21,56 23,752 24,5 39, , ,64 3, ,877 2,851 1, , ,787 7,658 6,869 4,29 5, ,21 19, , ,661 1,295 1,12 1, ,167 11,574 7,86 14,242 17, ,64 11,53 12,847 11,273 17, ,366 43,39 51,75 53,16 65, ,671 19,435 23,961 13,27 13, ,671 19,435 23,961 13,27 13, I Sector Board not Applicable Sub Total Told ,23 1, ,23 1, ,37 1, , , ,425 17, ,939 Notes: 1. The table includes ESW Reports, Other ESW and TA products. 2. The Deliveries (No.) and the Initiatbn to Completion Costs (US$ ') includes supplemental deliveries. 3. Effective July 1,24, "ConsuitatbndCountty Dialogue" and "ConferenceNVorkshop" output types are no longer valid for the ESW product line. 4. Initiation to Cornpletbn Costs include postdelivery costs. 72 of 83

156 m CO c m r-

157

158 m co c o r-

159 TABLE 4.6: AAA CONCENTRATION: Top Ten Countries by Number of Deliveries, FY3-7 Region SAR EAP EAP ECA EAP LCR MNA MNA EAP EAP Sub Total Total Country Delieveries (No.) Initiation to Completion Costs (US$ ') BB TF Total India ,944 12,16 4,959 China ,661 13,279 31,94 Indonesia ,524 27,386 41,91 Russian Federatlon 95 14,54 4,366 18,96 Vietnam 88 7,88 3,856 1,944 Brazil 84 12,41 1,181 13,591 Saudl Arabia 84 13,5 13,5 Morocco ,49 Phlllppines 67 7,3 4,649 11,679 Thailand 64 5,964 2,929 8,893 1,3 131,18 7,3 21,318 5,16 618, , ,969 %of Total 21% 21% 29% 23% Notes: 1. The table includes ESW Reports, Other ESW and TA products. 2. The Deliveries (No.) and the Initiation to Completion Costs (US$ ') includes supplemental deliveries. 3. Effective July 1, 24, "Consultations/Country Dialogue" and "ConferencelWorkshop" output types are no longer valid for the ESW product line. 4. initiation to Completion Costs includes post-delivery costs. * Expenditures from the BB budget are reimbursed by governments of Saudi Arabia under the reimbursable technical assistance program. TABLE 4.7: AAA CONCENTRATION: Top Ten Countries by Cost of Deliveries, FY3-7 Region Country Delleveries (No.) EAP Indonesia 149 SAR lndla i 7a EAP China 153 ECA Russlan Federation 95 LCR Brazil a4 MUA EAP EAP SAR AFR Sub Total Saudi Arabla Philippines Paklstan Vietnam Nlgerla ,17 Initiation to Completion Costs (US$ ') BB TF Total 14,524 27,386 41,91 28,944 12,16 4,959 18,661 13,279 31,94 14,54 4,366 18,96 12,41 1,181 13,591 13,5 13,5 7,3 4,649 11,679 1,363 1,185 11, , ,43 1, ,276 7,321 24,597 Total , , ,969 % of Total 2% 22% 29% 24% Notes: 1, The table includes ESW Reports, Other ESW and TA products. 2. The Deliveries (No.) and the Initiation to Completion Costs (US$ ') includes supplemental deliveries. 3. Effective July 1,24, "ConsultationdCountry Dialogue" and "ConferencelWorkshop" output types are no longer valid for the ESW product line. 4. initiation to Completion Costs includes post-deiivery costs. * Expenditures from the BB budget are reimbursed by governments of Saudi Arabia under the reimbursable technical assistance program. 76 of 83

160 TABLE4.8: AAA DELIVERIES AND COSTS BY OUTPUT TYPE Rope- Dellvwler (No.) FYOS FY4 MOS FYOS MOT ESW Reports Core Dlagnostlc Reports PA CEM/DPR CFM CPAR I FA Other Dlagnostlc Reporb Other Advisory Reports ESW Pollcy NoWOther Product I Palicy Note ConferencelWorkshop NA NA NA CcmsultationslCountry Dialogue NA NA NA Inbtlon to ComolsUon Cosla luss 'OOO1 FYOS FY4 FYOS FYO8 FY7 89,532 82,W 98,712 14,817 12,54 25,557 24,557 22,483 22,132 25,178 4,524 6,666 4,379 7,161 6,5 7,819 6,62 1,29 7,649 5,241 6,653 5,961 5,199 3,543 9,679 3,551 2, ,571 2,211 1, ,199 3,192 14,896 18,542 22,729 25,932 18,659 7,152 4,637 3,21 4,449 3, ,96 19,527 21,463 15, ,839 55,52 56,753 58,929 22,819 27,451 36,894 2,332 18,489 13,973 15,555 35,694 2,332 16,469 5,429 6,196 NA NA NA 3,217 5,7 NA NA NA All ESW Producta TA Output Typea Client Dccument Review lnstitutimal Development Plan Knowledge-Shanng Fwum ModeUSurvey "HOW-TO" Guidance All TA Products ,95 11,4 134,48 125, ,135 5,52 2,993 2,955 2,276 7,119 14,343 16,669 13,679 12,753 16,269 11,936 13,645 19,29 13,22 17,863 2,177 1,564 1,695 1,667 3,74 16,76 6,672 11, ,459 49,585 43,744 49,19 46,778 92,84 All AAA Product. 1,65 1,37 1, , , ,421 17, ,939 Notes: 1. The table includes ESW Reports. Other ESW and TA products. 2. The Deliveries (No.) and the Initiation to Completion Costs (US$ ') includes supplemental deliveries. 3. Effective July 1, 24. "Consultations/Country Dialogue" and "CcmferencelWorkshop" output types are no longer valid fw the ESW product line 4. Initiation to Completion Cmts includes postdelivery costs. 77 of 3

161 Report Type Core Dlagnostlc Reports ACD E4P ECA LCR MNA SAR TABLE4.9: ESW DEWERIES AND COSTS BY REPORT TYPE DellveMrles (No.) lnlatlon to Completion Costs (US$ ') FYOJ FYO4 FY5 FYO6 FY7 FY3 FYM FYO5 FY6 FY ,299 7,234 9,117 6,868 9, ,251 1, ,295 8,819 4,624 4,365 5, ,779 4,116 5,276 5, ,512 2,226 1,773 1, , MULTl Total Core Dlagnostlc Reports SO ,557 24,567 22, AFR EAP ECA LCR MNA SAR MULTI Total Other Diagnostic Reports ,463 3, , ,531 2,53 3,959 3, ,199 4,611 4, ,418 2,54 3,299 6,15 2, , 2,796 1,695 2,236 2, ,742 3, ,896 18,542 22,729 25, Advlrory Reports AFR EAP MNA MULTl Total Advlaory Reportr. I ,423 9, , , ,474 12,529 12, , ,696 7,32 8, , , ,175 4,33 4, ,332 5,822 7,135 7,484 1, ,765 5,183 11,663 3,621 4, ,79 39,PS 53,52 58,919 5,929 All ESW Reaam AFR ,962 17,466 24,138 28,824 24,493 EAP ,55 1,426 17,749 23,441 ECA ,939 18,322 17,643 15, LCR ,32 12,355 16,497 19, MNA ,53 7, , SAR ,327 9,121 11, ,147 MULTI ,279 5,183 11,924 3, Total All ESW Reportr ,332 82,946 98,712 11,983 12,663 Notes: 1. The table includes ESW Repcits only. 2. The Deliveries (No.) and the Initiation to Completion Cmts ($) includes supplemental deliveries.

162 . 9 r 4 I: 5 P r r 3: m 3 c 2

163