PROGRAMMATIC ADJUSTMENT LENDING RETROSPECTIVE

PROGRAMMATIC ADJUSTMENT LENDING RETROSPECTIVE Operations Policy and Country Services July 11, 2003

ABBREVIATIONS AND ACRONYMS APL/C CFAA CPAR DO EA ESW EU ICR IMF IP JSA M&E MDG MTEF OC OED PER PRGF PRSC PRSP PSAL/C QAE QAG Adaptable program loan/credit Country Financial Accountability Assessment Country Procurement Assessment Review Development objective Environmental assessment Economic and sector work European Union Implementation Completion Report International Monetary Fund Implementation progress Joint Staff Assessment Monitoring and evaluation Millennium Development Goal Medium-Term Expenditure Framework Operations Committee Operations Evaluation Department Public Expenditure Review Poverty Reduction Growth Facility Poverty reduction support credit Poverty Reduction Strategy Paper Programmatic structural adjustment loan/credit Quality-at-Entry Review Quality Assurance Group

PROGRAMMATIC ADJUSTMENT LENDING RETROSPECTIVE CONTENTS Page Executive Summary...iii I. Introduction... 1 II. Initial Experiences with Programmatic Adjustment Lending... 1 A. Objectives and Types of Programmatic Lending... 2 B. Policy Framework for Programmatic Adjustment Lending... 3 C. Early Trends in Programmatic Adjustment Operations... 5 III. Country Context, Content, and Coverage... 12 A. Country Context and Ownership... 12 B. Analytic Underpinnings... 12 C. Sectoral Coverage... 13 D. Poverty Impacts... 13 E. Fiduciary and Safeguard Issues... 15 F. Collaboration with the IMF and Other Development Partners... 17 IV. Issues of Design and Implementation... 21 A. Design and Phasing... 21 B. Triggers and Prior Actions... 21 C. Monitoring and Evaluation... 25 D. Risk Assessment... 26 V. Conclusions and Issues Going Forward... 29 A. Conclusions... 29 B. Issues Going Forward... 30 Figures Figure1. Lending Approvals.... 5

ii Boxes Box 1. Programmatic and Other Adjustment Lending: Selected Features... 3 Box 2. Review and Approval Processes for Programmatic Adjustment Loans... 4 Box 3. Staff Views of the Programmatic Approach... 10 Box 4. Borrower Feedback on the Programmatic Approach... 11 Box 5. Participatory Approaches in PRSCs... 14 Box 6. Poverty Impacts Good Practice... 15 Box 7. Fiduciary Good Practices... 16 Box 8. Fiduciary Issues in the Burkina Faso PRSCs... 16 Box 9. PRSPs, PRGFs, and PRSCs... 18 Box 10. Donor Coordination on Programmatic Adjustment Lending: Good Practice Examples... 20 Box 11. Triggers Good Practice... 22 Box 12. Monitoring and Evaluation (M&E)... 25 Box 13. Monitoring and Evaluation: Peru Programmatic Social Reform Loan... 26 Box 14. Risk Assessment Good Practice... 28 Box 15. Good Practices for Programmatic Adjustment Operations... 30 Annexes Annex A. Programmatic Adjustment Operations... 33 Annex B. IBRD and IDA Lending Commitments, FY90-03: Investment and Adjustment (Programmatic and Nonprogrammatic)... 35 Annex C. Topics Covered in Staff Survey... 37 Annex D. Questionnaire for Client Survey... 39

PROGRAMMATIC ADJUSTMENT LENDING RETROSPECTIVE EXECUTIVE SUMMARY 1. Programmatic adjustment lending supports reforms that involve multiyear policy changes and institution building. It is not a new instrument, but a new approach to adjustment lending that has evolved with changes in the development paradigm and borrowers needs. It was introduced to foster country ownership, to provide reliable financial support for successful programs, and to be flexible in accommodating the uncertainties inherent in medium-term reforms. 2. Purpose and Actions Requested. This retrospective reviews experience with these operations, summarizes lessons, and highlights emerging good practice. It responds to Executive Directors questions about the approach, and Management welcomes their views on the findings. It does not advocate changes in lending policy, nor does it request Executive Directors decision on, endorsement of, or guidance on changes to the policy. A. Experience to Date 3. Programmatic adjustment lending has three distinguishing features: (a) a well-defined medium-term program; (b) a phased series of loans based on completed actions, with notional amounts and dates; and (c) a framework of triggers for moving from one operation to the next. Since the first operation was approved in October 1999, there have been 42 operations (including 11 poverty reduction support credits) through the end of FY03. These comprise 30 programs in 23 countries, and they are less than 5 percent of the Bank s 915 lending operations in FY00-03. Interviews with Bank and borrowers staff have found favorable impressions of the programmatic approach. In the 23 countries where it has been used, it is replacing the traditional approach, accounting for 80 percent of adjustment lending in FY03. 4. Country Context, Content, and Coverage. The programmatic approach has been robust and effective in a range of country and sectoral circumstances: Cost and Quality. The approach compares favorably to other adjustment lending in terms of processing costs and early quality ratings by the Operations Evaluation Department and the Quality Assurance Group. Scope. The scope can be broad or narrow. PRSCs tend to be wide-ranging, multisectoral operations, but many operations, like the Mexico Environmental PSAL, focus on a single theme. Content. The core of most operations has been public administration reforms, particularly public financial management. Other sectoral emphases include health, education, financial sector, and trade and industry. Capacity Building. The programmatic approach has dealt with capacity risks by enhancing capacity in 30 of the first 33 operations. Country Variations. The approach has been used for both strong-performing borrowers and for turnaround cases. By basing lending on prior accomplishments rather than anticipated actions and conditions it supports borrowers as they are building a track-record of successful reforms.

iv B. Program Design Features 5. The programmatic approach is evolving and design features have been improved as lessons are incorporated from earlier operations, both overall and within each programmatic series. 6. Triggers. Programmatic operations contain a set of triggers which form the basis of the Bank s decision to proceed with the subsequent operation. When these triggers are reached they can become the prior actions on which the next loan is based. This design feature, which has emerged as good practice, combines flexibility with the discipline imposed by a coherent medium term framework. Some early programmatic operations had an extensive list (sometimes more than 100) of indicative triggers for the second phase, and then depending on progress specified prior actions before appraisal. More recent operations have reduced the number and focused the triggers. The average number of secondphase triggers declined from 32 per operation in FY00-01, to 10 in FY03. 7. Fiduciary Coverage. All PRSCs and most PSALs/Cs have been supported by a set of core fiduciary analyses that was complete or well under way at the time of approval, and 13 of the first 24 programs have public expenditure, financial management, or procurement as a major theme. 8. Monitoring and Evaluation. Experience suggests the importance of clear and measurable indicators. With outcome and/or progress indicators that can be rated as very clear and monitorable in just over half of the first 24 programs, there is room for improvement. 9. Poverty Links. The Bank developed PRSCs to give predictable, performance-based support for low-income countries poverty reduction strategies. They follow specific guidelines and have emerged as a promising approach for sustained engagement on borrowerdriven, medium-term reforms and capacity building. PSALs/Cs are more diverse; some are comparable to PRSCs in being directly focused on poverty reduction, but others target poverty reduction through improving prospects for economic growth. C. Objectives and Emerging Experience 10. Programmatic adjustment lending addresses three principal objectives. Experience to date suggests that these objectives are being met: Country Ownership. Built on national plans and strategies, and grounded in strong analytic foundations and phased multiyear programs, the approach combines country ownership with systematic implementation. It recognizes actual performance, rather than promises, and thus reduces the tension between country ownership and conditionality. Flexibility. Programmatic adjustment lending supports complex medium-term reforms whose scope, sequencing, and pace are unpredictable. It relies on completed reform actions, rather than future conditions, to accommodate these uncertainties. Triggers have provided a flexible structure to link the series of operations: In second-phase operations most triggers (70 percent) have been met

v as originally written, and often exceeded. Where triggers were not met as anticipated, the Bank chose one or more of several options: where the program remained on track, recognize other comparable achievements as prior actions; reduce the loan amount; or postpone or cancel the operation. Predictable Resource Flows. The programmatic approach provides multiyear support for ongoing reforms. Disbursements can be timed with the borrower s budget and policy cycles. Thus it has provided reliable financing to countries that have maintained reform progress. Six second-stage operations were approved at the time they were expected, and three others were delivered within two months of expectations. Where programmatic operations have been aligned with the country s budget cycle, a virtuous circle of progress assessment, policy dialogue on expenditure priorities, and regular donor support has emerged. However, where reforms have lagged, financing has been postponed: seven second-stage operations have been delayed beyond two months, and five of them have been delayed more than a year. D. Conclusions and Issues Going Forward 11. Early performance indicators are positive, largely because of design features that provide flexibility in program implementation and phase financing to facilitate a stronger focus on results, participation, and harmonization. The programmatic approach has been robust and effective in wide-ranging country and program circumstances, and borrowers and staff have been largely pleased with it. 12. Principal Issues. There are three principal issues going forward: How will emerging initiatives in donor coordination affect programmatic adjustment lending? Programmatic adjustment lending supports reforms designed to strengthen government systems in areas related to harmonization procurement, public expenditure management, and so forth. Early experience in Burkina Faso and Uganda has shown that it can be central in donor coordination activities. Will the programmatic approach deliver better results? The programs are in the early stages of multiyear operations and it would be premature to conclude that the approach leads to, or is associated with, better results. Of particular interest will be the results associated with strengthening capacity. These operations, particularly the PRSCs, are being used in countries that have weaker implementation capacities and that may have weaknesses in the national public expenditure, procurement, or financial management systems. The operations include components to overcome these weaknesses, and the overall results will depend on the success of these components. How widely will the programmatic approach be applied? The approach has been quickly integrated into adjustment lending, rising from six percent of lending approvals in FY00 to 15 percent in FY03. The approach is expected to be used more widely in the coming years as more IDA countries complete their poverty reduction strategies and as donor coordination expands.

PROGRAMMATIC ADJUSTMENT LENDING RETROSPECTIVE I. INTRODUCTION 1. Programmatic adjustment lending is an approach to country assistance first used in 1999 that adapts Bank lending to borrowers changing needs and circumstances. It is designed to provide assistance in pace with a country s accomplishments and aligned with its policy and budget cycle, and to improve development outcomes by fostering country ownership, strengthening fiduciary capacity, and building institutions. Through the end of FY03, the Bank had approved 42 programmatic adjustment operations (see Annex A). This retrospective reviews experience with these operations, summarizes lessons, and highlights emerging good practice. It builds on earlier studies of adjustment and programmatic lending 1 and on interviews with borrower and Bank staff and managers. It is intended to inform the designs of future operations and to contribute to the updating of the Bank s adjustment lending policy. 2. Organization of the Retrospective. Chapter II summarizes experience with the first programmatic adjustment loans and discusses emerging trends. Chapter III examines the country context, focusing on how the principal features of programmatic operations have been applied and adapted. Chapter IV describes issues in designing and implementing the programmatic approach, and Chapter V summarizes conclusions and lessons and discusses next steps. II. INITIAL EXPERIENCES WITH PROGRAMMATIC ADJUSTMENT LENDING 3. This chapter describes the objectives and operational policy of programmatic adjustment lending and reviews the initial experience. In quality and cost, these operations compare favorably with traditional adjustment lending. Given the relatively small number of operations, it is too early to judge whether these promising results will be sustained over time. 4. Evolution of Adjustment Lending. Since adjustment lending was introduced in 1980, it has been used to support countries macroeconomic and sectoral policy reforms. Over the past two decades, in response to borrowers changing needs and broader reform agendas, adjustment lending has evolved. The narrow focus of the 1980s on short-term stabilization and reducing policy distortions gave way in the 1990s to a longer-term developmental perspective, with growing attention to reducing poverty, building institutions, and implementing complex social and structural reforms. Today the approach is driven by country-owned development strategies (set out in a Poverty Reduction Strategy Paper in IDA countries), and most often supports institutional reforms in public sector management and in the financial and social sectors. 1 See Adjustment Lending Retrospective (SecM 2001-0215), June 15, 2001, and Adaptable Lending: Third Review of Experience (CODE 2002-0071), September 4, 2002.

2 A. Objectives and Types of Programmatic Lending 5. Programmatic adjustment lending is an adaptable, medium-term approach, not a new instrument. 2 It involves an integrated medium-term framework of reforms, with notional amounts and dates linked to a country s policy and budget cycle. Within this framework is a planned series of operations, phased to support the country in achieving its reform program, with monitorable indicators of progress and triggers for moving from one operation to the next. Programmatic adjustment lending addresses four principal objectives. Country Ownership. Built on national plans and strategies, and grounded in strong analytic foundations and phased multiyear programs, the approach combines country ownership with systematic implementation. Flexibility. Programmatic lending usually supports complex medium-term reforms whose scope, sequencing, and pace are unpredictable. Each loan under the programmatic approach relies on completed reform actions, rather than future conditions, to accommodate these uncertainties. Predictability of Resource Flows. Programmatic adjustment lending offers multiyear support for ongoing reforms, and disbursements can be timed with the borrowing country s annual budget cycle. Donor Coordination. The programmatic approach can offer a comprehensive platform for coordinating donors support around a country-owned program. 6. Contrasts with Earlier Adjustment Lending. Experience suggests that an exclusive focus on conditionality based on ex ante commitments or ex post results may not be practical or useful for World Bank policy lending. Not all tranching and phasing options are appropriate in every country or sector. One-off, single-tranche operations can be an episodic, unsustained approach to policy reform. Traditional multiyear, multitranche designs have been useful when countries needed to establish a reform track record. Floating tranches can help with discrete reforms with uncertain timing, without holding up the rest of the program. The programmatic approach has been used to provide sustained, flexible, and predictable support to credible medium-term reform programs or strong performers with good track records such as Brazil, Latvia, Mexico, and Uganda. It has also been useful in countries with sound reform programs but weaker capacity or track records, or those emerging from crisis or instability such as Jamaica, Peru, Turkey, and Ukraine. Early experience suggests that the programmatic approach is better suited to foster country ownership, provide reliable support for successful mediumterm programs, and accommodate the uncertainties inherent in medium-term reforms. (Box 1 compares features of programmatic adjustment loans and other adjustment loans.) 2 There can be confusion over the term programmatic. Donors frequently use it to refer to aid that is provided in the form of budget support.

3 Box 1. Programmatic and Other Adjustment Lending: Selected Features One-off single-tranche Multitranche Feature operation adjustment loan Board approval Individual operation Prior to first disbursement Basis of support Prior actions Prior and/or expected actions Medium-term framework and accountability Movement forward in lending program If progress does not follow plans Required, but no accountability No explicit link to subsequent operations No explicit follow-up required Established through tranche release conditions Disbursement on subsequent tranches when conditions are met (or waived) Waiver required Series of programmatic loans Each operation in the series Prior actions Required. Accountability through triggers. When expected actions (triggers) are met or revisions are agreed Adaptable in light of overall progress 7. Two Types of Programmatic Adjustment Lending. The programmatic approach is used in programmatic structural adjustment loans/credits (PSALs/Cs) and in poverty reduction support credits (PRSCs). PSALs/Cs focus on medium-term, step-by-step capacity building and institutional reform. They are intended to strengthen public expenditure management and improve public governance, the efficiency of public resource allocation, and public services, especially those delivered to poor people. PRSCs support IDA countries poverty reduction strategies (PRSPs). They are grounded in specific analytic underpinnings for fiduciary, social, structural, and sectoral reforms. The PRSP/PRSC framework allows donors to support a single program, with consistent and harmonized monitoring and evaluation of results at three levels: the operation, the program, and the country. Over time, PRSCs are expected to become an important vehicle of IDA financial support to low-income countries with strong poverty reduction programs. B. Policy Framework for Programmatic Adjustment Lending 8. Programmatic adjustment lending is subject both to the general policy provisions for adjustment lending (Operational Directive 8.60, Adjustment Lending Policy) and to specific operational guidelines. Sectoral programmatic adjustment operations are also subject to OP 4.01, Environmental Assessment. In addition, Guidelines for Programmatic Adjustment Loans/Credits (February 11, 2000) clarify several features of programmatic adjustment lending, including its justification in the context of the CAS, the focus of PSALs/Cs on capacity building and institutional reform (typically in the public sector), and the specification of the medium-term program and multiyear monitorable indicators and benchmarks. Box 2 outlines the review and approval process for programmatic adjustment operations.

4 Box 2. Review and Approval Processes for Programmatic Adjustment Loans Management Review PSALs/Cs. Each operation is subject to a corporate review, either by a Regional Operations Committee, chaired by a Regional vice president, or by the Bankwide Operations Committee, chaired by the Managing Director, Operations. PRSCs. Proposed PRSCs are subject to a corporate review through the Bankwide Operations Committee. Guidance from the Operations Committee to the task team typically covers the social, environmental, and fiduciary aspects of a proposed PRSC. The Managing Director, Operations, signs off on the final documentation before it is presented to the Board. Board Approval PSALs/Cs. The Board approves each PSAL/C operation under regular procedures. The first PSAL/C in the series is presented to the Executive Directors together with a multiyear framework. The Board discusses and endorses, in principle, the multiyear umbrella program and approves the first PSAL/C. Approval of an individual PSAL/C does not express, directly or indirectly, a commitment by the Bank on subsequent PSALs/Cs in the series. Each subsequent PSAL/C in the series is brought to the Board for separate approval. Amendments or waivers of conditions of tranche release for each individual PSAL/C are subject to Board approval. PRSCs. The program, along with results-focused indicators and progress benchmarks for monitoring implementation of the poverty reduction strategy, is set out in a multiyear matrix of policy and institutional reforms. Each credit in a PRSC series is presented to the Board for its review under regular procedures. After a transition period to build up the pipeline of PRSPs and PRSCs, it is expected that PRSCs will normally be finalized and presented to the Board together with or shortly after the PRSP, Joint Staff Assessment (JSA), and CAS. 9. PRSC Guidelines. The Interim Guidelines for Poverty Reduction Support Credits (May 31, 2001) specify further that each PRSC Draws from and elaborates on the country s PRSP; Draws on an ex ante analysis of the social (including gender) and poverty reduction impacts of the Bank-supported reform program; Includes an assessment of the country s public financial accountability arrangements (covering its public expenditure, procurement, and financial management systems), which could be met by a Public Expenditure Review (PER), Country Financial Accountability Assessment (CFAA), Country Procurement Assessment Review (CPAR), or equivalent; and Includes a cross-cutting assessment of the country s social, structural, and sectoral development policies, covering the policy reform and institutional development priorities for sustainable growth and poverty reduction. This includes enhancing positive and mitigating adverse impacts that the reforms may have on poor people, other vulnerable groups, or the environment. 10. Adjustment Lending Policy Update. The operational policy for adjustment lending is being updated in a new OP/BP 8.60. An issues paper has been discussed by

5 CODE and made public 3 and discussed internationally in extensive public consultations. A new OP/BP 8.60 is being drafted and will be submitted for Board consideration in FY04. It is expected that the provisions for programmatic lending, including PRSCs, will be incorporated in the new policy. The new policy will also spell out the required analytic foundations for policy based lending. C. Early Trends in Programmatic Adjustment Operations 11. The first PSAL was approved by the Board of Executive Directors in October 1999; and the first PRSC in May 2001. As of June 30, 2003, there have been 30 programs in 24 countries, comprising 42 operations (30 first-stage operations and 12 second-stage operations), of which 11 operations have been PRSCs. 4 Preliminary indications are that these operations compare favorably with traditional adjustment lending across the dimensions of preparation time, costs, and quality. 1. Quantity 12. The number and volumes of programmatic adjustment operations have been small, but they are growing even while total adjustment lending has remained relatively stable at about one-third of Bank lending volume. (Figure 1 and Annex B provide details.) The four programmatic adjustment operations approved in FY00 were 2 percent of the 223 operations approved. In the next two fiscal years, seventeen more programmatic adjustment operations were approved, and 21 operations were approved in FY03 the same number as in the previous three years combined. Figure 1: Lending Approvals Number and Volume, FY00-03 25 Lending Volume, $bn 20 15 10 5 0 $15.3 bn 223 operations $17.3 bn 225 operations $19.5 bn 229 $18.5 bn operations 238 operations $3.5 bn $2.8 bn $2.1 bn $0.9 bn 8 4 operations 9 operations 21 i FY00 FY01 FY02 FY03 All Lending PSAL/PRSC 3 4 See From Adjustment Lending to Development Policy Support Lending: Key Issues in the Update of World Bank Policy (CODE2002-0036/1), June 10, 2002. This report is based on approvals through the end of FY03. However, most analyses in Chapters III and IV used an earlier cut-off date the third quarter of FY03. At that time 33 operations had been approved by the Board of Executive Directors (see Annex A) which are referred to in this report as the first 33 operations.

6 13. Commitments. Over $9 billion has been committed for programmatic adjustment operations: $7.3 billion by IBRD and $2.0 billion by IDA. The annual volume has grown from $882 million in FY00 to more than $2 billion in FY01 and $3.5 billion in FY02. There has also been growth in relative terms: programmatic operations accounted for 6 percent of $15.3 billion in new commitments in FY00 (17 percent of $5.1 billion in adjustment commitments); in FY01 their share rose to 12 percent and 36 percent, respectively, and in FY02 to 19 percent and 38 percent. 14. Loan Sizes. The average amount of these operations has been $222 million, although the averages are skewed by two large loans to Turkey in FY02. In FY03, the average loan was $135 million, as the programmatic approach was extended with smaller loans or credits to new countries (Ecuador, Ghana, Guyana, Nicaragua, Sri Lanka). 15. Lending Trends. Overall lending trends (see Annex B) show that the programmatic approach is being widely used and accounts for a growing share of overall Bank lending. During the period of its use, the programmatic approach has gained in volume, even while total adjustment lending has remained relatively stable at about onethird of Bank lending volume. Adjustment lending was roughly constant in FY00-03 33 percent of Bank approvals in each year except FY02, when it was 50 percent (see Annex B, line 6). This is an increase from the 1990s, when it averaged 29 percent. Programmatic adjustment lending rose from 6 percent of lending volume in FY00 to 15 percent in FY03 (see Annex B, line 8). This increase corresponds to declines in traditional adjustment lending, as the share of adjustment lending remained unchanged. In the 23 countries where the programmatic approach has been used, it now accounts for the bulk of adjustment lending. It rose from 19 percent of adjustment approvals in these countries in FY00 to 80 percent in FY03 (see Annex B, lines 10-12). These 23 countries made greater (but variable) use of adjustment lending than other countries, between 41 percent and 56 percent of approvals during FY00-03 (see Annex B, line 15). There are 119 countries where the programmatic approach has not yet been used. These countries, representing 55 percent of total FY03 lending volume, use less adjustment lending 24 percent of the FY00-03 lending volume (see Annex B, lines 18-22). 16. Factors in the Trends. The traditional and the programmatic approaches to adjustment lending can each be effective in different circumstances. Traditional adjustment loans are useful to support programs of macro-stabilization or structural reforms aimed at removing policy distortions. Typically, they are episodic, and the financial support needs that give rise to them may have been unforeseen. The programmatic approach supports medium-term reforms and capacity building, grounded in strong analytic foundations. In these programs, policy and fiduciary environments are

7 more important, the financing needs can be foreseen in the development programs, and the underlying analytic work can be used to derive better monitoring indicators. These lending trends (also evident in the FY04 pipeline) probably reflect the growing emphasis in the Bank and among an increasing number of its borrowers on the types of policybased sectoral reforms that can best be supported by adjustment lending and within adjustment lending by the borrower-driven, medium-term reform programs that are well suited to the programmatic approach. 17. CAS Envelope. Programmatic adjustment lending has averaged about half of a country s CAS envelope. The size of a first-phase programmatic operation relative to a country s annual CAS envelope (excluding subnational operations in India) averaged 46 percent, ranging from 12 percent for the Mexico Environment PSAL to 120 percent for the Jordan PSAC and the Guyana PRSC. 5 18. Program Details. Most programs or CASs set out both the number of operations and the amount of support in the overall program. Only four operations Jordan and the three subnational operations in India gave no clear indication. Some programs, including all PRSCs, did not specify duration, but used a three-year rolling approach, with each stage articulating the program through the next three years. 19. Regional Distribution. The 30 programs span all six Regions, with LCR accounting for the largest number (13), followed by ECA and AFR (5 each), and SAR (4), EAP (2), and MNA (1). LCR has five programs in the second phase, while AFR and ECA each have two, and MNA, SAR, and EAP each have one. 20. Preparation Times and Costs. Although programmatic operations have been prepared in difficult policy environments, by large teams, relying on extensive analytic foundations, average preparation times and costs have not been substantially greater than those for other adjustment lending. The time from project concept to approval for programmatic operations has averaged 270 days, about the same as for other adjustment lending (272 days) in FY00-02. 6 The preparation costs for PSALs/Cs have averaged $385,000; those for PRSCs, with their more rigorous analytic underpinnings, $453,000. These compare to the average of $352,000 for other adjustment lending. However, these costs do not include two factors mentioned by most task managers as sources of additional effort higher visibility during internal reviews and rigorous analytic foundations. The first of these factors is likely to diminish with time, as programmatic operations become more common and processing requirements and expectations become clearer; the second is not. Credible Bank programmatic lending requires analytic understanding of the relevant sectoral and cross-sectoral issues. Several task managers emphasized the necessity of ongoing research in countries with programmatic operations, particularly where, as in Burkina Faso and Uganda, elements of the development agenda that were managed through investment operations are being incorporated in PRSCs. It is 5 6 The average CAS envelope was calculated as the total envelope divided by the number of years it is meant to cover, hence a percentage above 100 is possible. This result holds even when programmatic operations (most of which are single tranche) are compared only to other FY00-FY02 single-tranche adjustment loans. The averages exclude the Vietnam PRSC, which was not begun as a PRSC and was a high-side outlier in terms of cost and time.

8 premature to say whether these results can be sustained as more programmatic operations are developed, though it is reasonable to expect that, within a program, time and costs will drop in successive phases as experience is gained. 2. Quality 21. The Bank s Quality Assurance Group (QAG) in its fourth and fifth Quality-at- Entry Reviews (QAE4 and 5) found that programmatic adjustment operations are as good as, or better than, other operations. Each of the seven operations 7 it assessed was rated satisfactory or better, in line with the overall result of 92 percent satisfactory or better for all adjustment lending (in QEA5). The Karnataka Economic Restructuring Loan was one of only three adjustment operations rated highly satisfactory, and it was cited as an example of good practice for environmental due diligence in adjustment lending and privatization in QEA4. The Albania PRSC was one of four operations rated highly satisfactory in QEA5, and was particularly praised for an exemplary consultative process. In addition, QAG judged each of the seven operations as likely to meet all stated objectives, and it considered six of them likely to obtain sustainable, long-term results a rating achieved by 91 percent of adjustment operations in QAE4 and 54 percent of operations in QEA5. All seven were rated as satisfactory with respect to assessing and managing risks, compared to a Bankwide satisfactory rate of 85 percent. Furthermore, all seven included institution and capacity building as a priority and were judged as likely to achieve that objective; for adjustment operations as a whole, 90 percent had institutional development and capacity building as an important goal and 78 percent were judged likely to achieve it. 22. QAG Recommendations. QAG recommendations included adopting clearer, monitorable progress indicators, and outcomes that focused on tangible results rather than process steps. Reviewers noted the importance of specifying a well-conceived, multiyear reform, rather than simply describing the beginning and end points of the process. There have been some improvements in these aspects in more recent operations. 23. Project Supervision Ratings. All 33 programmatic adjustment operations have Project Supervision Report ratings for development objectives and implementation progress, and all were rated satisfactory or better on both dimensions. These ratings were achieved in challenging environments: 27 operations were rated as having substantial risk, and 2 (Bolivia and Turkey) as having high risk. 24. Implementation Completion Reports. The implementation completion reports (ICRs) have rated eight first-phase operations 8 as satisfactory or better on outcome, Bank performance, borrower performance, quality at entry, and likely sustainability. The ratings have been confirmed by the Bank s Operations Evaluation Department (OED). By comparison, 77-84 percent of adjustment operations had similar results in FY00-01. Five 7 8 Albania PRSC; Brazil Fiscal Reform PSAL; Burkina Faso PRSC; India Karnataka Economic Restructuring Loan/Credit; Jamaica Bank Restructuring and Debt Management Loan; Jordan Public Sector Reform Loan; and Turkey PFPSAL II. Two subnational operations in India, and PSALs in Jamaica, Latvia, Peru, Thailand, and Brazil s Fiscal and Financial Sector loans.

9 of the eight programmatic operations evaluated were seen as having substantial institutional development impact, compared to 56 percent for adjustment lending in general. 25. Simplification of ICRs. Programmatic lending is designed to produce results in the medium-term, and after a series of lending operations. The Bank s ICR system evaluates each operation at its completion, rather than at the end of a sequence of operations. Management is working with OED to simplify the ICRs for programmatic lending to accommodate the intrinsic differences in the programmatic approach and to allow for more seasoned judgments of outcomes and sustainability. 26. Bank and Borrower Staff Views. Bank and borrower staff view the programmatic approach positively. Interviews with Bank task managers (see Box 3) and borrower staff (see Box 4) elicited these impressions: Enhanced Flexibility. In the context of a medium-term, step-by-step government program, the programmatic approach required detailed planning of the sequencing and timing of reforms and allowed for flexibility in the face of uncertainty about the pace of reforms. In some cases, attention to monitoring indicators helped the Bank and its borrowers to improve the program and to link it better to governments policy objectives (Albania, Peru). Less Burdensome Political Constraints. The programmatic approach facilitated reforms and dialogue on issues that transcended national political cycles, and it imposed a policy discipline in some countries with frequent or impending changes of government or high turnover of staff and management in government ministries (Albania, Jordan, Latvia, Peru, Thailand, and the states of India). Better Ownership. In many cases, as with the poverty reduction strategies that underlie PRSCs, the government articulated and published the program before the Bank s operation (Bolivia, Burkina Faso, Latvia, Peru, and Uganda). Donor Coordination. The PRSP process is based on coordination and consultation among all stakeholders, domestic and foreign, but in some programmatic operations, the government and donors have agreed on the program and used it to consolidate support (Bolivia, a CDF pilot country). Risk Management. Risk management in programmatic operations has been enhanced by focusing on results and measuring support against progress. When there is uncertainty about the government s ability to execute reforms, the programmatic approach keeps Bank support aligned to the extent and the pace of reforms (India, Jamaica, Tanzania, and Ukraine). Drawbacks. Because the operations tend to be multisectoral, involve extensive institution building and capacity building, and require comprehensive analytic

10 foundations 9 and collaboration among all stakeholders they have been harder to prepare and to monitor, both for the Bank and borrowers. Box 3. Staff Views of the Programmatic Approach Interviews with 25 Bank managers and staff, following the interview protocol in Annex C, revealed nearly unanimous favorable impressions of the programmatic approach. In several operations, staff expressed the view that the approach led to greater country ownership and better-designed programs. They cited the programmatic approach itself as best suited for the program it was supporting as well as building good team dynamics within and among the Bank and their counterparts. They felt that developing the analytic underpinnings was time-consuming and the requirements were high, but no one thought that they were excessive. In terms of program scope and design, the advice was to be selective and to have a small number of outcome-oriented triggers. The down-side of the programmatic approach included complaints that the processing steps were too hard. There were too many requirements social and environmental and a lot of extra work for something that could have been equally achieved in a normal adjustment operation. There were concerns about managing team dynamics and the risk that the requirements tend to increase the size of the policy matrix. Some highlights from the interviews: Task managers like the instrument o The perfect solution for specific circumstances. If you want to do development, there is no other instrument. o Overcomes some problems with conventional adjustment lending: timing, flexibility, lack of reform track record; match the lending to the amount of actual reform. o Highlights up-front accomplishments. By building in flexibility, the programmatic approach is better able to cope with both capacity problems among borrowers as well as political economy constraints. o It s useful to have a lending arrangement that does not go into a legal and adversarial mode when complications occur or program targets slip. o Programmatic lending is to other adjustment lending as the carrot is to the stick. ESW is essential identify and explore policy options o Only convince African governments through ESW, policy dialogue. o It allows for the necessary ESW to be funded and conducted and forces a continuing dialogue as compared with multiple tranche operations in which supervision missions are largely concerned with checking to see if tranche conditions have been met. o It leads to cooperation and supports flexibility between the Bank and the borrower. Sectoral colleagues resisted the instrument but came around o They would have preferred to do an endless stream of APLs. o Sector guys showed up with a long list of conditions to discuss with the government, but you can t impose your will on these people. o Advantage: You can discuss policy with government officials instead of clearing letters of no objection. Supervision and preparation of follow-on operations merge o Monitoring is continuous. o Monitoring is more complex because of the nature of the program. o It s harder work to prepare, for the government and the Bank, because it s more complex big teams, multisectoral, new way of doing things. Some requirements are too onerous, and do not make sense in light of the multi-year nature of the approach. o The requirement of an ICR after each operation is not meaningful for a multi-year program: the evaluation should be done no earlier than after three operations, or at the end of the program. o Could the Board approve the multi-year framework and delegate the decision to proceed with subsequent operations to management, as they do with multi-tranche operations. o It s too comprehensive with too many requirements (environment, fiduciary) that should be dealt with at the country level. 9 Although the analytic requirements have been high, Bank staff and borrowers typically would have preferred more analysis, rather than less.

11 Box 4. Borrower Feedback on the Programmatic Approach Borrower feedback was obtained largely with the assistance of Bank field staff, and only a limited sample (16 countries) participated. Staff were asked to use the interview protocol shown in Annex D. Feedback was generally positive, with many borrowers pleased by their ability to control the design of their own programs. They said the process encouraged interministerial work and beneficial dialogue with civil society, which improved the government program. The comprehensive reform policy and programs developed under the operation created a clear road map of reforms and helped create understanding of reforms among various stakeholders. Having the Bank as a partner added credibility to the government program. Although triggers were time-consuming to design, they served to strengthen commitment and ensure progress. Mutually agreed milestones helped maintain momentum and facilitated debate on government objectives. Some borrowers suggested that triggers need to be measurable, unambiguous, and within government control; some expressed concern that too many triggers weakened the more important ones. Some countries appreciated better monitoring because it helped to clarify outcomes and program design. Borrowers complained about inconsistency of Bank processes between country offices and Washington, and suggested that to increase transparency, funding from the Bank should be more predictable and synchronized with the budget cycle. There were also some concerns about articulating risk factors and considering back-up options as well as ensuring that program design is not too ambitious for national circumstances. Some highlights from the interviews: Borrowers like the flexibility of the approach. o The programmatic approach allows governments to set their priorities; it increases country responsibility. o Programmatic operations adapt to changing government priorities. o The up-front actions fostered dialogue with civil society and improved the overall dialogue with the government. Clear message on triggers: focused and fewer - In the end it is the quality and not the quantity of conditions that contribute to the effectiveness of the operation o Triggers need to be measurable, unambiguous, and within government control. o Too many triggers weakened the more important ones. o Triggers help clarify a realistic commitment and ensure progress. Bank performance room for improvement o The operation took a short time to prepare, the Bank was responsive and moved fast. o The Bank team was highly qualified, flexible, technically strong, and responded to the government. o To increase transparency, funding from the Bank needs to be more predictable and synchronized with the budget cycle. o Bank processes were inconsistent between country office and Washington. Going forward o Each measure proposed needs to be realistic, recognizing its true costs and efforts involved, including low government capacity in certain countries. o More needs to be done to ensure that milestones have sufficient checks and balances so that they cannot be reversed. o All risk factors should be articulated and back-up options considered. To increase the likelihood of implementation, the government should make a realistic assessment of the cooperation with Parliament in terms of the legislative part of the work.

12 III. COUNTRY CONTEXT, CONTENT, AND COVERAGE 27. This chapter focuses on the operational content of the programmatic adjustment loans in relation to countries development needs. It reviews the use of analytic underpinnings in these operations, sectoral coverage and poverty impacts, fiduciary and safeguard issues, and collaboration with the IMF and other development partners. A. Country Context and Ownership 28. Task teams rate country ownership of programmatic operations as strong. Several aspects of the programmatic approach foster and support ownership: (a) developing the PRSP that underpinned the PRSC; (b) mapping out the medium-term frameworks in PSALs/Cs and translating them into actions, program stages, and triggers; (c) delivering support through the government s budget instead of an off-budget entity; and (d) arranging monitoring and evaluation so that the government takes ownership of outcomes and can adopt policies and measures to deliver them. 29. Links to CAS. Programmatic operations have had strong links with CASs. Nearly all operations were in a prior CAS or were presented to the Board with a new CAS. PRSCs (and CASs) have been based on the countries PRSPs, thus aligning the Bank s support to the government s program. The PRSC has been the Bank s principal instrument for supporting the government s poverty reduction strategy in every country that has a PRSC except Albania (where the CAS supports elements of the PRSP with other instruments). 30. Adaptability to Country Reform Record. The programmatic approach has been adapted to countries with strong or weak reform records. Weaker performers benefit because the approach is based on prior actions: the loan is approved only when there are sufficient accomplishments to warrant financial support, and with a series of singletranche operations, there is no risk that the government will not fulfill conditions. By contrast, in some countries with strong reform records, Bank staff report that officials have welcomed the flexibility inherent in the programmatic approach and have used it to design bolder multiphase programs, knowing that programmatic lending can adapt to the pace and strength of accomplishments. Officials contrast this with conditions in multitranche operations, which are more problematic because they are restrictive and could be politically sensitive if they are disclosed. B. Analytic Underpinnings 31. There are no ESW policy requirements for adjustment lending, but PRSCs require integrative assessments of (a) the country s social, structural, and sectoral development policies, and (b) the country s public financial accountability arrangements, covering its public expenditure, procurement, and financial management systems. These draw on poverty assessments, public expenditure reviews, country procurement and financial accountability reviews, environmental and social analysis, or other work by the Bank, the borrower, or development partners. Assessments may be prepared collaboratively with the borrower or other stakeholders to build analytic capacity and policy ownership.

13 32. Core ESW. Two-thirds of the countries with programmatic operations had the core ESW 10 ready or well under way when the first operation was designed. In addition, many programs benefited from analyses by local authorities, scholars, or research organizations. Sectoral programs, such as financial reforms in Jamaica and Turkey or the environment PSAL in Mexico, were based on sectoral analyses in addition to the core diagnostics. ESW in the form of seminars and workshops has been used widely to broaden program support among stakeholders. C. Sectoral Coverage 33. The focus of most programmatic adjustment lending has been public administration reforms (19 of 23 first-phase operations including all PRSCs), particularly public financial management. Programmatic adjustment operations have also focused on reforms in health and education, the financial sector, and trade and industry. Financial sector reform has been important in PSALs/Cs, but it has been insignificant in PRSCs (except for the Vietnam PRSC), which have given greater emphasis to social sectors (health and education). The share of PRSC conditions devoted to health, education, and social protection was almost five times greater than in other adjustment lending to IDA countries. The sectoral focus of PSALs/Cs tends to be narrower than that of PRSCs. On average, PSALs/Cs cover four sectors, with more than 85 percent of conditionality in a principal sector. 34. Sectoral Versus Multisectoral Focus. Programmatic adjustment lending has been used both for sectoral reforms and for cross-cutting, multisectoral programs. Thus it has the potential to subsume stand-alone sectoral projects and to strengthen support for the system-wide policy and institutional reforms that are critical to reaching the MDGs. A sectoral project can, at best, improve outcomes in its particular sector and it cannot by itself create systemic improvements such as public expenditure management reform. However there are limits to what can be accomplished. Not all the deep institutional reforms that are needed in each sector can be accomplished through a programmatic adjustment loan and investment projects may provide necessary incentives and technical assistance. There can be tensions (see Box 3) between those who would prefer sectorally oriented investment loans and those who recommend the more encompassing reforms, but these can only be resolved on a case-by-case basis, in light of the country s needs, priorities, and circumstances in each country s CAS. D. Poverty Impacts 35. PRSCs are models for addressing poverty reduction. The Bank developed PRSCs as its central approach to supporting low-income countries poverty reduction strategies, which are developed through a participatory process involving all stakeholders, civil society, and development partners, including the World Bank and IMF. In some 10 The Bank is aiming to complete the core ESW for all active countries by the end of FY04. Core diagnostics comprise four analyses: (a) Country Economic Memorandum/Development Policy Review, (b) Public Expenditure Review, (c) Country Financial Accountability Assessment, and (d) Country Procurement Assessment Review.