Document of The World Bank IMPLEMENTATION COMPLETION AND RESULTS REPORT. (Trust Funds-TF090670) ON A GRANT IN THE AMOUNT OF US$1.

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Document of The World Bank IMPLEMENTATION COMPLETION AND RESULTS REPORT (Trust Funds-TF090670) ON A GRANT IN THE AMOUNT OF US$1.0 MILLION TO THE LEBANESE REPUBLIC FOR THE Report No: ICR EMERGENCY SOCIAL PROTECTION IMPLEMENTATION SUPPORT PROJECT Human Development Sector Middle East and North Africa Region March 31, 2010

2 CURRENCY EQUIVALENTS (Exchange Rate Effective December 31, 2009) Currency Unit = Lebanese pound LP 1.00 = US$ US$1.00 = LP 1,507 FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS CAS DG EOSI ESIA ESPISP ESPISP II GDP GOL HROST ISN IST MENA MIS MOF MOPH MTBF NAPS NPTP NSSF PEM PMT RIDPL SAP SDR TA TFL UNDP Country Assistance Strategy Director General End-of-Service Indemnity Program Economic and Social Impact Assessment Emergency Social Protection Implementation Support Project Second Emergency Social Protection Implementation Support Project Gross Domestic Product Government of Lebanon Health Reform Options Simulation Tool Interim Strategy Note Implementation Support Team Middle East and North Africa Management Information System Ministry of Finance Ministry of Public Health Medium Term Budget Framework NSSF Active Patient System National Poverty Targeting Project National Social Security Fund Public Expenditure Management Proxy Means Testing Reform Implementation Development Policy Loan Social Action Plan Special Drawing Rights Technical Assistance Trust Fund for Lebanon United Nations Development Program Vice President: Country Director: Sector Director: Sector Manager: Task Team Leader-ESPISP: Task Team Leader-ICR: ICR Primary Author: Shamshad Akhtar Hedi Larbi Steen Lau Jorgensen Roberta Gatti Haneen Sayed Surat Nsour Richard Carroll ii

3 LEBANESE REPUBLIC EMERGENCY SOCIAL PROTECTION IMPLEMENTATION SUPPORT PROJECT (ESPISP) CONTENTS Data Sheet A. Basic Information... i B. Key Dates... i C. Ratings Summary... i D. Sector and Theme Codes... ii E. Bank Staff... ii F. Results Framework Analysis... ii G. Ratings of Project Performance in ISRs... vi H. Restructuring (if any)... vi 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1 Project Costs and Financing Annex 2 Outputs by Component Annex 3. Economic and Financial Analysis Annex 4 Bank Lending and Implementation Support/Supervision Processes Annex 6. Stakeholder Workshop Report and Results Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Annex 9. List of Supporting Documents available in the project portal Map iii

4 A. Basic Information Country: Lebanon Project Name: Lebanon Emergency Social Protection Implementation Support Grant Project ID: P L/C/TF Number(s): ICR Date: 03/31/2010 ICR Type: Core ICR Lending Instrument: ERL Borrower: Original Total Commitment: Revised Amount: USD 0.0M Environmental Category: C Implementing Agencies: World Bank Cofinanciers and Other External Partners: GOVERNMENT OF LEBANON USD 0.0M Disbursed Amount: USD 0.0M B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: Effectiveness: 09/06/ /06/2007 Appraisal: Restructuring(s): Approval: 09/07/2007 Mid-term Review: Closing: C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Moderately Satisfactory High Moderately Satisfactory Moderately Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Moderately Quality of Supervision: Satisfactory Agency/Agencies: Unsatisfactory Overall Bank Performance: Overall Borrower Moderately Satisfactory Performance: Moderately Unsatisfactory i

5 C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project Yes at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: No Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health Other social services Theme Code (as % of total Bank financing) Health system performance Social safety nets E. Bank Staff Positions At ICR At Approval Vice President: Shamshad Akhtar Daniela Gressani Country Director: Hedi Larbi Joseph P. Saba Sector Manager: Roberta V. Gatti David J. Steel Project Team Leader: Haneen Ismail Sayed Haneen Ismail Sayed ICR Team Leader: ICR Primary Author: Surat F. Nsour Richard J. Carroll F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective of the Emergency Social Protection Implementation Support Project is to accelerate and improve the quality of the implementation of the package of social sector reforms presented by Lebanon at the Paris III donor conference in the areas of social insurance, safety nets, and health expenditures. This will be achieved through the provision of implementation support and capacity building for the National Social Security Fund (NSSF), Ministry of Social Affairs (MOSA), and Ministry of Public Health (MOPH). Over the medium term, the set of reforms supported by this Project will be expanded in order to ensure the transition to a social protection system that provides ii

6 adequate income protection and security to all population groups, while ensuring financial sustainability and equity, and minimizing economic distortions. Revised Project Development Objectives (as approved by original approving authority) No revisions (a) PDO Indicator(s) Indicator Indicator 1 : Value quantitative or Qualitative) Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years The new systems targeted by this operation are put into place and are functional according to specificiations. Systems to be Systems are developed. New systems not yet functional, but not Systems to be developed. yet put into place in operational in 1-2 any hospitals. hospitals Date achieved 09/30/ /30/ /30/2009 Comments (incl. % Mostly achieved. achievement) Indicator 2 : Essential components of the overall social sector reform strategies are further developed Value See intermediate See intermediate quantitative or None indicators indicators Qualitative) Date achieved 09/30/ /30/ /30/2007 Comments (incl. % achievement) Mostly achieved. However, this "outcome indicator" is only a description of an output. ICR must infer benefits from completed actions as there are no specific outcome indicators. (b) Intermediate Outcome Indicator(s) Indicator Indicator 1 : Value (quantitative or Qualitative) Date achieved Comments Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Complete comprehensive review of hardware and software requirements for the new claims and utilization control system, and the new register for plan members and contribution collection system to ensure consistency with NSSF reforms. Mostly achieved. All activities were completed except, by mutual agreement of iii

7 (incl. % achievement) Indicator 2 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 3 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 4 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 5 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 6 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 7 : NSSF and the Bank, the new register for plan members and the contribution collection system. (see Section 1.6) Complete design of software requirements for the claims and utilization control IT system, the new register for plan members and the new contribution collection system. Mostly achieved. Similar to previous indicator, activities completed except those relating to the register for plan members and the new contribution collection system. Development of training plan and completed training of staff on new functions and responsibilities stemming from the business process reengineering of claims processing, and administration of registers of plan members and contributors. Mostly achieved. Similar to previous indicator, activities completed except those relating to the register for plan members and the new contribution collection system. New guidelines and admission protocols are defined and issued. Achieved. Unified identification mechanisms for plan members and contributors operational. Not done. These activities have been picked up in the follow up project, ESPISP 2 Targeting formula developed and approved. Achieved. A Proxy Means Testing formula was developed and approved Operational manual for implementation of targeting completed iv

8 Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 8 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 9 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 10 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 11 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Indicator 12 : Value (quantitative Achieved. This manual was completed and used in the pilot. MIS to support the pilot developed and tested. Achieved. Utilization review (UR) function is introduced in the MOPH which can oversee and implement more precise admission criteria to reduce unnecessary admissions (paid by MOPH) to public and private facilities. Not achieved. The utilization review function has been introduced in the sense that there is a UR coordinator. However, the function has not been institutionalized. Enhanced IT module developed and implemented within the current MOPH IT system providing more precision and better tracking mechanisms for identifying unnecessary admissions. Not achieved. This activity has not been completed and progress remains very slow. 30 Admission Criteria/Protocols agreed and established by the provider community and the MOPH. Not achieved. This activity has not been completed and progress remains very slow. Physicians trained in new criteria to encourage better management and quality of care. v

9 or Qualitative) Date achieved Comments (incl. % achievement) Indicator 13 : Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Not achieved. This activity can be completed only when the previous activity has been completed. Completed plan to realign MOPH coverage towards more cost-effective health services and hospital facilities. Partially achieved. MOPH Committed adopted and implemented the early stages of a roadmap to introduce performance-based contracting for hospital services. G. Ratings of Project Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD millions) H. Restructuring (if any) Not Applicable I. Disbursement Profile vi

10 1. Project Context, Development Objectives and Design 1.1 Background and Context at Appraisal 1. Lebanon has a population of 4.1 million, of which 87 percent live in urban areas. Lebanon is also a middle income country with a GNI per capita of US$5,770 (2007). GDP growth was 2.0 percent in 2007 and is estimated to average around 4.0 percent for the period The country is heavily indebted with the present value of debt to GDP at percent in 2006, which makes the success of the Emergency Social Protection Implementation Support Project (ESPISP) especially important because it aims to improve the efficiency of public expenditures. 2. The ESPISP Project Document reported that, at the time of appraisal in early to mid-2007, and well before that, Lebanon s social insurance and safety net systems were weak. Lebanon did not have a pension system for private sector workers (rather an End-of-Service-Indemnity, EOSI). The health insurance system was fragmented and inefficient. Half of the population had no formal insurance and relied on Ministry of Public Health (MOPH) benefits for in-patient coverage. The program document described formal safety nets as small, scattered, and not welltargeted. The country relied on its strong confessional-based network of social services (NGOs, welfare organizations, remittances, etc.) which played a safety net role, but which were also vulnerable to economic or political shocks. The lack of an adequate safety net meant that the GOL did not have the necessary tools to help absorb shocks that might result from economic adjustment measures. In addition, prior to 2006, the GOL did not have a strategy or plan in place to address the above issues. 3. The Government was beginning to address social issues in a comprehensive manner through its Social Action Plan (SAP) of June 2006, as well as individual sector strategies, when hostilities broke out between Lebanon and Israel. The July 2006 hostilities took a devastating toll on the population and the economy, reversing the positive economic trend in the first half of 2006 and further straining the weak social protection system The GOL quickly resumed the agenda once the hostilities abated. The creation of an Inter-Ministerial Committee for Social Policy (IMC-SP) was an important step in this direction. For the previous four years, consecutive Lebanese governments had been advancing the pension reform agenda, which led to a draft pension law which was (and still is) pending in Parliament for enactment. The draft law, once enacted by Parliament, would provide Lebanon with a fullyfunded defined contribution pension system. 5. Social protection reforms were part of the broader program of reforms presented to the international community at a donor conference held in Paris (Paris III) on January 25, The 2 Over 1,000 people were killed, 5,000 injured and close to 1 million (a quarter of the population) internally displaced. Additionally, brain drain accelerated as more than 200,000 people left the country. The Lebanon Economic and Social Impact Assessment (ESIA) estimated total direct damage inflicted by the hostilities at US$ billion, with a further US$ million in indirect damage. Approximately, 30,000 jobs were permanently lost, adding 3 percent to the unemployment rate. 1

11 ESPISP can be viewed as a small but critical part of the broader, medium-term social protection reforms, which pursued six objectives: 1. Preparation of a policy framework for social insurance and coverage expansion; 2. Improvement in the governance and administration of the NSSF; 3. Reforms to guarantee the medium- and long-term sustainability of the NSSF; 4. Reforms to strengthen income protection programs (pensions and family allowances); 5. Establishment of a social assistance system that is able to target safety net programs to the most vulnerable population groups; and 6. Controlling costs in MOPH. 6. These reforms would help Lebanon in its transition towards a social protection system that provides adequate income protection and security to all population groups, while ensuring financial sustainability and equity, and minimizing economic distortions. Table 1: Timeline of Activities, Events, Documents Relevant to ESPISP Activity/ Event/Document Date Social Action Plan (SAP) June 2006 Hostilities with Israel July-August 2006 Lebanon Economic and Social Impact Assessment (ESIA) December 2006 Trust Fund for Lebanon (TFL) established September 2006 Inter-ministerial Committee for Social Policy (Social-IMC) December 2006 established Paris III Presentation of GOL Development Strategy to donors January 2007 Social Action Plan adopted as part of GoL Program in Paris III January 2007 Interim Strategy Note Approved by Bank Board July 2007 ESPISP I Effective September 7, 2007 Closed April 30, 2009 Reform Implementation DPL (RIDPL) Effective September 27, 2007 Closed January 31, 2009 ESPISP II Effective August 2008 Closing date: March 31, 2012 RIDPL II Concept Note Project Withdrawn 7. As laid out in the Interim Strategy Note (ISN), the Bank supported the GOL reform program with the Reform Implementation Development Policy Loan (RIDPL), which focused on energy reforms, as well as early actions in the areas of social protection and business development. The ISN placed a high priority on the social protection aspect of the loan, stating, It is particularly important that the wide-ranging structural reforms that Lebanon needs to undertake be facilitated with first addressing the currently inefficient social protection agenda. The US$100 million RIDPL was presented to the World Bank Board of Executive Directors on August 2, It was the first time that Lebanon agreed to a policy-based loan from the World Bank. Historically, the GOL had eschewed policy-based operations, partly because of the challenges posed by conditionality and partly because of budget support that was forthcoming from the Gulf States. 3 RIDPL II (initially called the Social Protection Development Policy Loan SP DPL), as described in the ISN, was to be the second stand alone DPL supporting the GOL s reform program. It did not proceed because of lack of progress in the overall Paris III reform program. 2

12 8. The social protection reform agenda built upon the SAP adopted by the Lebanese Government and upon the analysis and recommendations outlined in the Economic and Social Impact Assessment (ESIA) carried out by the World Bank in the aftermath of the hostilities. The ESIA covered health, education, pensions, social safety nets, and labor markets. The focus of the ESPISP grant was on social insurance (pensions and health insurance), social safety nets and health.4 9. There had been a lack of action in reforming social sectors in the past, particularly social protection, and this has been well-documented in Bank analytical and evaluative work. The ESIA stated that The development of an effective safety net in Lebanon has been hampered by the absence of a social policy framework, weak coordination and governance, politically-influenced targeting and lack of reliable data (for poverty and also for the outcomes of interventions, including ex-post evaluations). The high degree of political factionalism and difficulties in building sustainable capacity within the Government have been particularly hindering to a functioning social protection system. After years of relying on in-house units of consultants in the ministries, the Government showed little ability to implement outside of the enclaves of consultants. The ESPISP grant was launched in this challenging environment. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 10. The overall objective of the ESPISP was to accelerate and improve the quality of the implementation of the package of social sector reforms presented by GOL at the Paris III Donor Conference in the areas of social insurance, safety nets, and health expenditures. This objective was to be achieved through the provision of implementation support and capacity building for the NSSF, MOSA, and MOPH. The ESPISP supported the development and implementation, over the short term, of key new processes and policies in the three agencies. 11. The project s specific objectives were to: 1. improve the control of expenditures and revenues within the NSSF, as well as the quality of services, by setting up an electronic claims processing system, a utilization control system, and a registration and contributions collection system; 2. ensure financial sustainability of the NSSF by identifying the needed set of reform measures to control expenditures and optimize revenues; 3. improve the poverty focus of existing or new safety net programs in MOSA by developing and piloting a new targeting mechanism; and 4. control health expenditures by the MOPH through establishing a utilization review function, developing hospital admission criteria and protocols, implementing hard ceilings on contracts with private hospitals, and realigning MOPH coverage towards more cost-effective health services and hospital facilities. 4 The education sector reform was not included in ESPISP I because there was an on-going Bank-financed project (the Education Development Project) which was supporting reforms in the sector. The Bank was ready to reallocate from the EDP to support the post-2006 war recovery in the education sector (particularly to help reconstruct destroyed schools) and proposed it to the GOL. However, the GOL was fortunate to receive significant grant financing, particularly from Arab donors, for school reconstruction and rehabilitation. 3

13 12. These objectives were to be achieved through the provision of implementation support to the three entities with a focus on the NSSF (70 percent of the funds). See Section 3.2 Achievement of Development Objectives for details of achievements and key indicators. These objectives were relevant to priority constraints in social protection and aligned well with the SAP, though, as stated, they were somewhat overreaching (see Section 3.1). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and Reasons/Justification 13. The PDOs and the key indicators were not revised. 1.4 Main Beneficiaries 14. The three agencies benefiting directly from the ESPISP are the NSSF, MOSA and MOPH which received support to develop critical tools to improve these agencies efficiency and effectiveness. When the tools are broadly applied, as in the case of the MOSA Proxy Means Testing, benefits will extend to both the poor and the GOL. The poor will benefit through better targeted programs, and the GOL will benefit through more efficient use of government funds. 1.5 Original Components (as approved) 15. The originally planned components and activities of ESPSIP were as follows: Component 1: Modernization of the Social Insurance System (US$0.7 million) 1. Provision of technical assistance for the design and implementation of an electronic claims processing and utilization control system. 2. Provision of technical assistance for the design and implementation of a registry of plan members and collection of contributions system. 3. Provision of technical assistance to identify and implement measures to secure the financial sustainability of the health insurance branch, including the refinancing of accounts payable and accounts receivable. 4. Capacity building, including training, and project implementation and coordination support at the NSSF. Component 2: Strengthening of Social Safety Nets (US$0.2million) 1. Development of a proxy-means testing (PMT) targeting formula. 2. Development of procedures for a new targeting mechanism and preparation of an operational manual that will detail the mechanism for the distribution of benefits to the poor and vulnerable. 3. Development of a management information system to support implementation of the targeting mechanism, including provision of equipment and software. 4. Provision of technical assistance to carry out testing of the targeting mechanism and method of benefit distribution. Component 3: Health Sector Cost Rationalization (US$0.1 million) 1. Provision of technical assistance to introduce a MOPH utilization review function. 2. Provision of assistance in developing 30 inpatient admission criteria within the MOPH. 4

14 3. Provision of technical support to prepare a plan to realign MOPH coverage towards more cost-effective health services and hospital facilities. 16. These components also aligned reasonably well with the priority constraints in social protection as described in the ESIA. The ESIA synthesis stated the following: Implementation of the measures will face many challenges, given: (i) the pre-existing lack of criteria for selecting private service providers; (ii) the absence of adequate data for objective household targeting mechanisms (based on household needs ); (iii) the limited capacity of the Government to design, administer and monitor social programs; and (iv) the fiscal implications of the recovery measures. 17. The ESIA also concluded that health spending in Lebanon, which affects Components 1 and 3, should yield better value for money. The report pointed out that Lebanon outspends all Middle East and North Africa (MENA) countries on health, yet its health status indicators compare unfavorably with those of countries with similar income levels. The measures in ESPISP Components 1 and 3 were focused on improving the cost-effectiveness of health care. Regarding targeting, the specific recommendations of the ESIA included that the Government should target its transfers directly to beneficiaries and that social ministries need to adopt transparent and scientifically-based targeting mechanisms (options include means-testing, categorical, geographical or a combination of them). The ESPISP Component 2 aimed at improving the targeting of public programs. Although ESPISP measures were well-justified given country needs and priorities, it is difficult to argue that they constituted real emergency measures, as one might have expected, given the designation of the project as emergency support, as described in OP 8.00 Guidelines. 1.6 Revised Components 18. During implementation, it was learned that the NSSF had already outsourced the development of a registry of (NSSF) plan members and collection of contributions system (Component 1) to a consulting firm outside of the ESPISP. Thus, this activity was dropped from the project. The freed up resources were used to fund a study of the implicit liabilities of the EOSI (details in Section 3). 1.7 Other significant changes 19. There were no other significant changes. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 20. The ESPISP responded to the Government s request for emergency funds from the Trust Fund for Lebanon (TFL) to assist the NSSF, MOSA, and MOPH to implement the reform agenda for the social sectors. Due to limited institutional capacity in the concerned agencies, GOL requested that US$1 million be allocated from the TFL to initiate the design and implementation of social protection reforms. 21. Justification for Bank execution. In its letter to the World Bank Country Director for Lebanon dated May 12, 2007, GOL also requested that this US$1 million grant (forming the 5

15 initial phase of a reform implementation program) be Bank- executed. The impact of the hostilities with Israel allowed the Bank team to make the case of extraordinary strains on the GOL to maintain and sustain social services, and thus invoke OP 8.00, paragraph 10. As required by OP 8.00, paragraph 10, clearance was granted by the World Bank Managing Director for Bank execution. The rationale provided for Bank execution was as follows: 1. The National Social Security Fund (NSSF) 5 lacks the capacity to execute this implementation support program in a speedy and efficient manner; 2. The NSSF has no prior exposure to World Bank policies, procedures, and guidelines, and time is of the essence to kick-start and advance implementation of the proposed reforms; 3. The NSSF is an autonomous institution, and as such there is no viable alternative government agency to execute the grant; 4. The grant would be limited to technical assistance and very small contracts for startup goods; and 5. Bank execution will be a measure to mitigate the fiduciary and reputational risks inherent in the current situation at the NSSF. 22. The proposed grant was consistent with the objectives of the TFL, which aim at providing timely assistance in sectors key to post-conflict rehabilitation. (See Section 3.1 for further discussion of project design). 23. Political gridlock. The ESPISP was prepared and implemented under extraordinarily difficult conditions. Not only was it prepared in the wake of widespread destruction from hostilities with Israel, but the period between June 2006 and November 2009 was one of political gridlock that included the closure of Parliament, the withdrawal of the opposition from Government, and an increase in the frequency of political violence. All of these factors created an unstable safety and security situation, which affected activities in all sectors of the Lebanese economy. Finally, as a result of the May 2008 Doha Accord, Parliamentary elections were held in June 2009 after which a National Unity Government was formed in November 2009, more than six months after ESPISP closed. 24. Project design. The project design was of appropriate scope with activities that could reasonably be completed within the 18-month timeframe. The design benefited from other countries experience that automation of business processes for expenditure controls was important to successful social insurance schemes. The program document for ESPISP also cited favorable experiences using the PMT in a number of countries. It was found that the PMT is suitable in countries with a large informal sector and seasonal incomes, and where income is not an accurate indicator of household welfare; and it delivers good targeting outcomes if implemented effectively. 25. Project risks. Project risks were assessed and addressed to some extent. The main risks cited were resistance from the NSSF Board, weak capacity at beneficiary agencies and delays in implementation because of efforts to build ownership. To address the resistance of the NSSF Board, the Bank called for its reform citing a need to reduce the size of the Board. However, the Board was not reformed, which caused problems during ESPISP I and its successor ESPISP II (see next section). The risk posed by weak capacity was addressed by designating ESPISP as 5 The NSSF was the implementing agency for the largest component of the ESPISP. 6

16 Bank-executed, which did aid implementation (see next section). The mitigation for the third major risk, assigning a project coordinator, was appropriate. However, the NSSF did not appoint one for the project (see next section). 2.2 Implementation 26. Importance of Bank-Executed to implementation. The designation of ESPISP as a Bank-executed activity helped reduce implementation risks. Under normal circumstances, an operation such as ESPISP would be recipient-executed, but because the NSSF did not have the capacity to execute the ESPISP in a timely and efficient manner and because there was no alternative agency to implement ESPISP, Bank execution was a better option. Designating the operation as Bank-executed also mitigated fiduciary and reputational risks. The main benefit of designating ESPISP as Bank-executed was that, despite the challenging political environment, the project s main (though not all) activities could be executed without significant delay. 27. Resistance from the NSSF Board. While the ESPISP measures were strongly supported by the Prime Minister s office and the Ministry of Finance (MOF), there was resistance elsewhere to the ESPISP-supported reforms, particularly from the NSSF Board of Directors. ESPISP s activities aimed to improve the transparency and efficiency of business processes and public expenditures, so there were implications for substantially reducing fraud. For example, the non-transparent, paper-based claims process that existed at the NSSF had offered opportunities for inaccurate billing and other misuses of funds. On the day of negotiation of the ESPISP project, the Board of the NSSF rejected the project. Subsequently, the Prime Minister took the decision to proceed and approved the project through the Council of Ministers. With such a level of resistance, the question arises whether the NSSF claims system component should have gone forward. Because of the Bank-executed nature of the project and support at the top level of the GOL, this risk could be mitigated and the main activity carried out. 28. Project coordinators. The Bank team liaised directly with the Director General (DG) of NSSF, and the DG liaised with the NSSF Board. The ESPISP financed an Implementation Support Team (IST) of national and international advisors and specialists to support implementation. A Project Coordinator was not recruited as originally planned, since the role of coordination at the NSSF side was undertaken by the DG. The implementation team worked closely with concerned heads of departments and units in the NSSF. The MOPH and the MOSA each appointed an Implementation Coordinator (IC) who reported to the respective DG, and who was supported by a technical task team. In the case of MOSA, a committee was established to oversee and coordinate the targeting pilot, composed of heads of the key sections of the ministry. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 29. Given the emergency nature of ESPISP, the design of the results framework was fairly basic; monitoring indicators consisted of recording completed actions and milestones. This results framework was used during supervision to keep project activities on schedule. There was no expectation that the longer term, main development outcomes of ESPISP would be measurable until at least the conclusion of ESPISP II when the new processes could be rolled out. The following were listed as outcome indicators for the project development objectives in the ESPISP results framework in the Project Document: The new systems targeted by this operation are put into place and are functional according to specifications. 7

17 Essential components of the overall social sector reform strategies are further developed. 30. These indicators are not outcome indicators, but refer to completed actions and outputs. The second outcome indicator is particularly vague and does not establish a reasonable basis with which to assess achievement of outcomes. Although benefits can certainly be inferred from these outputs, the Project Document needed to describe what the expected benefits of the actions would be. In addition, while the longer term benefits would not be measurable for several years after project closing, intermediate outcomes should have been specified for the short term, with indicators that describe capacity building as well as progress in technical areas and in institutional processes. 2.4 Safeguard and Fiduciary Compliance 31. Safeguards. The ESPISP was rated as environmental category C and no safeguard policies were triggered. 32. Financial Management and Procurement. The designation of the ESPISP as Bankexecuted helped ensure that appropriate fiduciary practices were followed under the extraordinary conditions in Lebanon. Financial management and procurement were conducted by the Bank. All expenditures were disclosed and a final accounting presented in a letter to the Minister of Finance on October 7, Of the allocated US$1 million for the ESPISP, US$957,923 (95 percent) was disbursed. US$813,993 (81 percent) was expended for consultants (local and international), US$138,444 for goods (14 percent) and US$5,081 for operating costs (0.5 percent). 2.5 Post-completion Operation Next Phase 33. Several follow-up policy operations are planned and ongoing. The Second Emergency Social Protection Implementation Support Project (ESPISP II) is designed to build on the achievements of the first ESPISP. ESPISP II was launched to maintain progress and continuity in social sector reform, and to bridge the period until the next country assistance strategy (CAS). The ESPISP II grant was also prepared under emergency procedures, as stipulated in the TFL,6 but, unlike ESPISP I, is recipient-executed. A proposed Reform Implementation DPL II (RIDPL II) was planned in line with the Interim Strategy Note (ISN), but was withdrawn as the policy environment deteriorated. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 34. Social protection was one of two areas of reform that the Bank supported through its RIDPL, with the other being energy sector reforms. The strategy of ESPISP was to provide critical support to key social protection initiatives of the RIDPL that would, in particular, relax important constraints to serving vulnerable groups. There appears to have been an implicit strategy in ESPISP that by developing fully functional technical tools (e.g., NSSF claim system, 6 The TFL was created with a transfer of US$70 million from IBRD surplus in September ESPISP was financed from the accrued interest income from the TFL. 8

18 and the PMT), there would be pressure to move the policy environment forward. The lack of a functioning tool could no longer be used as an excuse to do nothing. 35. The Bank team judged that the post-conflict period offered an opportunity for a breakthrough in these reform areas. Compared with the alternative strategy of waiting for a more settled political and security situation, the strategy of moving forward with the ESPISP measures has better positioned the GOL to achieve real improvements in efficiency and effectiveness of public expenditures. The emphasis of the Government s own formal SAP on social protection issues clearly made a case for an operation to support social protection reform. Although the ESPISP activities were not emergency measures per se, the development objectives were relevant to removing several key obstacles to improving the delivery of social protection services. These obstacles were identified by the ESIA and highlighted in the SAP, and had been encountered in previous Bank operations. The technical support under ESPISP aligned well with the reform measures. The composition of the operation as a whole was somewhat ambitious in the Lebanese context to reasonably expect smooth follow-up implementation given that certain constituencies were threatened. 36. The main PDO was to accelerate and improve the quality of the implementation of the package of social sector reforms presented by GOL at the Paris III donor conference in the areas of social insurance, safety nets and health expenditures. This objective was feasible and consistent with the project activities. Its precision could have been enhanced with more specific results indicators (see Section 2.3). The project-specific objectives (see Section 1.2) gave a picture of what the project was expected to achieve, but these objectives were somewhat overreaching in the way they were stated (see Table 2). For example, Component 1 s objective was to Improve the control of expenditures and revenues within NSSF. However, this objective would not be achieved until the claims system was actually implemented. A more precise formulation of the component objective might have read Create tools to improve the control of expenditures. Similar overreaching appears in the objectives for Components 2 and 3. In all three components, the component objectives jump ahead to the benefits that would be realized after full implementation, which was not within the scope of ESPISP. 3.2 Achievement of Project Development Objectives 37. In the midst of a largely unsuccessful reform program, ESPISP had some significant achievements. Although many of the performance indicators were process-oriented, referring to completed actions or outputs and not true outcome indicators, new processes and tools, essential for rationalizing public expenditures, were developed. As observed in Section 2, the ESPISP results framework listed only output indicators. The real major outcomes would be budget savings from a more efficient hospital claims and reimbursement system, improved patient services because medical controllers will be less burdened by administrative duties, and reduced poverty because government programs are better targeted. Again, realizing these outcomes will require broad application of the tools and systems. This rollout is part of the follow up ESPISP II. Therefore, it is not yet possible for this ICR to assess the longer term expected outcomes. Table 2 summarizes and evaluates the major activities of ESPISP. 9

19 Table 2: Results and Ratings of Major Activities of the ESPISP Activity Result ICR Rating Component 1: Modernization of the Social Insurance System Moderately Satisfactory Component Objectives: Improve the control of expenditures and revenues within the NSSF, as well as the quality of services, by setting up an electronic claims processing and a utilization control system and a registration and contributions collection system; and ensure financial sustainability of the NSSF by identifying the needed set of reform measures to control expenditures and optimize revenues. Provision of technical assistance for the design and implementation of an electronic claims processing and utilization control system. Provision of technical assistance for the design and implementation of a registry of plan members and collection of contributions system. Provision of technical assistance to identify and implement measures to secure the financial sustainability of the health insurance branch including the refinancing of accounts payable and accounts receivable. Capacity building, including training, and project implementation and coordination support at the NSSF. Completed and tested in 1 hospital, but as of date of ICR not yet operating live in any hospital. Dropped from ESPISP I at request of NSSF management because the task was contracted to another company. An End-of-service Indemnity (EOSI) valuation was conducted in lieu of the registry of plan members. Completed. Recommendations were made to improve the financial sustainability of NSSF s health insurance branch. However, the recommendations have not been acted upon by the NSSF Board. In a separate but related activity, the arrears to NSSF were assessed and Government paid an additional LP 80 billion in 2007 to NSSF to satisfy accounts payable. However, GOL failed to make its 2008 payment of LP 80 billion. Five people received training in the NSSF on Health Reform Options Simulation Tool (HROST). However the actuarial unit was not established Moderately Satisfactory No Rating Satisfactory Moderately Unsatisfactory Moderately Satisfactory Moderately Unsatisfactory Component 2: Strengthening of Social Safety Nets Moderately Satisfactory Component Objective: Improve the poverty focus of existing or new safety net programs in MOSA by developing and piloting a new targeting mechanism. Development of a proxy-means testing targeting formula. Development of procedures for a new targeting mechanism and preparation of an operational manual that will detail the mechanism for the distribution of Completed development of the PMT. Completed. Satisfactory Satisfactory 10

20 benefits to the poor and vulnerable. Development of an information management system to support implementation of the targeting mechanism, including provision of equipment and software. Provision of technical assistance to carry out testing of the targeting mechanism and method of benefit distribution. An information system was developed to determine the ranking, eligibility and progressive benefit calculation for candidates for a targeted benefit based on the PMT. A successful pilot was carried out in 3 Beirut districts. Credibility of the exercise is threatened because the incentive payment of LP50,000 to qualifying families has still not been paid. Satisfactory Moderately satisfactory Component 3: Health Sector Cost Rationalization Moderately Unsatisfactory Component Objective: Control health expenditures by the MOPH through establishing a utilization review function, developing of hospital admission criteria and protocols, implementing hard ceilings on contracts with private hospitals and realigning MOPH coverage towards more cost-effective health services and hospital facilities. Provision of technical assistance to introduce a MOPH utilization review function. Provision of assistance in developing 30 inpatient admission criteria within the MOPH. Provision of technical support to prepare a plan to realign MOPH coverage towards more cost-effective health services and hospital facilities. Partially completed The utilization review (UR) function has been introduced in the sense that there is a UR coordinator. However, the function has not been institutionalized. This activity has not been completed and progress remains very slow. Completed. MOPH Committed adopted and implemented the early stages of a roadmap to introduce performancebased contracting for hospital services, which included developing performance indicators and establishing data requirements for these indicators. Moderately unsatisfactory Unsatisfactory Moderately satisfactory Summary of Achievement of Objectives Component 1: Modernization of the Social Insurance System Moderately Satisfactory 38. Component 1 of ESPISP dealt with two of the three NSSF benefit programs (health insurance and the EOSI), and was the largest component of the project, using 70 percent of the project s funds. The achievements of this component represent a real breakthrough for the GOL. The existing hospital claims systems at NSSF had been a paper-based system, which gave broad scope for false claims and had led to excessive delays in reimbursing hospitals for services provided. In addition, the paper-based system did not provide data for managing hospitals, such as what types of patients were admitted, what types of procedures were performed, and what types of equipment were used. ESPISP financed a successful effort to develop an automated system that could greatly improve the efficiency of the hospital claims system. The expected 11

21 output was achieved, which is a functioning hospital claims system (NAPS) that has been tested using actual medical records. It was highly challenging to develop a consistent, functioning medical claims system, because the rules engine for the system had to incorporate thousands of rules from a number of different sources, as well as medical codes. The NAPS, if implemented, would reduce time periods to reimburse hospitals and help eliminate double and other fraudulent charging for services. It would also free medical controllers from a heavy burden of administrative work so that they could focus on a higher level responsibility of improving the medical process for the patient. 39. Another positive achievement was the resolution of the GOL s arrears to the NSSF. Because data were insufficient for an accurate calculation of arrears, it was decided that the GOL and NSSF would agree on an amount of arrears and then establish an installment plan to satisfy those arrears. Beginning in 2007, the GOL paid both its current obligation to the NSSF as well as an additional LP80 billion in order to pay down arrears that had accumulated during This achievement has contributed to the objective of the component which was improved financial stability of the NSSF. However, it is a matter of significant concern that, as of the date of this ICR, the GOL has still not paid its 2008 installment to NSSF of LP 80 billion. 40. Several areas that were less successful were the adoption of financial sustainability recommendations and the building of capacity within NSSF. The program document described the importance of the registry of plan members by stating that The management of the NSSF cannot be improved in the absence of an up-to-date computerized system to register plan members, and collect and reconcile contributions. However, this subcomponent was dropped from the project when it was learned that another consulting firm was working on the system. ESPISP II has since included funding to complete the task of creating a registry of plan members. 41. The resources that would have been spent on the registry under the first ESPISP were reallocated to a valuation of the End-of-Service Indemnity Program (as of December 31, 2006). This valuation was a measure toward financial sustainability of the NSSF by creating a better understanding of NSSF s financial position in line with the component objective. The current data set could not determine the detailed, specific amount to be transferred to future individual accounts in order to transition to a proposed individualized national pension scheme. However, despite this uncertainty, the valuation exercise did achieve a clearer understanding of NSSF s overall financial position, which is in line with the component objective. 42. Developing the recommendations to improve financial sustainability of the NSSF was successfully completed, but the NSSF Board of Directors has yet to act on these recommendations. Without action, the subcomponent is rated moderately unsatisfactory. Pressure to implement the recommendations was alleviated to some degree by the fact that wages increased, and the resulting increase in contributions improved NSSF s financial position. The establishment of the actuarial unit in the NSSF was not achieved, although five people received actuarial training from consultants. Thus, this subcomponent is also moderately unsatisfactory. Component 2: Strengthening of Social Safety Nets Moderately Satisfactory 43. This component also achieved a breakthrough. For the first time in Lebanon, the GOL would have a tool with which to target programs to the poor. Government subsidy and income support programs in Lebanon tend to be overly broad or confession-based. Obviously, this approach is very inefficient in reaching the poor. Non-targeted programs are also fiscally unsustainable which is why the MOF is a key supporter of targeting and specifically the proxy means testing type of targeting. Proxy means testing is an appropriate method because of the lack of reliable income data in Lebanon. 12

22 44. The MOSA, with assistance from consultants and Bank experts, successfully developed the PMT targeting formula, as well as the procedures for the new targeting mechanism (including preparation of an operations manual), and the information management system to support the implementation of the targeting mechanism. The PMT was piloted in three districts of Beirut. A core team of about 12 staff from MOSA operated and managed this pilot (7 staff from center and 5 staff from the regional centers). The MOSA staff demonstrated dedication as they worked overtime without additional compensation. Thus, the ESPISP was able to build capacity at MOSA. The Italian Government funded a portion of the cost of this pilot, to contract social workers who conducted house visits/surveys. The reason for lowering the rating to moderately satisfactory is the long and continuing delay in paying qualifying households their cash incentive for participating in the PMT pilot. Failure to make this payment could undermine the credibility of the PMT pilot. Component 3: Health Sector Cost Rationalization Moderately Unsatisfactory 45. As Table 2 shows, each of the activities at MOPH was only partially carried out. Regarding the utilization review (UR) function, while MOPH has taken specific actions in that direction, the UR function has not really been introduced in the sense that the original objective implied. The original objective referred to a function which can oversee and implement more precise admission criteria, which has not been achieved. However, the MOPH did: (i) set up a MOPH UR Committee with independent experts; (ii) assign a senior medical controller to become the UR coordinator within the MOPH; (iii) perform data analysis on the high impact procedures covered by the MOPH; (iv) prepare an in-depth study of diagnosed cases of gastroenteritis in a national workshop organized by the MOPH; (v) prepare a detailed blue-print for institutionalizing the UR function within the MOPH; and (vi) analyze the specifications for installing a developed rules engine as part of the MOPH IT system. 46. The activity to establish 30 admission criteria progressed slowly and was not completed. A MOPH Committee was formed for the clinical management criteria and the concept was presented at a national workshop. The key recommendations were to: (i) choose among two international vendors selling clinical management criteria systems; and (ii) work with the Union of Physicians to localize, adopt and approve a set of clinical management criteria to be used by the MOPH. However, these recommendations have yet to be implemented. Under the subcomponent to realign MOPH coverage towards more cost effective services, a consultant produced a detailed road map to introduce performance-based contracting for hospital services. A MOPH committee adopted the road map and implemented its early stages. The Committee developed a basic set of performance indicators and analyzed the data requirements to generate these indicators on a regular basis. As with the other sub-components, progress stopped short of actual implementation. Did ESPISP Make a Difference? 47. The answer to this question needs to be divided into the short-term and the long-term. In the short-term, it was a good strategic approach to use the TFL funds to develop tools that are badly needed and to use these tools to pressure decision-makers to move policy forward to rationalize public expenditures. In addition, capacity has been built, particularly at MOSA, but also to some extent at the NSSF, especially in understanding the value and use of the new tools. The longer-term question is whether hospital and basic social protection services will improve as a result of ESPISP. This development impact will depend on whether the tools are actually implemented on a broad basis, and that is uncertain at this point (see Section 4). 13

23 3.3 Efficiency 48. As a technical assistance project, no efficiency calculation was made for this project. However, one can note that, for modest resources of US$1 million, critical tools were developed that could substantially advance social protection benefits in the future. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 49. As the summary demonstrates, there were substantial achievements and it is clear that the operation made significant progress in a very difficult environment. Moreover, the main outputs (the electronic claims system and the PMT) were developed and tested and the quality of the outputs was satisfactory. Major questions persist largely because of the slowness of further progress of ESPISP initiatives in ESPISP II (additional details below) and with the inability of the GOL to pay the promised incentive payment on which the PMT pilot was based. Although Component 3 is rated moderately unsatisfactory, it represented a relatively small part of the overall project. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 50. Part of the expected impact was a system of improved social service delivery. Again, the ESPISP represented only a step in that direction. Therefore, the actual poverty reduction impact is only to be realized if and when the systems are implemented on a broad basis. (b) Institutional Change/Strengthening 51. The institutional development achieved by the ESPISP, although potentially substantial, must be rated as modest. Institutional change could be significant if the tools of the program actually become the way of doing business for the NSSF, MOSA and MOPH. In terms of capacity building, the NSSF benefited to some degree at the working level from the near constant presence of the NAPS consultants who worked intensively with NSSF staff on a daily basis for five months. This constant presence helped equip NSSF staff with necessary knowledge to operate the new system, and won acceptance of the system. However, the NSSF does not possess the IT capacity to run the new system. In addition, the specific capacity building that ESPISP aimed at NSSF did not materialize, namely that an actuarial unit would be formed. The MOSA benefited significantly in terms of capacity building as there is today a core team (of about 12 staff) who understand the PMT targeting mechanism, who can run the program, and who have developed the field expertise for the national rollout. 52. One change that appears necessary to the success of NSSF reforms is to ensure that the NSSF Board s mandate and composition are consistent with fiscal responsibility and respond to the needs of patients and doctors. (c) Other Unintended Outcomes and Impacts (positive or negative, if any) 53. None reported. 14

24 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 54. Neither a specific beneficiary survey, nor a stakeholder workshop was conducted as this ICR is not an Intensive Learning ICR. 4. Assessment of Risk to Development Outcome Rating: High 55. The country context for achieving more efficient public expenditures and developing and implementing social protection tools was known to be difficult. Although the important gains, such as the NSSF claims system and the PMT, were achieved, there is disconcerting evidence of lack of commitment in certain areas of the GOL to implement these tools. The NSSF Board continues to obstruct progress in the follow up ESPISP II by withholding contract signing authority from the DG at NSSF, and after 15 months of effectiveness, not a single contract has been signed. As mentioned, the credibility of the PMT exercise has also been jeopardized because the benefit payment to qualifying households is severely delayed and still held up in the Court of Accounts. This delay is difficult to explain, given that the government support for the initiative was secured prior to the project. On the positive side, it was reported that not only the MOF, but also various ministries, are strongly in favor of applying the PMT as a means to improve targeting of programs and making more efficient use of public funds. The next few months will be critical in determining whether the achievements of ESPISP I will have any lasting impact. 56. An encouraging development is that the GOL has finally formed a National Unity Government (November 2009), and is giving priority to reform of the NSSF and other aspects of the Social Action Plan. There is a new Minister of Labor who has declared support for the NSSF reforms. The new Government appears committed to greater fiscal discipline which the ESPISP outputs could potentially promote. Taking all the above factors into account, the ICR concurs with the most recent Aide-Memoires, which acknowledge that the risk to development outcome remains high. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Moderately satisfactory 57. The Bank responded alertly to the appeal of the GOL to support its SAP. The ESPISP was an important part of this response, which was based on ESW in the form of the ESIA, and policy support through the RIDPL. The Bank-executed operation ensured that the components of ESPISP were stated Government priorities. There was additional management review because of the waiver required to implement the ESPISP as a Bank operation. Management weighed the pros and cons of this decision with the main con being the potential for less ownership by the implementing agencies. The waiver was justified in large part by the fact that the main implementing agency (NSSF) was not in a position to carry out fiduciary roles. The Bank also helped ensure that the components of the project were closely tied into the GOL s need to become more committed to fiscal discipline. Without meaningful controls in health expenditures and 15

25 without rationalizing public expenditures for the poor, there could be no meaningful progress toward fiscal discipline. 58. Two areas where the Bank could have performed better are with respect to risk assessment and the results framework for ESPISP. The Bank could have better assessed the potential implementation obstacles of the NSSF Board (Component 1) and the lack of leadership at the MOPH (Component 3). Ultimately, it is not clear that there was anything the Bank could have done to ease these obstacles. However, it is clear that the program document underestimated these risks. Another area where the Bank could have performed better was in developing a results framework which was more relevant and useful in terms of measuring project outcomes and outputs. It is true that ESPISP was processed under simplified procedures, as OP 8.00 provides. It is also true that the ESPISP consists mainly of completing activities and developing tools. However, it would still have been possible during project preparation to identify meaningful indicators to measure benefits of the operation. (b) Quality of Supervision (including M&E arrangements) Rating: Satisfactory 59. As a Bank-executed project, supervision was more demanding for the Bank administratively than if it had been recipient-executed. Hiring and paying consultants, for example, was conducted by Bank staff. On the other hand, ISRs were not required for a Bankexecuted project. (Aide-Memoires constituted the official record of implementation.) In terms of engagement with counterparts, the Bank was fully engaged in both the administration and substance of the project. The Bank-executed project was completed, with qualifications noted above, within the originally envisioned 18-month timeframe. The Bank is currently witnessing the contrast of the greater challenge to implementation of a recipient-executed project through the slow implementation of the ESPISP As discussed in Section 2, ESPISP was implemented in an extraordinarily challenging political environment. Because of the factionalism in the country, implementation is difficult under normal circumstances. Bank execution also had to contend with a near standstill in political decision-making for the life of the project. The specific challenges included the fallout from the hostilities with Israel and the run up to the elections in June The Bank also made successful use of an existing model for medical claims, which consultants were able to adapt to the Lebanese context. It was an impressive feat to balance all of the rules governing medical claims in the country. The NAPS activity also had to navigate through political obstacles that created issues for the technical architecture of the system. The ESPISP NAPS consulting team decided to maintain good working relations by not interfering with ongoing efforts to resurrect a problematic out-patient claims system (which NSSF had previously outsourced). Instead, the consultants focused on the in-patient claims system. They also determined that it would not be feasible, initially, to create a claims system that would be compatible with the pre-approval system. In both instances, the team showed good judgment in terms of maintaining a high level of cooperation in NSSF and in determining a feasible, but still challenging, scope of the assignment. 62. As discussed, the M&E framework to be used during supervision should have been more substantial, even for a process-oriented project in which the main outputs were completed actions. Still, the real outcomes would not be realized until perhaps several years after project closing when the payment system and targeting mechanism and other process improvements were rolled 16

26 out. The project was run fully out of the Bank s Country Office, which satisfactorily provided daily oversight to the project. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 63. The combination of moderately satisfactory performance of the Bank at preparation and satisfactory performance at supervision leads to a moderately satisfactory performance overall. Advocating on behalf of the GOL s own reform program arguably serves the interests of the client. The efficiency of resource use also was satisfactory, given how much was achieved with the modest envelope of resources. 64. An additional issue of performance, though it is not clear what the impact would be on rating of Bank overall performance, is the ratio of Bank budget costs to overall project costs. As Annex 1 shows, BB lending phase costs for ESPISP were US$206,210 and supervision phase costs were US$386,280, for total BB costs of US$592,490. This total is more than 60 percent of total project disbursements (US$957,923), which must be considered high by any standard. Mitigating this high BB to project cost ratio are that most of the BB costs were a result of the fact that ESPISP was Bank-executed and that Bank staff were providing a high level technical support to the component activities. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 65. The Ministry of Finance spearheaded initiatives in the GOL to rationalize public expenditures. The ESPISP responded to that effort. Although ESPISP was Bank-executed, the MOF laid much of the political groundwork in the three implementing agencies for the activities of the project. However, the MOF could not control everything, including the slow (still pending) approval by the Court of Accounts of the PMT incentive to pilot participants. Although there have been roadblocks to completing the vision of ESPISP, the MOF remains committed to rolling out the achievements of ESPISP. For its sustained commitment and strong preparatory work, the MOF deserves a satisfactory performance rating. Difficulties elsewhere in the Government, e.g., the excessive delay at the Court of Accounts in paying the benefit to qualifying participants in the PMT pilot, lower the Government performance rating to moderately satisfactory. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory 66. The three implementing agencies had a mixed performance. Overall, the implementing agencies (IAs) struggled and relied heavily on consultants. Table 3 provides the basis for government and individual ratings of the IAs. 17

27 Table 3: Summary of Individual Implementing Agency Ratings Agency Rating Comment Ministry of Finance Satisfactory Performed well in building networks to support ESPISP. Encouraged other ministries to comply with program. NSSF Moderately Unsatisfactory While the DG was supportive of the project, the NSSF Board took steps to undermine it. MOSA Satisfactory The component implemented well and with participation of MOSA staff and building of MOSA capacity. MOPH Moderately unsatisfactory MOPH only partially completed the activities for which it was responsible. 67. In addition to the difficulties already mentioned at NSSF, there was a serious issue of reliable, uninterrupted electricity supply that the NSSF Board appeared unwilling to address. During weekends, power to the building which housed the servers for the NAPS was routinely shut off. This interruption corrupted the software application and created havoc for the servers (there were five clustered servers), which had to be reloaded each Monday, a process that could take 48 hours or more to complete. The DG appeared powerless to remedy the problem in the face of inaction from the NSSF Board. The failure of the NSSF to appoint a project coordinator was a large part of the reason that the NAPS has not advanced further. 68. The performance of MOSA was constructive and through the project, the Ministry acquired an understanding of the PMT methodology and is applications. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory 69. The moderately satisfactory performance of the Government, combined with an unsatisfactory performance of the NSSF and a weak performance of the MOPH, leads to a moderately unsatisfactory Borrower performance overall. 6. Lessons Learned 70. The Bank needs to address more aggressively the issue of governance structures, which can foil project implementation. It was clear from the ESPISP that the NSSF Board intended to obstruct any type of reform of the NSSF programs. Leaving the Board un-reformed during the design and implementation of the follow-up ESPISP 2 has proven destructive to the potential outcomes of ESPISP achievements. The Bank understands that reforming the NSSF Board is essential to successful implementation, but does not have the power to ensure that this reform is undertaken. However, the Bank will need to explore other actions to mitigate such risks. 71. The option of Bank-execution to implement projects in fragile/post-conflict states is only a partial solution. The waiver that allowed the project to operate as a BETF rather than RETF clearly helped implementation occur on time. Bank execution made a difference because the nature of the activities was mainly hiring consultants. The streamlined execution, however, cannot substitute for managerial and political decisions that implementing agencies must take to ensure successful implementation. In addition, the ability to bypass the NSSF Board was only a temporary benefit of ESPISP I. 18

28 72. The Bank needs to develop a new approach to project execution that will work better in post-conflict/fragile states, particularly for Bank-execution. The BETF proved to be an unwieldy tool to implement ESPISP, as Bank policies and procedures are more oriented to Recipient-execution. Throughout the preparation process of the project, and during implementation, many guidelines had to be reinterpreted to fit them to Bank-execution. One possible solution would be for the Bank to assume a higher risk profile in such environments, which would necessitate revisiting existing guidelines to allow them to better facilitate Bankexecution. 73. Ownership at working levels can be developed in Bank-executed or consultantimplemented projects, but not necessarily at the decision-making levels. The case of the development of the NSSF hospital claims system (NAPS) demonstrated how consultants working long hours side-by-side with staff could help staff develop knowledge and acceptance of new, more efficient and effective processes in service delivery. This sustained work with counterparts does not ensure, however, that decision-makers will carry through in supporting a framework for implementation. 74. The Bank should improve its efforts to advertise the benefits of a project to win the support of the general populace. It was reported in the MOPH component that one reason for the resistance to the Bank project was that there was not enough understanding of what the project benefits would be, on one hand, compared with too much cynicism spawned by too many failed projects of the past, on the other. During the ICR mission, MOPH made the suggestion to hold workshops for stakeholders to inform them of what the project was attempting to achieve as this would provide the opportunity to lower resistance to reforms. 75. It is also well to reiterate a lesson from the ISN: 76. The complexity of the political environment for decision making in Lebanon should not be underestimated. The Bank, like other donors, has often made the erroneous assumption that simply because a piece of policy reform is in the national interest of Lebanon, it will be implemented. In Lebanon, the confessional system of governance and the resulting diffusion of political authority make such reforms subject to collective action, and such collective action is, for the same reasons, far from guaranteed. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/Implementing agencies 77. The ICR generally concurs with the GOL review of the operation (Annex 7). It needs to be reiterated, however, that until the tools created with the support of ESPISP I (the claims system and the PMT) are actually rolled out, the development impact will be minimal. 78. Two of the three of the lessons learned in the GOL review speak to the need to do a better job up front of communicating the benefits of the project to the stakeholders. The ICR also presents a lesson learned in this vein. 79. With regard to the MOPH comment on the evaluation of its component, the moderately unsatisfactory rating of the component is based on the fact that two of the three subcomponents were fundamentally not implemented. These shortfalls in implementation are specified in section 3.2. In the GOL comments, no additional data or explanation are provided with which to 19

29 reconsider the rating. Therefore, the ICR maintains the original rating of moderately unsatisfactory for Component 3: Health Sector Cost Rationalization. (b) Cofinanciers 80. There were no formal co-financiers of the project. However, the Italian Government provided substantial resources to conduct the PMT pilot. Regarding the comment to extend social protection to Lebanese women who are non-lebanese (see Annex 8), Italian Cooperation has the opportunity to support this goal during its continuing work with ESPISP II. 20

30 Annex 1. Project Costs and Financing a. Project Cost by Component (US$) Component Allocation Disbursed 1. Modernization of the Social 700, ,812 insurance System 2. Strengthening Social Safety Nets 200, , Health Sector Cost Rationalization 100,000 74,808 Operating costs (translations, etc.) 1,910 Total 1,000, ,923 b. Financing (US$) Source of Financing Allocation Disbursed Total IBRD/IDA Trust Funds 1,000, ,923 21

31 Annex 2. Outputs by Component Outputs by component are presented in Table 2 of the main text. 22

32 Annex 3. Economic and Financial Analysis An economic and financial analysis was not envisioned for this operation and was not carried out. 23

33 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility/ Specialty Lending Haneen Sayed Lead Operations Officer MNSHE Task Team Leader David Robalino Senior Economist MNSHE Pensions Celine Gavach Operations Analyst MNSHE Operations Ghassan N. Alkhoja Senior Operations Officer MNSSP Social Protection Aleksandra Posarac Lead Human Development Economist HDNSP Safety nets Sophie Urnelshian Program Assistant MNCLB Administrative Support Supervision/ICR Haneen Sayed Lead Operations Officer MNSHE Task Team Leader David A. Robalino Senior Economist HDNSP Pensions Firas Raad Zaid Al-Hussein Senior Health Specialist MNSHH Health Ghassan N. Alkhoja Senior Operations Officer MNSSP Social Protection Lina Fares Procurement Specialist MNAPR Procurement Celine Gavach Operations Analyst MNSHE Operations Aleksandra Posarac Lead Human Development Economist HDNSP Safety nets Surat Nsour Operations Officer MNSHD ICR Task Team Leader Amy Champion Operations Analyst MNSHH Quality team Hala Ballout Program Assistant MNCLB Administrative Support Richard Carroll Consultant MNSHD Completion Report (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY FY Total: Supervision/ICR FY FY FY Total:

34 Annex 5. Beneficiary Survey Results Not Applicable 25

35 Annex 6. Stakeholder Workshop Report and Results Not Applicable 26

36 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR 27

37 28

38 29

39 30

40 31

41 32

42 33

43 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders The comments of Italian Cooperation are as follows: In the next Proxy Means Testing (PMT) phase, appropriate methodological measures should be identified and agreed upon with the GOL in order to extend social protection to Lebanese women who are married to non-lebanese (ex. revisiting the definition of head of household and criteria for eligibility to receive social assistance accordingly), so as to enhance the propoor targeting of the mechanism. Securing consensus on the prompt establishment of the administrative link between the PMT database and the public provision of whatsoever form of social assistance (cash and in-kind) should be given the highest priority. However, enhancing the Ministry of Finance s perception of administrative ownership of the PMT should not be perceived as undermining the political ownership of the ministries and agencies directly affected by the re-structuring of social assistance provision. Rather, constructing a common discourse among these latter is key to the overall success of this reform process. Finally, potential opportunities and obstacles already embedded or to be created - in the new decentralization draft law should be timely weighed [comment referred to section 3.5.(b)]. 34

44 Annex 9. List of Supporting Documents available in the project portal Aide-Memoires for ESPISP I Documents on waiver/designation of ESPISP as BETF Emergency Social Protection Implementation Support Project, Program Documents, August 30, 2007 Interim Strategy Note for the Republic of Lebanon for the Period FY08-09, July 2007 Lebanon Economic and Social Impact Assessment, December 2006 Reform Implementation Development Policy Loan, Program document, August 2007 Reform Implementation Development Policy Loan, ICR, October 2009 Second Emergency Social Protection Implementation Support Project, Program Document, July 24,

45 Map