US$M): Sector Board : Public Sector (US$M US$M): Cofinancing (US$M. ICR Review

Transcription

1 Public Disclosure Authorized IEG ICR Review Independent Evaluation Group Report Number : ICRR Project Data: Date Posted : 07/24/2014 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Country: Philippines Project ID : P Project Name : Ph Judicial Reform Support Project Appraisal Actual Project Costs (US$M US$M): L/C Number: L7191 Loan/Credit (US$M US$M): Sector Board : Public Sector Cofinancing (US$M US$M): Governance Cofinanciers : Board Approval Date : 08/19/2003 Closing Date : 12/31/ /30/2012 Sector(s): Law and justice (100%) Theme(s): Judicial and other dispute resolution mechanisms (29% - P); Access to law and justice (29% - P); Legal institutions for a market economy (28% - P); Other public sector governance (14% - S) Prepared by : Reviewed by : ICR Review Group: Coordinator : Catherine C. Kleynhoff Michael R. Lav Lourdes N. Pagaran IEGPS2 2. Project Objectives and Components: a. Objectives: The Loan Agreement defines the objective of the project as to assist the Borrower in developing a more effective and accessible Judiciary that would foster public trust and confidence, through the implementation of the Supreme Court's Action Program for Judicial Reform (Schedule 2, p. 12). The PAD had a similar definition: to support the development of a more effective and accessible judicial system that would foster public trust and confidence (PAD, p. 3) It also defines four specific objectives : (i) improve efficiency of case adjudication and access to justice; (ii) enhance integrity of the judiciary; (iii) strengthen institutional capabilities and (iv) stakeholder support for reform of the judiciary. The Loan Agreement provides a set of performance indicators associated with the overarching objective and specific objectives. A similar set of indicators is also provided in the PAD. There is no material difference between the Loan Agreement and the PAD. This review will use the objective and the associated performance indicators in the Loan Agreement. b.were the project objectives/key associated outcome targets revised during implementation? No c. Components: A. Improving case adjudication and access to justice (appraisal : $6.37 million; actual : $5.69 million). (i) Design and implementation of an improved case management system at the Supreme Court and other courts;

2 (ii) Design and implementation of a court records management system in the higher courts and pilot model courts; (iii) Installation of computer-aided transcription technology in the higher and pilot courts; (iv) Reorganization of the jurisdictional structure of courts; (v) Identification of (a) policy options for the adoption of a social pricing mixed scheme for legal fees to improve the affordability of legal services, including use of the courts by the poor, and (b) mechanisms for public and private resource generation programs to support the availability of affordable legal services to the poor; (vi) Strengthening of the access to justice via mobile courts, (vii) Upgrading of the judiciary s information and communications technology. B. Enhancing institutional integrity (appraisal : $0.27 million; actual : $0.24 million): (i) assess current codes of ethics, develop a gender -sensitive Code of Ethics for justices, judges, lawyers and court personnel; (ii) strengthen the feedback mechanisms for monitoring their conduct and performance, and provide related training; (iii) computerize the Judicial Performance Management System, and implement it in three pilot model courts and in the higher courts; and (iv) formulate and implement a gender-sensitive Human Resource and Development Master plan for non -judicial personnel. C. Strengthening the institutional capacity of the judiciary (appraisal : $ million; actual : $ million) (i) Implement decentralization plan in pilot model courts : (ii) develop and implement a financial management improvement system; (iii) finance infrastructure improvements to set up pilot model courts in Mindanao, Visayas, and Luzon, to complement and showcase reforms; and (iv) establish an electronic judicial library and research facilities to strengthen policy and R&D capacity. D. Support for reform and strengthening the Program Management Office (appraisal : $0.84 million; actual : $0.72 million): Provision of technical assistance, training and workshops to (i) assess possible impacts of the Action Program for Judicial Reform (APJR (ii) carry out knowledge sharing programs among the judiciary, the legal community and non-governmental organizations; (iii) provide technical and institutional support for judges' working groups; (iv) design social communication programs to promote support for the APJR; and (v) prepare a program of internal and external communication for the courts, (vi) develop community outreach, and (vii) carry out gender-segregated user surveys for feedback and for refinement of reform activities. The sum of all component costs estimated at appraisal, plus a $1.96 million contingency and a $0.22 million front-end IBRD fee adds up to $24.42 million, the amount listed as Total Project Costs. The sum of all actual component costs, plus a $1.96 m contingency, a $0.22 million front-end IBRD fee, and a cancellation of $1.29 million, comes to $20.61 million, the amount listed in the ICR. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The final project costs at completion were 90% of the appraisal estimate. The final loan amount at completion was 94% of the amount approved by the Board. These reductions reflect the scaling back of the project scope in the last years to focus on the completion of activities most likely to help achieve the PDO. The Loan Agreement was amended 3 times: In 2005 to address the very slow implementation of the project and the related slow disbursements. It extended the deadline for reaching most performance indicators from Mid -Term Review (2006) to project completion (2009), In 2009 to address continued major delays in implementation. It extended the closing date by 18 months, cancelled one of the three pilot mobile courts because of land titling issues, and added a sub -component - renovations and additions to the headquarters of the Philippine Judicial Academy - to replace the previously cancelled third pilot mobile court. In 2011 to reduce the scope of the project and ensure compliance with social and environmental safeguards. It extended the closing date by one year and cancelled $ 0.5 m of the loan. The Borrower's contribution was $2.5 million at appraisal, and $2.5 million as actual. Funding was provided in a timely manner. 3. Relevance of Objectives & Design:

3 a. Relevance of Objectives: Negligible. By closing, the relevance of the objective to the country's current development priorities and to the Bank's current strategy had diminished. The project objective was one of the country's top priorities during preparation and early implementation. The FY02 CAS (table 3-1, page 17) presented judicial reform as part of its growth agenda, namely, Increase confidence in the rule of law and in the judiciary, in particular, through judicial reforms, and as part of its agenda to empower the poor, namely, improve access to justice and confidence in the judiciary. But the repeated changes in the leadership of the Supreme Court (the implementing agency), with five Chief Justices and an Acting Chief Justice during the nine years of implementation, resulted in a mixed and ultimately waning commitment to the project objective. The sporadic focus on implementing judicial reforms led to substantive reforms not being adopted. As implementation advanced, the Supreme Court focused more and more on the construction and equipment part of the project. By the end of the project, the lack of substantive measures taken demonstrated that although judicial reform remained a stated priority of the judicial and executive leadership, actual reform measures taken and their impact fell short of expectations, while difficult but necessary measures were not taken. The objective had also lost most of its relevance to the current Bank strategy as expressed in the 2011 Philippines CAS Progress Report which listed judicial reform in the Decreased /Shifted Emphasis section. This section is defined as "areas where the needs are significant but where the Bank's comparative advantage has not been confirmed, or earlier initiatives have not led to specific programs ". The CAS further noted that the continued engagement and dialogue in the justice sector reform would be pursued through AAA. (para 38, p 11). b. Relevance of Design: The relevance of design is Substantial. The PAD presents a solid results chain: the outcomes and related outcome targets are in coherence with the objective. The components, activities, and output targets are well aligned together, clearly linked to the objectives, and well presented. The causal chain between Bank funding, and intended outcomes and outputs is well justified and presented clearly. The choice of investment lending as the preferred and most relevant lending instrument is well substantiated. 4. Achievement of Objectives (Efficacy): The objective is a more effective and accessible judiciary that would foster public trust and confidence through the implementation of the Supreme Court's Action Program for Judicial Reform. Negligible. The achievement of this objective is assessed based on the achievement of four sub -objectives in the four areas as discussed below. Some of the actions necessary for the achievement of these dimensions were taken, and some of the output performance indicators were achieved. However, the lack of adoption of related reforms and the partial completion of several intermediate results /outcomes of each sub-objectives resulted in the project objective not being achieved. Improving efficiency of case adjudication and access to justice. ICT-based court and case management systems were piloted in two sites and introduced in the Higher Courts, but they have not yet been extended to the lower courts. They are operating fairly well in three Higher Courts, and are in the process of being introduced in the fourth Court. A study was carried out to explore options to improve access and expedite case dispensation, but its proposals were modest, and have yet to be acted upon. Vacancy rates for judge positions in lower courts increased during implementation, thus reducing the likelihood that case adjudication would be faster and better. An Enterprise Information Systems Plan was adopted, but not implemented as planned. The Justice on Wheels system was established to provide mediation, case adjudication, and information dissemination services to prisoners held without trial. There were three performance indicators: A reduction in case processing time in Higher Courts. This was achieved for one Court out of four Courts. A reduction in case backlogs in pilot model courts. This was achieved, and even exceeded in one court, but not achieved in the second. However, as discussed above, the impact of the pilot model courts cannot be assessed because they are not yet fully operational.

4 An increase in the number of citizens served by mobile courts after they become operational. Achieved. Upgrading institutional integrity. Development and adoption of a Code of Conduct and extensive related training, the development and adoption of a gender -sensitive Human Resource Development Plan, the adoption of a Strategic Gender and Development Mainstreaming Plan, and the decentralization of Human Resources processes. There were two performance indicators: half of the judges and court personnel receive training on the Code of Ethics. The target was exceeded: 51% of judges and 84% of court personnel were trained. Implementation of a gender-sensitive Human Resource Development Plan: Achieved. Strengthening the institutional capacity of the judiciary. Establishment of an e-library made Supreme Court decisions widely and more quickly available. A technical standard was adopted for the construction of courthouses, based on the experience gained during the construction of project -supported courthouses. A new system for the operation and maintenance of courthouses was designed, including a new protocol for preventive maintenance of courthouses and IT systems.. The decentralization options started to be reviewed five months before closing. The development and implementation of a financial management improvement program has yet to take place. Support for the reform of the judicial system. The assessment of impact of the country's program on judiciary personnel and the subsequent preparation and implementation of an impact mitigation program did not happen. During preparation and until 2010, the Supreme Court carried out several stakeholder consultations to explore options and get support for reforms, but this was not sustained in the last years of the project. Performance indicator: maintain an active program of public education, knowledge sharing, consultations, citizen feedback and dissemination of result. This was not maintained. Outcomes: A five percentage-point increase in user satisfaction and confidence in Higher Courts. Not achieved. A five percentage-point increase in user satisfaction and confidence in pilot model courts. Not achieved. There is some indication of increased user satisfaction, but it cannot be validated for lack of consistent data. Further, the pilot mobile courts are still not fully operational because the computerization of information systems and improvements in case and court management are only partially in place and the additional courtrooms remain empty because the new courts which are to use them had not been created or staffed at closing. The number of pilot model courts and decentralized administrative and financial management offices completed and tested, including modern case management systems and adequate physical infrastructure. Partly achieved. The system of pilot model courts was designed and partly implemented : one court was completed but the related IT systems are not in place yet. The piloting did not take place. Channels of internal and external communication and stakeholder participation developed and made operational. Partly achieved. During project preparation and until early 2010, the Supreme Court coordinated wide-ranging stakeholder consultations to explore options and get support for reforms. After early 2010, the Supreme Court was totally focused on internal matters : the appointment of the next Chief Justice, and, later, his impeachment and conviction. Attribution: It is difficult, if not impossible, to attribute benefits to the project initiatives when it is part of a wider Borrower s program. There were several other multilateral and bilateral agencies also supporting the Borrower's Action Program for Judicial Reform. The PAD did not address this. 5. Efficiency: Negligible. The PAD does not include an Economic Rate of Return. However, it indicates that the cost of the project would be offset by increased tax revenue as a result of improved compliance, better revenue from court fees and improved efficiency of the Supreme Court operations. The speedy disposition of cases would also contribute to increased tax liability and generate substantial savings in expenses. Improved judicial efficiency would result in courts delivering a larger volume of judicial services without an increase in operating costs.

5 The ICR points to several implementation factors which impeded the completion of activities, and thus reduced efficiency, many of which were also identified in ISRs : (i) the overly complicated procurement procedures and lack of clarity of land titling resulted in major implementation delays; (ii) the Borrower's progressive shift of focus from outcomes to construction and equipment and disbursement; (iii) the shifting (upward or downward) support to reforms every time there was a change of Chief Justice; and (iv) the continuously decreasing capacity of the Project Management Office (PMO) related to the loss of interest to reforms from the Chief Justice (the PMO was part of the Supreme Court infrastructure). a. If available, enter the Economic Rate of Return (ERR ERR)/Financial Rate of Return (FRR FRR) at appraisal and the re-estimated estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal ICR estimate No No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of the objective is rated negligible while relevance of design is rated substantial. Efficacy is rated negligible. Efficiency is rated negligible. a. Outcome Rating : Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The project objective was not realized, and few of the expected outputs were achieved, in spite of the Bank s sustained support over the nine years of implementation. The few achievements (none contributing to key project outcomes) are tenuous because the government is not fully committed to the objective of the project. It is therefore unlikely that future progress will be made, or that the few achievements of the project will be sustained. a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: a. Quality at entry: Project design reflected lessons learned from Bank supported judicial reform projects in other countries. The preparation team was solid in terms of skills and experience. The project objectives were well aligned with the country s priorities at that time, with Bank priorities, as reflected in the 2002 CAS for the Philippines, and with the findings of the 2002 World Development Report, which identified courts as one of the key institutions required for market development. The PAD did not address the issue of attribution. While several risks were correctly identified, their risk ratings and their mitigation measures; other risks were not appropriately identified and assessed (see below). There were a few serious shortcomings in preparation. Project design was too ambitious in scale for a Borrower (the Supreme Court) which had no prior experience in implementing Bank financed operation and which at the same time was implementing its broad program with several other development partners. The Bank did not see the scope and complexity of the PMO's functions as an issue. Several other risks, were identified but their mitigation measures were either not strong enough or inadequate. The risk of land titling issues was not anticipated The only safeguard policy relevant to the project was environmental assessment and it was well addressed. Quality-at at-entry Rating : Moderately Unsatisfactory b. Quality of supervision:

6 The project team was competent and had a wide range of highly relevant expertise. The two TTLs (a third one was appointed for only a few weeks ) who supervised the project during the nine years of implementation had strong technical backgrounds (a former judge and a lawyer/procurement specialist) and solid relationships with the Chief Justices and the Supreme Court. While the project team fully supported project implementation, the evidence of management guidance and support is not well documented in the project file. For example, the relevant section in the majority of ISRs is either empty or repeats points already covered in the main text of the ISR. However, managers participated in several supervision missions and made available full funding for supervision, with the result that spending on supervision was almost equal to what was spent on preparation. The Bank could have downgraded its ratings much earlier, which might have led to more realistic remedial actions. Slow implementation and lack of progress in most activities were noted in early in ISRs and Aide-Memoires, and at the Mid-Term Review. The concern about the Borrower s commitment to reforms was noted in later ISRs. Yet, for the first seven years of implementation, the ratings for the eleven areas of performance, and most importantly the ratings for development objective and overall implementation success, were downgraded and upgraded several times, all of them ending up as satisfactory in It was only in 2010 that most ratings were downgraded. The remaining few were downgraded in 2012 (ICR p 13). Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government provided counterpart funds in a timely manner. Government Performance Rating Satisfactory b. Implementing Agency Performance: The Supreme Court turned out to be much less capable and /or willing to implement the project's institutional reforms and civil works than anticipated. Implementation greatly depended on the Chief Justice (and there were five during implementation). The ICR notes that a key recurring feature of implementation was the Borrower's disregard for recommendations made in high -quality analytical and diagnostic work to explore and evaluate reform options for various parts of the Action Program for Judicial Reform. Either decisions were not taken or taken but not implemented. The PMO lost several of its key staff with the result that these functions were not strengthened as designed, and that the PMO was less effective at the end of the project than at the beginning. See also section 11 para b Implementing Agency Performance Rating : Unsatisfactory Overall Borrower Performance Rating : Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The project results chain was well articulated, and measurable outcome and output indicators were identified in the PAD. The strengthening of the PMO s M&E capacity was included in project design. b. M&E Implementation:

7 The quality of M&E implementation fluctuated throughout the life of the project. It was highly related to the commitment each Chief Justice demonstrated, which also affected the PMO performance. This explains why ISR ratings for the eleven areas rated cover the whole range of the scale, all of them ending in unsatisfactory, except for the satisfactory counterpart funding. This resulted in sporadic efforts for results collection and for overall M&E. There were severe gaps in results information for completed activities, which made the assessment of outcomes harder. c. M&E Utilization: Whatever information was collected, it was not well used in the feedback loop to help the leadership refine the reform program. M&E Quality Rating : Negligible 11. Other Issues a. Safeguards: The project was rated C, but its design included an environmental assessment in view of the large construction component. Compliance with the environmental safeguard was mostly satisfactory, although not consistently observed and not always in a timely manner during part of the construction activities. The project had an unanticipated positive outcome: consultations with stakeholders in the context of the country's Action Program for Judicial Reform (and the Bank project which was part of the Program ) led to the adoption of new measures for environmental protection and the development of a writ to protect human rights. b. Fiduciary Compliance: According to the ICR (p.14), financial management was satisfactory for the first half of the project, and then deteriorated. The PMO capacity and its attention to fiduciary issues dropped substantially. The lack of segregation of functions led to difficult internal control. Procurement had been a challenge from the start : cumbersome processes, some ineligible expenses, retroactive contracts, and late payment to vendors resulted in slow implementation. c. Unintended Impacts (positive or negative): Consultations with stakeholders carried out under the project led to the adoption of new measures for environmental protection and the development of a writ to protect human rights. The success of the Small Claims Cases and the rise in successful mediation have positively impacted the project by improving access to justice, especially by the poor and near -poor. The multiple changes of Chief Justices could not have been anticipated. Depending on each Chief Justice's d. Other: commitment, the project experienced periods of high activity, and periods of total passivity, which in the end negatively impacted the achievement of outcomes. The risk of land titling issues was not anticipated. It turned out to be a strong obstacle to the execution of activities necessary to the achievement of outcomes. 12. Ratings: ICR IEG Review Outcome: Unsatisfactory Unsatisfactory Risk to Development High High Outcome: Reason for Disagreement /Comments

8 Bank Performance : Moderately Unsatisfactory Borrower Performance : Unsatisfactory Quality of ICR : Moderately Unsatisfactory Unsatisfactory Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR posits a number of interesting lessons. In addition, IEG notes the following: 1. Lack of coherence between what the Government and Borrower say about commitment to reform and the actions taken to substantiate this commitment can be detrimental to the project outcomes. Commitment to reforms, reaffirmed throughout the life of the project, did not match actions taken. During implementation, it is critical for the Bank to assess the project's progress on the basis of actions taken, and not on the basis of promises. 2. A well trained and fully staffed Project Management Unit is essential to successful implementation. The PMO for this project was responsible for the coordination and monitoring of all the projects supporting the country s Action Program for Judicial Reform, the Bank-supported projects and the projects supported by several other development partners. The PMO was not adequately staffed to handle such a wide range of projects. The scope and complexity of its functions became overwhelming, aggravated by the loss of key personnel. 3. Strengthening capacity in procurement, financial management, and M&E during preparation is critical to avoid early implementation delays. Weak competence in these areas will slow down implementation at the very early stages. These delays are difficult to reverse. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is clear, concise, and well articulated. Its analysis is solidly substantiated. Its candor permeates throughout, especially when it gives less than satisfactory ratings to sensitive dimensions. Throughout the whole document, the focus is on results, and the results framework is well presented. There are three areas where the ICR performed less well : Its lessons learned section is not as strong as the rest of the document. It could have offered newer and sharper lessons. The lessons it offers apply to every project in every sector and are already well internalized by Bank staff. The ICR itself notes: It may be more reasonable to refer to these as lessons reinforced rather than lessons learned from the experience of this project. It only listed two of the four outcome/impact indicators presented in the PAD (p 24) a.quality of ICR Rating : Satisfactory

9