Philippines: Financial Market Regulation and Intermediation Program

Transcription

1 Validation Report Reference Number: PVR-340 Project Number: Loan Numbers: 2278 and 2715 November 2014 Philippines: Financial Market Regulation and Intermediation Program Independent Evaluation Department

2 ABBREVIATIONS ADB Asian Development Bank BSP Bangko Sentral ng Pilipinas CMDP Capital Market Development Plan DMF design and monitoring framework DOF Department of Finance GDP gross domestic product GFC global financial crisis MTPDP Medium-Term Philippine Development Plan PCR program completion report PDIC Philippine Deposit Insurance Corporation PDEx Philippine Dealing and Exchange Corporation P3F post-program partnership framework PSE RRP Philippine Stock Exchange report and recommendation of the President SEC Securities and Exchange Commission SPV special purpose vehicle TA technical assistance NOTE In this report, $ refers to US dollars. Key Words asian development bank, capital market development program, country assistance program evaluation, country strategy and program, financial market regulation and intermediation program, medium-term philippine development plan, philippines, program completion report, technical assistance The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

3 PROGRAM BASIC DATA Project Number: PCR Circulation Date: Sep 2013 Loan Numbers: 2278 and 2715 PCR Validation Date: Nov 2014 Project Name: Financial Market Regulation and Intermediation Program Country: Philippines Approved ($ million) Sector: Finance Total Project Costs: ADB Financing: ($ million) ADF: 0.00 Loan: OCR: Actual ($ million) Borrower: Beneficiaries: Others: Cofinancier: Total Cofinancing: Approval Date: Signing Date: Project Officers: Validators: Quality Reviewer: 6 Dec Dec Dec Feb 2011 V. Subramanian S. Schuster R. Limjoco, Consultant F. D. De Guzman, Senior Evaluation Officer, IED2 E. Gozali, Principal Evaluation Specialist, IED1 Effectiveness Date: Closing Date: Location: ADB headquarters ADB headquarters Peer Reviewer: Director: 11 Mar May Dec Apr 2012 From: Dec Dec Jun Dec Apr 2012 To: Dec 2006 Dec 2010 Apr 2012 L. Hauck, Senior Evaluation Specialist, IED2 B. Finlayson, IED2 ADB = Asian Development Bank; ADF = Asian Development Fund; IED1 = Independent Evaluation Department, Division 1; IED2 = Independent Evaluation Department, Division 2; OCR = ordinary capital resources; PCR = program completion report. A. Rationale I. PROJECT DESCRIPTION 1. The Medium-Term Philippine Development Plan (MTPDP) for targeted a gross domestic product (GDP) growth rate of 7% 8% by 2010 (which was lowered to 5.7% 6.6% in the updated MTPDP in 2008 after the global financial crisis [GFC]). The National Economic and Development Authority estimated that to meet the growth targets, a savings to GDP ratio of at least 30% and an increase in investment ratio of 28% of the GDP were needed. In general, there was a need to increase the contribution of the financial sector to economic growth by spurring private investments. This required a broad range of revamped

4 2 financial services and instruments. The Financial Market Regulation and Intermediation Program consisting of two single-tranche subprograms subprogram 1 and subprogram 2 was prepared to broaden and deepen the financial system in the Philippines. This was to be achieved through measures that improve market efficiency, promote stability, strengthen governance, and diversify investment alternatives. B. Expected Impact 2. The design and monitoring framework (DMF) in the two reports and recommendations of the President (RRP) 1 indicated that the expected project impact was increased contribution of the financial sector to economic growth, poverty reduction, and development of the Philippine economy. The impact statement and the impact indicators were modified in the DMF of the program completion report (PCR), which evaluated the cluster program as a whole. 2 C. Objectives or Expected Outcome 3. The 2006 RRP (for the program cluster) listed three expected outcomes: (i) enhanced financial system stability, (ii) strengthened nonbank financial sector governance, and (iii) improved securities market efficiency. Synthesizing these objectives and reflecting the new emphasis in subprogram 2, the PCR indicated only one intended, revised outcome a deeper, more diversified and resilient financial sector for the Philippine economy. D. Outputs 4. The subprogram 1 was designed to focus on capital markets and other nonbank services, including policy measures designed to reduce financial sector vulnerabilities and increase the depth and diversity of financial intermediation. As in subprogram 1, subprogram 2 had to undergo some adjustments. It originally consisted of an indicative reform agenda that aimed to continue the reforms of enhancing financial system stability and in the financial sector in general. However, in 2008, the GFC erupted and exposed the Philippine economy to the risk of global volatility. As a response, subprogram 2 was reformulated to address new considerations and the government s refocused priorities. These included (i) enhanced financial sector stability to cope with global financial turbulence, (ii) strengthened financial sector regulatory oversight to support government s efforts to coordinate micro and macro prudential supervision and mixed supervision caused by the use of separate, functional regulators to supervise complex business diversified conglomerates, and (iii) improved financial sector efficiency and liquidity. 5. Enhancing financial sector stability under subprogram 1 was to be achieved by strengthening a number of areas. First, the Bureau of Treasury was to improve its debt and risk management by reducing the share of variable rate international bonds, altering the composition of international debt (by diversifying and borrowing in a range of currencies while the government s policy was aimed at reducing foreign currency borrowing) and consolidating the outstanding domestic debt and creating benchmark for peso-denominated bonds. This was to 1 ADB Report and Recommendation of the President to the Board of Directors: Proposed Program Cluster, Loan and Technical Assistance Grant to the Republic of the Philippines for the Financial Market Regulation and Intermediation Program. Manila; and ADB Report and Recommendation of the President to the Board of Directors: Proposed Loan for to the Republic of the Philippines for the Financial Market Regulation and Intermediation Program. Manila. 2 ADB Completion Report: Financial Market Regulation and Intermediation Program in the Republic of the Philippines. Manila.

5 3 be followed up with the conduct of a needs assessment of the newly created debt and risk management division at the Bureau of Treasury, the drafting of a public debt management strategy, and preparation of a debt sustainability assessment under subprogram The second focus area under subprogram 1 was on further reducing nonperforming assets within the financial system by amending Republic Act No or the Special Purpose Vehicle (SPV) Act to extend the deadline for establishing SPVs and, correspondingly, extend the deadline for availing of tax benefits under the SPV act. The third focus area was on strengthening the capacity of the Philippine Deposit Insurance Corporation (PDIC) and broadening its methods for undertaking asset management resolution and liquidation to enhance the framework and guidelines for assisting distressed banks and ensure the adequacy of the country s Deposit Insurance Fund. 7. The fourth focus area under subprogram 1 was strengthening the financial condition of nonbank financial institutions by (i) raising the minimum capitalization for new life, nonlife, and reinsurance companies, and for existing insurance companies and insurance brokers; and (ii) issuing new risk-based capital rules for life and nonlife insurance companies. was also intended to strengthen nonbank financial subsector governance by adopting the Capital Market Development Plan (CMDP), , which is a blueprint for developing the domestic capital market. The aim was to strengthen accounting and financial reporting with the Securities and Exchange Commission (SEC) and the Insurance Commission by requiring financial institutions under their respective jurisdictions to comply with the new Philippine accounting and financial reporting standards. 8. To improve the securities market efficiency under subprogram 1, this involved setting up the Philippine Dealing and Exchange Corporation (PDEx) as a fixed-income exchange on an interdealer basis, serving dealers in government securities through a screen-based approach. The SEC was also to adopt rules to promote greater pricing transparency in the over-thecounter market. E. Provision of Inputs 9. The program cluster was to assist the Philippines with a $400 million loan comprising two single-tranches of $200 million equivalent each for subprograms 1 and 2. The loans were sourced from the Asian Development Bank (ADB) ordinary capital resources. The first tranche was released on 22 December 2006, soon after the loan effectiveness. The second tranche was delayed as subprogram 2 slipped and this was released on 13 September F. Implementation Arrangements 10. The Department of Finance (DOF) was the executing agency and is responsible for the overall implementation of the program, including compliance with actions in the two policy matrixes, program administration, disbursements, and maintenance of all subprogram records. Under subprogram 1, the SEC was the sole implementing agency. It was tasked to set up a program coordination committee headed by the SEC chairperson, and to comprise representatives from the Bangko Sentral ng Pilipinas (BSP), DOF, Insurance Commission, PDEx, PDIC, and Philippine Stock Exchange (PSE). The committee was responsible for implementing and sustaining the program components, particularly in implementing the action plan in the CMDP.

6 4 11. The performance of the SEC under subprogram 1 was not in accordance with the program s design. Jurisdiction over the SEC at this time had been transferred from the Department of Finance to the Department of Trade and Industry. Hence, under the reformulated subprogram 2, the SEC was transferred back to the DOF and the DOF assumed responsibility as the executing agency. The DOF chaired the coordinating committee that included representatives from the BSP, PDIC, Insurance Commission, PDEx, and PSE. 12. The technical assistance (TA) on Strengthening Regulation and Governance in the amount of $800,000, sourced from the Japan Special Fund on grant basis, was provided in tandem with the program cluster to support the Insurance Commission, PDIC, and SEC in preparing and implementing key policy reforms. The TA project had three main components. Component 1 was to assist the PDIC strengthen its bank resolution methods by developing a framework to increase the efficiency of asset disposal. This component was rated highly successful and led to amendments in the PDIC Charter, which strengthened its powers and flexibility. Component 2 was designed to strengthen the capacity of the SEC to supervise and regulate secondary trading of fixed-income securities, and to support the development of an action plan for moving to a dematerialized environment. The PCR noted that the implementation was incomplete and with numerous deviations, thus rated it partially successful. Component 3 was to provide support to the Insurance Commission. This component was considered unsuccessful as few of the targeted reforms were introduced within the time frame of the TA project. The assistance had to be withdrawn by ADB due to performance issues associated with the Insurance Commission s contribution to the program. Performance improved under the new Insurance Commissioner. On the whole, the TA project was rated partially satisfactory. 13. Other ongoing ADB TA projects were generally supporting the financial reforms. One was aimed at improving the national debt and risk management function and establishing a process of more effective monitoring and coordination of debt and risk management functions. 3 The others Support for Nonbank Financial Governance II 4 and Strengthening Governance of Securities Trading Markets 5 for the SEC were designed to evaluate compliance with the risk-based capital for brokerage firms and the effectiveness of the new, self regulatory organization structure of the PSE in addressing conflict-of-interest concerns and its capacity to effectively perform its surveillance and audit functions. All these measures, including a public awareness campaign on the role of the new fixed-income exchange, were expected to support the SEC and other key stakeholders in fulfilling the policy actions under subprograms 1 and 2. II. EVALUATION OF PERFORMANCE AND RATINGS A. Relevance of Design and Formulation 14. The PCR rated the program cluster relevant to the government s reform agenda. It indicated that the program was fully aligned with the MTPDP reform agenda, and together with the associated TA projects, it directly supported initiatives of promoting a more efficient financial sector that could finance profitable private investment projects at reduced cost and risk. This 3 ADB Technical Assistance to the Republic of the Philippines for Debt and Risk Management Project (TA 4717-PHI). Manila. 4 ADB Technical Assistance to the Republic of the Philippines for the Support for Nonbank Financial Governance II (TA 4168-PHI). Manila. 5 ADB Technical Assistance to the Republic of the Philippines for Strengthening Governance of Securities Trading Markets (TA 4321-PHI). Manila.

7 5 validation agrees and notes that the program was well focused on developing the domestic capital market (CMDP, ). It was also fully consistent with ADB priorities that recognized that one of the binding development constraints to the Philippine economy was the poor investment climate. However, the design of the program revealed certain limitations during the implementation of subprogram 1. At program formulation, the capacity of the SEC was not judged accurately. The SEC did not have the requisite capability to deliver what had been entrusted to it. It was unable to get a number of laws passed (PCR, para.15).the PCR noted that the SEC was responsible for 25 of 39 indicative policy actions. Only nine of these were cited as fully accomplished while one was reported as partially accomplished in a government readiness assessment in The SEC did not strengthen the market surveillance, monitoring of broker-dealers, and PSE self-regulatory functions within the planned time frame. In comparison, the Insurance Commission faced the most significant delays, with only two of six indicative policy actions accomplished, while the PDIC showed the most commitment to reforms (PCR, p.10, footnote 23). Due to misjudgment on the limited capacity of the SEC, the pace of reforms was not only significantly delayed (PCR, para. 6) but was also off the intended track. On top of it, the risks arising from the 2008 GFC added to the financial problems of the sector. 15. Taking advantage of the inherent flexibility of the cluster modality, subprogram 2 had to be reformulated in 2010 to address risks from the GFC. This enabled the government to focus on strengthening financial sector stability and supervision, and redressing weaknesses in the SEC and Insurance Commission (PCR, para. 23) while undertaking initiatives to preempt the effects of the 2008 GFC. The program rightly continued with the modality of a medium-term cluster program with front-loaded single tranches. 16. Given the major inadequacy in the formulation of the program related to the assessment of SEC s capacity to deliver the reforms, this validation s assessment should be expected to be less than relevant. The SEC should not have been the sole implementing agency as it did not have the leverage to push difficult reforms in other areas besides its own. It lacked both the capacity and budget to even enforce regulations. However, the program displayed alertness to the emerging GFC and showed flexibility in turning the program around by a skillful reformulation. This validation concurs with the move to shift the program to the DOF, which holds the purse strings and has better leverage over the involved agencies to implement their own reform programs. The DOF was able to manage the implementation of subprogram 2 quite well. Taking the two together, this validation assesses the program relevant. B. Effectiveness in Achieving Project Outcome and Outputs 17. The PCR rated the program effective in achieving the outcome in view of the achievements of a deeper, more diversified, and resilient financial sector. It indicated that eight out of nine outcome indicators in the original and revised DMF, including those discontinued under subprogram 2, either met their targets (5) or progressed (3) well. In particular, it reported that the country s improved fiscal position, declining budget deficits, and the extended maturity structure of outstanding government debt led to an upgrade of the Philippines credit rating to investment grade. 18. This validation notes that the size of the local debt and equity markets both increased with issuances of equity and bonds, representing 40% of total financing in 2012 although the share of nonbank financial sector, as a percentage of total financial assets, barely met the 23% target. The stock market to GDP ratio overshot the targeted 50% 55%, with a ratio of 73.6% in 2011, although this may be due in large part to the accommodative monetary policy in the United States and Europe.

8 6 19. However, the outputs and outcomes envisaged at appraisal have been far from satisfactory. Shortfalls under the original subprogram 2 included the fact that subsequent to disbursement in 2006, the OIC implemented a significantly different reorganization plan without addressing the structural weaknesses that had given rise to the original plan, 6 or notifying ADB (PCR, footnote 22). By 2011, program and TA activities under the program cluster had contributed to the target achievement of only 48% of 44 performance indicators in the original and the revised DMF. The PCR noted that progress was visible in another 30% of the DMF indicators, while performance deteriorated for the remaining 23% (para. 31 and Appendix 6). Outcomes under subprogram 2 have been better and much of this success is attributed to the reformulated program. However, this validation considers that reform progress under subprogram 2 was due mainly to the change in government in 2010 and, to a lesser extent, a response to this program. The breadth and depth of reforms originally envisaged at appraisal had not been realized at program closing (PCR, Appendix 6). On this basis, this validation rates the program less than effective. C. Efficiency of Resource Use in Achieving Outcome and Outputs 20. Precise assessments on the efficiency of a program loan are difficult to be accurate. This is all the more difficult in this case where the implementing agency could not deliver a satisfactory performance under subprogram 1, as described earlier in the outputs and outcomes. Determination of efficiency in the context of the financial sector is especially hard given the fluidity of the sector markets and its dynamics. The contextual conditions of operations keep changing constantly rendering the previous prescriptions of the policy matrixes out of date. These have to be revisited and need to be calibrated periodically. Due to the significant delay in the implementation of subprogram 1, which necessitated an extension of the program cluster period by 2 years, the PCR rated the program less than efficient. This delay was attributed to (i) a lack of political will to complete difficult reforms in the insurance sector, (ii) the time needed to develop legislation and resistance to a narrow range of reforms within the SEC, and (iii) select capital market stakeholders. This validation agrees with the assessment and rates the program less than efficient. D. Preliminary Assessment of Sustainability 21. The PCR rated the program likely sustainable. The reforms are largely legislative in nature and these contributed to program delays. Others pertained to the adoption of international best practice, especially accounting and financial reporting standards, International Organization of Securities Commissions, and Basel framework and principles. The PCR indicated that with the strengthening of the capital base of the agencies involved under the program, the capacity and institutional development efforts are unlikely to be reversed. Moreover, these reforms are supported by other multilateral institutions and the global standard-setting agencies. Given the institutional nature of these reforms, this validation concurs that these are likely to be sustained. More importantly, under the program, the government and ADB agreed to a post-program partnership framework (P3F) to establish a direction for continuing reforms. This medium-term action plan contained a series of indicative actions designed to continue the momentum of reform beyond subprogram 2. This P3F initiative is laudable if implemented earnestly as it is likely to contribute tremendously to the sustainability of reforms not only in the financial sectors but also in other sectors of ADB operations in the Philippines. The downside of the situation is that the Philippine country 6 OIC or office of the Insurance Commission is the same as the Insurance Commission.

9 7 partnership strategy for does not prioritize financial sector development. Notwithstanding, this validation rates the sustainability of the program likely sustainable. E. Impact 22. The impact of the program was rated moderate by the PCR. The PCR acknowledged the difficulty of isolating the effects of the program reforms on overall economic and financial development especially after the GFC. Of the 10 impact indicators in the original and revised DMF, four targets were achieved, four indicators progressed, and two regressed. The four achieved indicators refer to real GDP growth, investment ratio to GDP, Financial Stability Index, and quarterly bond turnover ratio (annual average). The four indicators that progressed or not fully achieved were savings to GDP rate, Business Constraint Index access to credit, money and close substitutes for money (M2)/GDP ratio, and overall financial access score. 23. The full impact of financial sector contribution to economic growth consequently to poverty reduction and the development of the country is not likely to be immediately visible although GDP growth rate has been better than expected in recent years. The PCR noted a number of constraints to the economy in this regard. These included (i) weak fiscal balances; (ii) a weak investment climate; (iii) inadequate infrastructure; (iv) poor management of assets, land, and resources; (v) low institutional capacity; and (vi) geographical inequalities (PCR, footnote 4). However, considering the mild effect of the GFC on the economy (partly due to the low export dependency of the country), the cost to the economy could conceivably have been higher if not for the reforms. Nonetheless, it should be noted that the Philippine financial system remains over concentrated, with the savings rate still lagging behind its larger neighbor countries in the Association of Southeast Asian Nations (ASEAN), 7 deposit intermediation is still relatively low (only about 30% of households have savings with 23% of them keeping their savings at home), 8 while other parts of the financial market remain relatively underdeveloped. The contribution of the reform program, on balance, is therefore considered moderate. 24. No adverse environmental, resettlement, safeguards, governance, or fiduciary concerns were identified. III. OTHER PERFORMANCE ASSESSMENTS A. Performance of the Borrower and Executing Agency 25. The PCR found the performance of the borrower and the executing agency less than satisfactory. The legislative process was slow and, in retrospect, too much authority for managing reforms may have initially been assigned to the SEC, which was not able to utilize the opportunity to implement reforms effectively. The level of ownership and support of the CMDP also varied across other agencies and private sector organizations. Given these constraints, even the commitment of SEC waned over time. Performance improved after the replacement of the SEC under subprogram 2. This validation concurs with the PCR rating of less than satisfactory owing to the considerable 2 years delay in program implementation. B. Performance of the Asian Development Bank 26. The PCR rated the performance of ADB satisfactory citing that ADB actively engaged with the government, sought to resolve outstanding issues, improved implementation 7 The World Bank DATA Gross Savings (% of GDP). Washington, D.C. 8 Bangko Sentral ng Pilipinas Consumer Expectations Survey: First Quarter, Manila.

10 8 arrangements, and ensured program readiness at implementing agencies. There were no formal program review missions after appraisal under subprogram 1. However, subprogram 2 was more closely followed with the help of one consultation and three loan review missions before closure. Since the Philippines is where the ADB headquarters are situated, policy dialogue is assumed to have continued with the government representatives in-between subprograms 1 and 2. Likewise, considering the nature of the reforms, many of which required legislative action and contentious changes, such as capital increases, it is not likely they could have been achieved even with the delay without ADB intervention. This validation agrees with the PCR rating of satisfactory. IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS A. Overall Assessment and Ratings 27. Overall, the validation assesses the program as less than successful. It is relevant in view of its consistency with both the government s and the ADB strategies and plans. The choice of single modality instrument was appropriate to give subprogram 2 the flexibility to adjust to circumstances. However, the program is considered less than effective since substantial measures envisaged at appraisal were not realized up to 2011, while alternative reforms under subprogram 2 progressed mainly because of the severe impact of the GFC and less due to the program. The program was less than efficient in view of the substantial implementation delay, which is critical for the financial reforms. The program is likely sustainable since the reforms achieved at completion are not likely to be reversed and that ADB and the government have agreed to continue to work together under P3F. Its impact is moderate, as per the sector indicators, and since the financial sector reforms are still ongoing. This validation notes that from the program closing until 2014, the current government has carried out more financial sector reforms, including in areas originally envisaged under the program. These include reforms in the pension and nonbanking sector. Given the continuing high level of commitment to reforms and institution building, this validation supports the continuing engagement of ADB in the financial sector. Overall Ratings Criteria PCR IED Review Reason for Disagreement and/or Comments Relevance Relevant Relevant Effectiveness in achieving project outcome and outputs Effective Less than effective Efficiency in achieving outcome and outputs Less than efficient Less than efficient Preliminary assessment of sustainability Likely sustainable Likely sustainable Impact Moderate Moderate Overall assessment Successful Less than successful Borrower and executing agency Less than satisfactory Less than satisfactory Substantial measures envisaged at appraisal were not realized, while alternative reforms under subprogram 2 made headway mainly because of the severity of the global financial crisis and less attributed to the program (paras ).

11 9 Performance of ADB Satisfactory Satisfactory Quality of PCR Satisfactory Sufficiently detailed with clear grasp of program components. Linkage of various components between subprograms and the choice of reform actions were not adequately explained. The report was clear and well written (para. 35). ADB = Asian Development Bank, IED = Independent Evaluation Department, PCR = program completion report. Note: From May 2012, IED views the PCR rating terminology of "partly" or "less" as equivalent to "less than" and uses this terminology for its own rating categories to improve clarity. Source: ADB Independent Evaluation Department. B. Lessons 28. One of the lessons drawn by the PCR is that programs and TA work should not depend on the passage of law in countries where legislative processes are slow and cumbersome. On the other hand, ADB backing and support may be crucial for authorities to be able to push for reforms in difficult political environments, such as the Philippines. This should be weighed more carefully in undertaking critical reform programs, although attention should be paid for improving the process. 29. The second lesson noted by the PCR is the need to undertake a deeper and broader analysis of the political economy for ADB to better gauge the prospect of reforms within the program time frame. The PCR also noted, for example, that the CMDP was initiated unilaterally by the SEC and, clearly, it was not reflective of industry sentiment, hence, it did not enjoy widespread support. In the case of the Insurance Commission, pressure from the industry to postpone higher required capitalization contributed to a change in the Insurance Commissioner and a reduction in political will in support of reform. Reforms clearly require consensus building and consistent support. Moreover, in countries such as the Philippines, one needs to have an influential and bold champion behind reforms. 30. The third lesson learned is program design should focus on implementing reforms not only where commitment is strong but also where resources and external factors are adequate to ensure successful implementation. A program administration memorandum that defines actions, timelines, and costs, especially for complex, complicated reform programs may be useful. 31. Parallel reforms in related areas are also critical components to ensure the success of sector reform programs, particularly where reforms may depend on reforms in other sectors (such as governance and fiscal management) or cut across sectors. C. Recommendations for Follow-Up 32. has a P3F that could be followed up, including unfulfilled commitments, such as increased budget allocations. Policy dialogue could continue to leverage reforms in the key aspects of the reform program cluster, particularly since the ADB country partnership strategy for does not prioritize financial sector development. 33. As in other PCRs and validations, the observation that DMF performance indicators should exhibit a close causal relationship with contemplated program activities remains valid.

12 10 As noted by the PCR, the use of broad economic indicators introduces additional factors into the assessment and complicates the evaluation of effectiveness. V. OTHER CONSIDERATIONS AND FOLLOW-UP A. Monitoring and Evaluation Design, Implementation, and Utilization 34. It appears that for complex, complicated programs, these would benefit from a problem tree analysis that could help in understanding the proposed program by (i) defining the core problems; (ii) putting together all the various components of the reform program; (iii) showing what key issues are being resolved and how they are related; (iv) defining their place in the continuum of reforms needed to meet the country s development objectives; and (v) helping set up the crucial linkages between outputs, outcomes, and impact. Further, in program implementation, a program administration memorandum, as mentioned earlier, may be of great value. B. Comments on Project Completion Report Quality 35. The PCR quality is deemed satisfactory. It contains sufficient detail and supporting data and candidly discussed challenges, particularly in implementing subprogram 2. However, the linkages and the choice of reform actions supported under each subprogram and differences between subprograms 1 and 2 were not dealt with at length. It was clear, well-written, and reflected an understanding of the program cluster and its various components. C. Data Sources for Validation 36. This validation relied mainly on the (i) PCR; (ii) RRPs of subprograms 1 and 2; (iii) ADB mission reports (appraisal and subprogram 2 monitoring); (iv) the country strategy and program, ; (v) country assistance program evaluation, 2003 and 2008; (vi) MTPDP, ; (vii) CMDP, ; and (viii) the International Monetary Fund (IMF) Financial Sector Assessment Program. D. Recommendation for Independent Evaluation Department Follow-Up 37. A program performance evaluation report could be undertaken to have more in-depth assessment of policy actions and data.