Validation Report. Tonga: Economic Support Program. Independent Evaluation Department

Transcription

1 Validation Report Reference Number: PVR-318 Program Number: Grant Number: 0185 September 2014 Tonga: Economic Support Program Independent Evaluation Department

2 ABBREVIATIONS ADB Asian Development Bank DMF design and monitoring framework GDP gross domestic product MFNP Ministry of Finance and National Planning PCR program completion report PFTAC Pacific Financial Technical Assistance Center RRP report and recommendation of the President SPFMP Strengthening Public Financial Management Program TA technical assistance NOTE In this report, "$" refers to US dollars, unless otherwise stated. Key Words adb, financial accountability, fiscal framework, global crisis, pefa, performance evaluation, policy reform program, public expenditure, public financial management, sustainability, technical assistance, tonga. The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

3 PROGRAM BASIC DATA Program Number: PCR Circulation Date: Oct 2012 Grant Number: 0185 PCR Validation Date: Sep 2014 Program Name: Economic Support Program Country: Tonga Approved ($ million) Actual ($ million) Sector: Multisector Total Program Costs: ADB Financing: ($ million) ADF:10.00 Grant: Borrower: OCR: 0.00 Beneficiaries: Others: Cofinancier: Total Cofinancing: Approval Date: 3 Dec 2009 Effectiveness Date: 21 Mar Feb 2010 Signing Date: 21 Dec 2009 Closing Date: 31 Dec Jun 2011 Program Officers: Validator: Quality Reviewer: M. Melei E. Veve E. Veve G. Walter, Consultant E. Gozali, Principal Evaluation Specialist, IED1 Location: SPSO SPSO ADB headquarters Peer Reviewer: Director: From: Dec 2009 Mar 2010 Jan 2011 To: Feb 2010 Dec 2010 Jun 2011 J. Asquith, Senior Evaluation Specialist, IED1 W. Kolkma, IED1 ADB = Asian Development Bank; ADF = Asian Development Fund; IED1 = Independent Evaluation Department, Division 1; OCR = ordinary capital resources; PCR = program completion report; SPSO = Pacific Subregional Office. A. Rationale I. PROGRAM DESCRIPTION 1. The $10.0 million Economic Support Program grant to the Government of Tonga was approved on 3 December 2009, to be provided in two tranches of $5.0 million each. The report and recommendation of the President (RRP) 1 indicates that the program is a budget support at the time of the global economic crisis, designed to help Tonga stimulate the economy and continue with structural reforms to maintain stability, promote more equitable and sustainable growth, and create an economy that is more resilient to future shocks. At the same time it would protect the most vulnerable members of society from the impacts of the crisis by avoiding cutting back on essential expenditures on health and education services. 2. The global crisis had a severe impact on Tonga s economy and on household incomes, with declines in remittances, reduction in trade tax receipts, weak export demand, declining tourism revenues, and weakening of credit flows within domestic banking. Projected government revenue for FY2010 was down by 18.6%, with a projected fiscal deficit of 10.3% of gross domestic product (GDP), and GDP growth down to 0.4%. Remittances, which historically accounted for around 30% of GDP, had declined by 17% placing pressure on the informal social safety nets this might have been critical with a deep cut in budgets. 3. To maintain sound fiscal management, expenditure needed to be reduced, and critical reforms introduced to address underlying constraints of the economy. Both the RRP and the 1 ADB Report and Recommendation of the President to the Board of Directors: Proposed Asian Development Fund Grant to the Kingdom of Tonga for the Economic Support Program. Manila.

4 2 program completion report (PCR) indicate a limited government capacity to undertake reforms because of a commitment to raise wages and to fund reconstruction of parts of the city commercial area following the civil service strike in 2005 and civil unrest in The grant would help minimize cuts in essential services while maintaining fiscal responsibility and enabling the government to implement an appropriate expenditure package to promote growth, and continue structural reforms to strengthen foundations for growth. The Asian Development Bank (ADB) had provided significant analytical work and technical assistance (TA) in public sector financial management, 3 and structural reform since 2001, with specific TA in 2004 and 2006 in strategic planning, budgeting, and developing medium-term budget framework. 4 B. Expected Impact 4. The anticipated impact of the program was higher and more inclusive economic growth. The provision of budget support would allow the government to respond to the economic downturn caused by the global economic crisis by stimulating the economy, while maintaining essential services to the vulnerable population. Over time, the program would help in the transition of the economy to more inclusive growth. The program s design and monitoring framework (DMF) included indicators for GDP to grow progressively by about 3% a year, together with improvement in the relative standard of living of the most vulnerable groups, and reduction in the proportion of the population living below the national poverty baseline. Quantitative baselines, however, were not provided to be able to measure such improvements. C. Objectives or Expected Outcome 5. The DMF states that the anticipated program outcome was for a timely and effective response to the global economic crisis (RRP, para. 42). This response included reforms in public sector management, state-owned enterprises, and the private sector to improve economic productivity and medium-term growth. Efforts were to be made to maintaining essential services to the vulnerable groups. Indicators for good outcomes in the DMF were (i) targeted positive economic growth in 2010 and 2011, (ii) an improvement in the business environment (as signified by a rise in the country s ranking in the World Bank s Doing Business survey), and (iii) health and education service standards and access in the outer islands were maintained. D. Outputs 6. The policy matrix and the program were structured around five outputs. It included 16 policy actions under the first tranche, and 18 policy actions under the second tranche. 7. Output 1: Budget support to protect expenditure on basic social services for the vulnerable. This was to be achieved in the FY2010 budget by not cutting essential social services, but by making savings through a planned and strategic cut in lower priority expenditures. This was a first tranche condition. 2 ADB Completion Report: Economic Support Program in Tonga. Manila. 3 The Development Policy Letter notes that improved public sector financial management in particular is a priority, together with enhanced capacity of the public service. 4 ADB Country Partnership Strategy; Tonga, Manila. See also paras. 26 and 31 of the RRP.

5 3 8. Output 2: Maintenance of fiscal responsibility. This focused on financial management reforms and included five first tranche release conditions (covering public financial management reforms, including debt risk management and mitigation policy), and six second tranche conditions (including improved processes for macroeconomic and revenue forecasting, introduction of financial ratios in budget guidelines, and procurement guidelines). 9. Output 3: Structural and governance improvements. This area focused on improving governance of public enterprises, and included four first tranche conditions (improved management and rationalization of public enterprises) and six second tranche conditions (including introduction of legislations covering public enterprises and personal property securities, companies registrar, and infrastructure investment plan). 10. Output 4: Support to the vulnerable groups through targeted actions. This involved the maintenance of access and standards of education and health services in the outer islands, and implementation of a new community program in at least two districts. The three first tranche release conditions were (i) Cabinet adoption of the community development design, (ii) programs established in at least two districts, and (iii) commitment to review services in the outer islands and the district and community health centers. The three second tranche release conditions were (i) public expenditure tracking system for health and education commissioned, (ii) action plan to implement review recommendations adopted, and (iii) Cabinet consideration of a paper on issues for vulnerable persons. 11. Output 5: Communication and ownership. An effective communications strategy was to be established, explaining the government s response to the global economic crisis, and to include three first tranche and three second tranche release conditions (communication and engagement strategy and release of enterprise and other performance data). E. Provision of Inputs 12. The grant proceeds were released in two tranches of $5.0 million each, the first in March 2010, and the second in June The first tranche was delayed for 3 months while the second tranche was delayed for 8 months. The local currency funds generated by the grant were to be used to support the continued implementation of structural and public financial management reform measures, and the protection of core social expenditure. 13. Technical assistance. There were no consultancy services directly associated with the grant, however, ADB had provided TA in priority reform areas including structural, public sector, and financial management reform and these continued during program implementation (e.g., assistance to develop a medium-term budget framework) In addition, ADB and the Pacific Financial Technical Assistance Center (PFTAC) helped the government develop a road map for public financial management reform based on the 2007 and 2010 findings of the Public Expenditure and Financial Accountability Assessments. F. Implementation Arrangements 15. Implementation arrangements set out in the RRP were followed. The Prime Minister's Office was the executing agency and assumed overall responsibility for program implementation, including coordination of the policy reform measures undertaken under the 5 ADB Technical Assistance to the Kingdom of Tonga for Support for Economic and Strategic Management. Manila (TA 7475-TON).

6 4 program. The implementing agencies were the Ministry of Finance and National Planning (MFNP); the Ministry of Public Enterprises; the Ministry of Labour, Commerce and Industries; the Ministry of Education; and the Ministry of Health. The Expenditure Review Committee functioned as the steering committee, providing guidance for program implementation and coordination, and the Policy Unit of the Prime Minister s Office was the committee s secretariat. II. A. Relevance of Design and Formulation EVALUATION OF PERFORMANCE AND RATINGS 16. The PCR rated the program relevant, 6 budget support being highly relevant and timely, and the choice of two tranches helping ensure effectiveness. It also noted that the design of the outputs was in line with the country s strategic development priorities; the ADB Country Partnership Strategy, ; 7 and the Country Operations Business Plan, The PCR further referred to the program s alignment with Tonga s Strategic Development Plan Eight ( ) and the 2009 National Strategic Planning Framework. 9 The PCR stated that the interventions were appropriately designed to achieve the intended outcomes and impacts, but there was not enough evidence to show that key outcomes could be achieved through the proposed interventions and reforms (paras ). It also listed the lessons from the ADB experience in policy-based lending in the Pacific, which the RRP indicates were incorporated into the design (e.g., government ownership, political and technical feasibility of the reforms, focused policy actions, and champions for change) but did not demonstrate their relevance and how these were incorporated. Technical feasibility, however, is implicitly covered in the section on output achievements, while political feasibility and commitment was implicitly covered in the section on performance of the borrower. 17. This validation rates the program relevant, verifying that the program was in line with the government s and ADB plans and strategies. As noted in paras. 2 3, the global economic crisis had seriously affected Tonga and the government faced constraints in trying to stimulate the economy due to revenue shortfalls and expenditure commitments (wage increase and reconstruction) it had made after the civil service strike and civil unrest. Budget support from the grant would enable the government to continue its reform program and maintain expenditure for social programs. The reform program was designed to improve financial management and address the prime causes of low productivity within the economy, including the low rate of return on assets controlled by public enterprises, and the challenging business environment faced by the private sector. Reforms in public enterprises would deliver higherquality but lower-cost services needed for economic growth, and provide opportunities for local businesses to participate in service delivery. However, the timetable for reform was perhaps overambitious given capacity constraints, a lesson that has emerged from several reform programs in the Pacific and elsewhere. B. Effectiveness in Achieving Program Outcome and Outputs 18. The PCR rated the program effective. However, it provided limited analysis at the outcome level to support this. The main program outcome was for a timely and effective 6 Appendix 3 of the PCR rated the program highly relevant. A rating of relevant, however, would not have affected the overall rating of successful. 7 ADB Country Partnership Strategy: Tonga, Manila. 8 ADB Country Operations Business Plan: Tonga, Manila 9 Government of Tonga Kingdom of Tonga: Strategic Development Plan Eight: 2006/ /2009. Nuku alofa; and Government of Tonga National Strategic Planning Framework. Nuku alofa.

7 5 response to the global economic crisis. The DMF tracked this achievement through three outcome indicators. The first is positive economic growth for FY The DMF showed a GDP growth of 1.9% and 1.4% for the 2 years, thus meeting the outcome target. As a second outcome indicator, the DMF included the World Bank s Doing Business ranking as an indicator for improved business environment. It reported Tonga s moving up from rank 62 to rank 58 between 2011 and 2012, and there has been a general rise in the ranking since Hence, the second target was achieved. A third outcome target was maintaining access and service standards in outer islands, which were to be tracked through data from the health and education ministries and the Public Service Commission. Achievement of these service standards were not presented in the PCR, instead, the PCR reported that funding to outer island health facility positions had been maintained. 19. The program was designed to achieve five output areas; achievements have been substantial as summarized below. This validation has verified these to the extent possible, but apart from the report of the 2010 program review mission, the only documents readily available for this validation are the June 2013 Concept Paper and the September 2013 RRP for the follow-on grant for Strengthening Public Financial Management Program (SPFMP), 11 which records progress in public sector financial management and fiscal reforms. 20. Output 1: Budget support to protect expenditure on basic social services for the vulnerable. This validation notes that according to the mission report dated March 2010, 12 budget allocations had actually been cut for basic health and primary education, which contravened a tranche 1 condition. The DMF likewise recorded that the FY2010 budget outturn for basic health was 2.5% lower than the FY2010 budget target, which contravene the anticipated program targets. The PCR noted, however, that the education budget was increased in FY2012 by T$8.0 million and the health budget by T$20.0 million. Further, the RRP for the follow-on SPFMP reported that for FY2014, allocations rose by 4.0% for health and education by 6.0%. Therefore, from a medium-term perspective, basic social service expenditures appeared to be protected and growing. 21. Output 2: Maintenance of fiscal responsibility. The PCR cited an impressive list of policy actions and achievements in meeting tranche conditions and key indicators cited in the DMF of improved Public Expenditure and Financial Accountability Assessment scores, debt, financial ratios, and reporting. Achievements include improved processes and tools for forecasting economic growth and revenue, 13 public procurement, debt risk management, and mitigation policy; 14 improved treasury management financial ratios to guide budget 10 See There was a change in method and, therefore, the DMF ranking in 2009 cannot be used as a baseline anymore. It is worth noting that the ease of doing business ranking did not cover any progress made in reforming public enterprises, although these were the focus of the structural and governance improvement under the program. 11 ADB Report and Recommendation of the President to the Board of Directors: Proposed Policy-Based Grant to the Kingdom of Tonga for Strengthening Public Financial Management Program. Manila, approved for $4.5 million on 24 September There were two back-to-the-office reports from the March mission one by the Regional Director focusing on macroeconomic developments; and one by the team leader, which is the one referred to in this validation. 13 ADB TA helped to develop a budget simulation model; PFTAC and ADB jointly developed quantitative tools for better revenue and economic forecasting. 14 The policy contains processes and responsibilities for the analysis and approval of proposed new guarantees and debt, processes for prioritizing and scheduling debt repayment, and a schedule of timely and standardized reporting on the government s debt portfolio. The major threat to Tonga s fiscal position in the medium term is the commencement of repayments for the Export Import Bank of China (China Eximbank) loans, scheduled to begin in FY2014. Annual debt servicing is expected to double to over T$20 million as a result.

8 6 management and which are reported in the budget; 15 improved reconciliations; and improved budget administrative arrangements and reporting. External debt was reduced from 36.3% of GDP in FY2011 to 31.1% of GDP in FY2012. Other achievements cited include the freeze in government personnel recruitment, with no real increases in the wage bill (increases were in nominal terms of 1.2% in FY2012 and 6.6% in FY2013). 22. Output 3: Structural and governance improvements. The main emphasis of this component was public enterprise reform; achievements include the privatization of Leiola Duty Free; liquidation of the Tongatapu Machinery Pool; and development of rationalization strategies for the Tonga Power Board, Tonga Water Board, and the Waste Authority. A Public Enterprise Amendment Billwas enacted and 10 public enterprises were subsequently restructured. 16 To improve the business environment, a Personal Property Securities Bill and a Companies Amendment Bill were enacted, facilitating the use of chattels as loan collateral and clarifying procedures for incorporation. Several other business regulatory laws were under different stages of drafting (e.g., Foreign Investment Act) and company registration was simplified. 23. Output 4: Support to the vulnerable through targeted actions. Public expenditure reviews were conducted for health and education, which recommended increases in budget allocations, some of which were provided in FY2013. A community development program approach was adopted, which focused on alleviating poverty in communities through devolving decision making and responsibilities. 24. Output 5: Communication and ownership. The PCR reported that the target for this component was met as the government rolled out a communication and engagement strategy to explain the government s response to the global economic crisis. In concrete terms, this activity had helped the government exercise greater openness in disclosing key macroeconomic and public financial information (e.g., monthly trends in inflation, remittances, exports, revenues, expenditures, net lending, and total debts) and data on public enterprises (PCR, paras. 31 and 32). This was a substantial achievement. 25. Tranches. The grant proceeds were released in two tranches. The first tranche of $5.0 million was disbursed in March 2010, more than 3 months after the expected disbursement date. The PCR noted that the delay was due to protracted consultations with the government in clarifying ADB procedures and the government s delay in complying with the prerelease actions. The second tranche should have been released by November 2010, but was delayed until June 2011 as not all tranche conditions had been complied with, 17 and 15 Financial ratios were developed to promote fiscal sustainability over the medium term, and guide preparation of budgets from FY2011 onward (e.g., measures of capital expenditure and maintenance, public debt, government personnel costs, and other [non-personnel] operating costs). The PCR noted that while a few of the ratios were yet to be met, the government has incorporated the ratios into its medium-term budget framework, thereby setting a medium-term fiscal path. 16 The bill (i) enhances the commercial governance practices of public enterprises,(ii) strengthens the monitoring framework for public enterprises, (iii) formalizes the government s decision that politicians should not serve as public enterprise directors, (iv) establishes a principle objective for all public enterprises to operate as successful businesses, (v) clarifies the rules and processes associated with the approval of community service obligations, (vi) establishes clear guidelines on the selection and appointment of directors, and (vii) clarifies accountabilities of directors. The RRP for the follow-on program notes that the act provides a sound legislative framework that supports good governance, encourages a strong commercial focus, and establishes strong accountabilities and transparency. 17 The PCR noted that the government had adopted largely medium-term targets for the financial ratios rather than easily achievable short-term targets, and they could not meet this medium-term target in FY2011, although moving toward it. A waiver was thus sought and obtained.

9 7 again, the PCR noted that this was largely related to the lack of clarity on ADB verification procedures. 26. Based on experience with past reform programs in the Pacific, this validation questions whether so many tranche conditions were needed given the capacity of the government and the limited size of the budget support grant. 27. Covenants. The PCR reported that the government complied with the grant covenants listed in the grant agreement PCR Appendix 2 shows the status of compliance with each of these. Many of these are reporting requirements that this validation cannot independently verify although a review of the ADB review mission back-to-office reports indicate these have been met. 28. This validation rates the program effective based on the achievement of the outcome target of positive growth, together with the substantial achievement of output targets, particularly the progress in improving financial public management and public enterprise reforms. C. Efficiency of Resource Use in Achieving Outcome and Outputs 29. The PCR rated the achievement of outputs efficient. The PCR seemed to base this on the steady progress made in complying with actions for the release of the first and second tranches, with assistance from the ADB TA program and support from ADB 18 and the Joint ADB/World Bank Liaison Office. However, the PCR stated that efficiency could have been improved if ADB engagement had been more consistent. 19 It also referred to adjustments to potentially contentious issues (e.g., timing of tranche release) being negotiated and resolved through consultations between ADB and the government. There is a degree of ambiguity in these PCR statements, which seemed to focus on the actual tranche release rather than on the efficiency of program implementation and delivery, in particular, by mentioning tranche releases having to be negotiated and resolved. This seems to suggest that the government had not been made fully aware of what meeting the tranche conditions really meant, or was not fully committed to these. Perhaps there were different expectations within the government on this, and between the government and ADB. 30. The PCR referred to the differing verification requirements among donor partners regarding implementation of budget support, stating that to some extent this contributed to the delays in the release of the tranches, which led the government to resort to rolling over treasury bills as bridging finance to cover its operational costs. 31. The grant was provided as budget support to help address the impact of the crisis and to avoid expenditure cuts in some social sectors. As such, the size of the grant was not related to any calculation of the cost of reform, 20 and thus rate of return calculations are not appropriate, 18 There seems to be some ambiguity over the PCR claim about the regular support from ADB after grant approval (see section on performance of ADB). 19 Areas that needed improvements included (i) maintaining consistency in ADB representation throughout the program; (ii) putting in place adequate contingency measures to take into account movements of key ADB staff to ensure that there is no delay in processing; and (iii) accounting, during the planning phase, for the time taken to clear issues between ADB headquarters and the resident mission, and among development partners. 20 A qualitative assessment of size seems to have been made, the RRP noting that its size was based on the projected magnitude of the economic, fiscal, and social impact of the global economic crisis on Tonga; scope and fiscal implications of an effective policy response to the crisis; and government's continued commitment to a significant economic and public sector reform agenda.

10 8 nor possible. While substantial progress was made in the achievement of outputs in the short time of the program, which could make an efficiency rating appropriate, efficiency of process is a concern. There seem to have been delays in the early stages of the design of the program (two appraisal missions in August and September 2009), and the release of both the first and second tranches, which may not all relate to the lack of clarity on verification requirements and but perhaps to a lack of understanding by some parties of the importance of the policy actions. The PCR noted (see section on the performance of ADB) that the earlier missions had difficulties convincing the government of the vulnerability of the economy to the crisis. Thus, the process was clearly beset with problems and was not efficient. Overall, this validation rates the program less than efficient. D. Preliminary Assessment of Sustainability 32. The PCR rated the program sustainable quoting as evidence the fulfillment of all policy actions and ongoing implementation of reforms such as medium-term budgeting. It noted, however, that Tonga s macroeconomic position is still at risk, and ADB and other development partners have identified a number of areas in which more support is needed to enhance the sustainability of benefits of the program. These include (i) preparation of a macroeconomic reform road map, (ii) implementation of the public financial management reform road map, and (iii) continuation of ongoing budget support over the medium term. 33. The RRP for the SPFMP (follow-on program) and the ADB Pacific Department reported during this validation that Tonga has further improved its public financial management since the program closing as shown in the following: (i) real reduction in fiscal spending (e.g., freeze in civil service recruitment) since 2010 while maintaining social spending, (ii) "no new borrowing policy" until the International Monetary Fund debt sustainability analyses moved Tonga back from high to medium risk of distress in 2013, (iii) strengthened revenue mobilization (reduced tax exemptions and broadened the tax base), (iv) expanded use of program budgeting and medium-term fiscal framework, (v) accelerated reforms in stateowned enterprises, and (vi) improvements in the business environment. 21 With the follow-on support under SPFMP and coordinated support from other donors using the joint policy matrix, this validation rates the program likely to be sustainable. E. Impact 34. The PCR noted that the intended impact of the program was to contribute toward achieving higher and more inclusive growth. As noted earlier, the DMF showed GDP growth varying at 1.9%, 1.4%, and 1.6% for the 3 years from 2010, thus, meeting the DMF outcome target of positive growth for these years, yet below the longer-term impact target of 3%. However, the RRP for the follow-on reform program states that growth was 3.3% in FY2010 and 2.9% for FY2011, although growth declined to 0.8% in FY2012 and 0.5% in FY2013. These figures need reconciliation, which this validation cannot do. The program s impact on growth is thus moderate. Data was not provided to show whether the relative standard of living of the most vulnerable groups changed or whether the proportion of those living below the poverty line was reduced. The PCR did not give a rating for impact. This validation rates the program s impact moderate. 21 The ADB Pacific Department reported that given the reform commitment of the new government, the World Bank and Australia approved policy-based operations for Tonga in 2011, 2012, and 2013; and are expecting to approve further operations in An Australian review of policy-based operations across the Pacific in 2013 also highlighted the experience in Tonga as being best practice in the Pacific in establishing a coordinated policy dialogue, based on a single, joint reform matrix.

11 9 III. OTHER PERFORMANCE ASSESSMENTS A. Performance of the Borrower and Executing Agency 35. The PCR rated the performance of the grant recipient and the executing agency satisfactory, however, this rating seems to be inconsistent with its text discussion. The PCR pointed out the initial difficulties in getting Cabinet approval for the program despite the economy-wide risks facing Tonga. These difficulties appear to have continued after program approval with the Minister of Finance and Minister of Public Enterprises having difficulties to enact some key legislation. These difficulties were eventually resolved when the extent of the financial problems became clear in early FY2012. The PCR observed a continuing doubt and lack of high-level government ownership of the program earlier in the program implementation. High-level ownership was more confined to the Ministry of Finance, but seemingly not for the government as a whole. In addition, the PCR noted that the momentum to implement reform waned with the delay of the first tranche, and more so after the longer delay with the second tranche release. Genuine appreciation and commitment to structural reforms did not seem to take root and these were seemingly overshadowed by the immediate need to receive the grant and resolve expenditure obligations. There was also a lack of attention during implementation to monitor program outcomes on social services for vulnerable groups, which was a key outcome parameter for the program. The PCR noted that the relationship between ADB and the government was cordial and constructive, but due to the aforementioned issues, this validation gives the rating of less than satisfactory. However, commitment to reform seems to have regained momentum with the follow-on public financial management strengthening program. B. Performance of the Asian Development Bank 36. The PCR rated the performance of ADB partly satisfactory. The PCR noted that ADB drew from its experiences in program lending in the Pacific and elsewhere to support the government s implementation of appropriate and achievable policy actions. The PCR also noted the concerted policy dialogue between ADB and the government, the close coordination between development partners (Australian Aid and the European Union) to support the ADB policy matrix, and the value of TA support provided by ADB in implementing reforms all of which demonstrated a positive commitment. However, as pointed out earlier by the PCR, verification requirements and standards should be clearly understood at the outset, and that the government should be fully aware of the risks of noncompliance with policy actions. It also noted that ADB undertook four missions to explain and design the program, which together with changes in ADB personnel involved in the program, may have led to the PCR rating of partly satisfactory on ADB performance. This validation notes that after the appraisal mission in September 2009, there was only one mission carried out in March 2010 to assess second tranche release conditions, and before the 2012 PCR mission. It is understood that the joint liaison office in Tonga was involved in the program on a regular basis, but given the changing government commitment to the reform program, more should have been done at senior official and minister levels, between the appraisal and the two tranche releases, to discuss commitments. 37. The assessment of program effectiveness (para. 18) suggests that a key outcome indicator of this program the status of health and education services standards on the outer islands was not successfully monitored. The implementation reports and the PCR were unclear whether this was due to the absence of such data in the first place or because the requisite data were not collected during the program implementation. Although a lower level

12 10 output indicator (para. 20) suggested that budget allocations for social services for vulnerable groups had been protected over time and by extension, so did the social services standards the lack of attention to this outcome indicator reflects a substantive shortcoming by ADB and the government in monitoring the program for its potential adverse social impact. This validation, therefore, concurs with the PCR and rates ADB performance less than satisfactory. 22 IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS A. Overall Assessment and Ratings 38. The PCR rated the program successful, based on the assessments above, and summarized in the table below. The PCR concluded that all outputs had been achieved, the program implemented as designed, except for the waiver of the second tranche condition on financial ratios (para. 25). This validation notes the substantial achievements in improving public financial and fiscal management, progress in public enterprise reform, and the likely sustainable rating, and therefore, rates the program successful, although at the lower end of the rating scale. Overall Ratings Criteria PCR IED Review Reason for Disagreement and/or Comments Relevance Relevant Relevant Appendix 3 of the PCR actually rated the program highly relevant although the main text rated it relevant (footnote 6). Effectiveness in achieving outcome and Effective Effective outputs Efficiency in achieving outcome and outputs Efficient Less than efficient Preliminary assessment Sustainable Likely to be of sustainability sustainable Overall Assessment Successful Successful Impact Not rated Moderate Refer to para. 34. While steady progress was made in meeting tranche conditions, there were delays; more importantly, the whole process from design to tranche releases was not efficient, with delays and misunderstandings (paras ). The follow-on RRP for SPFMP provides important support (para. 33). Borrower and executing agency Satisfactory Less than satisfactory The grant proceeds seem to have been the motivating factor, not the reform program. Inability to monitor the program outcomes on social services, which was a key outcome parameter was noted (para. 35). 22 After program completion, the ADB Pacific Department reported that the early and extensive engagement of ADB in the implementation of the operation (initially largely alone among donor partners), was critical in establishing a coordinated, medium-term, policy dialogue with a wider range of partners, which successfully encouraged considerable reform momentum.

13 11 Performance of ADB Partly satisfactory Less than satisfactory More should have been done to ensure that the government understood its obligations and be more committed to the reform program. There was lack of attention in monitoring program impact on vulnerable population and social services (para. 36). Quality of PCR Satisfactory Rating was only marginally satisfactory (para. 42). ADB = Asian Development Bank, IED = Independent Evaluation Department, PCR = program completion report, RRP = report and recommendation of the President, SPFMP = Strengthening Public Financial Management Program. Note: From May 2012, IED views the PCR rating terminology of "partly" or "less" as equivalent to "less than" and uses this terminology for its own rating categories to improve clarity. B. Lessons and Recommendations for Follow-Up 39. This validation concurs with the PCR lesson and recommendation for other externally supported programs to adopt a consolidated approach. Each proposed program is to link itself to a single, consolidated government-led policy matrix. This approach will enhance coordination and collaboration and government ownership. Another critical lesson unique to Tonga is to have the exact method for ADB verification of each tranche and/or policy action to be discussed and agreed upon beforehand with the government. 40. The PCR also listed other lessons that are generally applicable to future programs, such as (i) policy actions need to match capacity; 23 and (ii) provision of a TA that can effectively aid program implementation. Based on the program s lack of success in tracking its impact on the health and education services standards (para. 18), this validation recommends that future programs should exercise closer scrutiny to ensure that outcome indicators are available and accessible. Generally, for reform actions meant to assist or to protect vulnerable groups, a deeper analysis on the program design needs to be undertaken, rather than focus on maintaining or enlarging certain budgetary allocations. After program completion, the PCR recommended that a continuing monitoring be done by ADB through its various missions to Tonga to ensure sustained benefits of the policy actions. This validation supports this recommendation and notes that the follow-on SPFMP is a vehicle to help ensure impact sustainability. V. OTHER CONSIDERATIONS AND FOLLOW-UP A. Monitoring and Evaluation Design, Implementation, and Utilization 41. The tranche progress matrix included in the DMF and the Policy Matrix was used by ADB to monitor and report the progress of the reform program and tranche release conditions. TA consultants and PFTAC monitored progress as part of their support activities. There was substantial collaborative work among ADB, PFTAC, and the International Monetary Fund/World Bank on debt sustainability analysis. The ADB liaison office in Tonga also oversaw program activities. It is unclear whether the MFNP established the performance evaluation system for the program as set out in the RRP, or whether any data was collected 23 Reflecting on the program design, the PCR reported that some ADB development partners considered that a tougher stance should have been taken to promote deeper structural reforms (e.g., reduce civil service redundancies) as first tranche policy actions. Some of the adopted actions (e.g., achieving financial ratios) were considered too easy to implement. This illustrates the challenge one would have in balancing reform ambitions with institutional capacity and political will on the ground.

14 12 by MFNP for monitoring outside the ADB review mission, but a government-administered monitoring and evaluation system could have helped establish ownership and commitment. The follow-on program has developed a joint policy reform matrix (FY ), which is being used by all donors in policy reform discussions and monitoring. B. Comments on Program Completion Report Quality 42. The PCR is well written but its quality is just satisfactory. Section II (evaluation of design and implementation) and section III (evaluation of performance) are weak with some conclusions inconsistent with its text. For example, the assessment on the protection of social expenditures (output 1 implementation) refers only to achievement in budget allocations in FY2012 after the program was completed. The assessment of program effectiveness does not refer to any outcome or output indicators and covers challenges in tranche release, which related more to program efficiency. More analysis should have been provided in section III drawing on earlier parts of the PCR. The PCR was prepared in October 2012, but could usefully have provided a more updated assessment of progress based on the preparation for the 2013 follow-on policy-based grant for strengthening public financial management. It should be noted that some of the figures quoted in the PCR are different from those provided in the RRP for the follow-on grant (e.g., GDP growth). C. Data Sources for Validation 43. The primary sources for this validation were the PCR; RRP for this program and RRP for SPFMP; reports of loan review missions; TA completion report for TA 7475 (Support for Economic and Strategic Management); the Country Partnership Strategy, ; and the Country Operations Business Plan, D. Recommendation for Independent Evaluation Department Follow-Up 44. As the program was immediately followed by a further policy-based grant supporting a reform program in strengthening public financial management, which has been completed (its PCR is still under preparation), and which drew on lessons from this program, a separate program performance evaluation report at this stage may not have much value and may best be done as a joint evaluation activity with the second program. However, there would be more value in examining these programs as part of a special study of reform programs in the Pacific, following the 2009 IED special evaluation study, but perhaps with a focus on the smaller states only where capacity is weakest.