CPSCR Review Independent Evaluation Group

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized 1. CPS Data Country: Kazakhstan CPS Year: FY05 2. Executive Summary For Official Use Only ed by: Peer Reviewed by: Coordinator Florence Charlier Senior Economist, IEGCC Igor Artemiev Consultant, IEGCC i. This review examines the implementation of the FY05-FY11 Kazakhstan Country Partnership Strategy (CPS) of FY05 and the CPS Progress Report (CPSPR) of FY08, and evaluates the CPSCR. The strategy was joint between IBRD and IFC and this review covers the joint program of the two institutions. ii. The WBG sought to support Kazakhstan s efforts organizing its objectives around four pillars that revolved around the central issue of improving its competitiveness. Under Pillar1, the WBG s support aimed at reducing losses in competitiveness through prudent management of the oil revenues and increased public sector efficiency. Under Pillar 2, the WBG strategy focused on buttressing efforts to promote competitiveness by strengthening the government s capacity to identify and reduce barriers to businesses and private investors. Under Pillar 3, the WBG supported building the foundation for future competitiveness by investing in human capital and basic infrastructure. Under Pillar 4, the WBG support aimed at supporting Kazakhstan in ensuring future growth will not harm the environment and past liabilities are mitigated. These objectives were reaffirmed at the time of the CPSPR. iii. IEG rates overall outcome of the WBG s strategy as moderately unsatisfactory. Kazakhstan saved part of its oil revenues in the National Fund of the Republic of Kazakhstan (NFRK) through the CPS period, and introduced a medium term expenditure framework in Progress in complying with the Extractive Industries Transparency Initiative (EITI) has been slower than expected, and the NFRK has mixed ratings within the sovereign wealth funds. Macroeconomic policy was thwarted by failures in macro-prudential regulations of the banking system and bank supervision. Kazakhstan s efforts to decentralize to lower levels of government through legal changes have yet to overcome the culture of strong centralization in decision making. Similarly, Kazakhstan s moves to improve governance, including through better customs and tax administration, have had little effect. Kazakhstan has slowed its accession to the World Trade Organization (WTO) path by entering into trade agreements inconsistent with WTO principles. Similarly, the country has made scant progress in technology transfer, and innovative production has declined as a share of GDP. The share of SMEs in GDP seems to have declined, reflecting the contradictory policy framework that they face regarding competition on the one hand, and import substitution and government support schemes on the other. The share of agriculture in GDP and in total exports also declined, and there is no information on the effectiveness of the changes in agricultural quality and safety standards. Kazakhstan high coverage of education remains challenged by its quality, with students performing below OECD standards. Health indicators show the country making progress, backed in part by a large increase in public expenditure in health. In the provision of basic services, Kazakhstan has improved the electric grid, and provides its people with ample improved water. The quality of electric service, however, remains weak. Kazakhstan s progress in improving basic services has been slowed by the regulatory framework, which discourages investments in electricity generation and water treatment. Kazakhstan improved its road infrastructure, but remains challenged by their maintenance. Kazakhstan s efforts to move towards environmentally sustainable growth, as demonstrated by new Shoghik Hovhannisyan Consultant, IEGCC CPS Period: FY05 FY11 Period: FY05- FY11 Date of this review: April 25, 2012 Xiaolun Sun Senior Evaluation Officer, IEGCC Jaime Jaramillo-Vallejo Lead Economist, IEGCC

2 For Official Use Only 2 legislation and higher expenditures, have yet to yield results despite having been in place for five years. The circumscribed interventions of the WBG in helping Kazakhstan address regional development issues have been mostly positive, but have failed to bring about their scaling up or replication in the country. iv. IEG stresses five findings. First, full country ownership of an advisory agenda does not guarantee its effectiveness. While being a necessary condition, it is by no means a sufficient one as the Kazakhstan example shows. Second, a flexible approach such as the one embodied in the JERP calls for effective and regular monitoring of the two factors that were identified in the FY05 CPS as central to the relevance of the interventions, namely the contribution of these interventions in terms of results (value added) and the progress being made in the policy framework. Third, WBG s flexible approaches with middle income countries need to be coupled with results frameworks that allow valuable and operative monitoring and evaluation of the strategy and its implementation throughout its period. In the absence of monitoring, WBG efforts may turn out to be ineffective from a development perspective. Fourth, the CPSPR is the stage where the objectives of the strategies can be adjusted to what is feasible and to the WBG contributions. The Kazakhstan strategy could have been seen in a better light under a more realistic set of objectives. Fifth, careful constant monitoring of the macroeconomic conditions, including the financial sector, and a persuasive policy dialogue with the government about them would allow the WBG to be a more effective and opportune development partner. A more compelling early engagement by the WBG in dealing with the core issues may have lessened the effects and duration of the domestic banking crisis of Kazakhstan. 3. WBG Strategy Summary Overview of CPS Relevance: Country Context: 1. Over , Kazakhstan grew rapidly at around 10% driven mostly by its hydrocarbon and metal extraction industries, and GDP per capita reached US$7,800 (PPP). During this period, the country saved some of the revenue stemming from its extractive industries in the National Fund of the Republic of Kazakhstan (NFRK), which was created in 2000 and is administered by the Central Bank. Kazakhstan s fast pace of growth continued through 2007, but slowed considerably with the global financial crisis. Kazakhstan s banking sector was particularly hardly hit by weak funding practices before the crisis, and remains weak. At the heart of the weakness of the domestic banks lies their overleveraging using foreign financial sources (over 140 percent of domestic deposits at its peak in 2007), which dropped considerably with the global financial crisis. Accordingly, the ratio of nonperforming loans of the financial sector rose sharply to over 25 percent, as domestic banks tried to adjust their balance sheets to the lower levels of financing available from abroad. Nevertheless, GDP growth rebounded in 2010, aided by favorable commodity prices, an accommodative monetary policy and the use of US$17 billion of NFRK resources to support banks and a fiscal expansion. Meanwhile, poverty declined from 46.7 percent in 2001 to 6.5 percent in 2010, but is twice as high in rural areas as compared to urban areas. On the operational side, Kazakhstan had created, together with IBRD, a Joint Economic Research Program (JERP) in 2006, which was cofinanced and served as to frame IBRD s contributions to the country through analytic and advisory activities (AAA). Under this framework, the yearly AAA program was defined based on Kazakhstan s demand for services. 2. The Kazakhstan Government s vision for the country s development was initially laid out in 1997 Prosperity, Security and Ever Growing Welfare of All the Kazakhstanis, which was further updated in a Presidential Address in The updated Strategy s main priorities include (i) the successful integration of Kazakhstan in the world economy through existing and new competitive advantages; (ii) the management of growth and sustainability through diversification, infrastructure development and the development of hi-tech industry; (iii) providing education and professional training at international standards; (iv) the adoption of a modern social policy in housing, healthcare

3 For Official Use Only 3 and pensions; (v) strengthening the foundation of the political system; and (vi) facing the new regional and geopolitical responsibilities of Kazakhstan. Objectives of the WBG Strategy: 3. The WBG sought to support Kazakhstan s efforts organizing its objectives around four pillars that revolved around the central issue of improving its competitiveness. Under Pillar1, the WBG s support aimed at reducing losses in competitiveness through prudent management of the oil revenues and increased public sector efficiency. Under Pillar 2, the WBG strategy focused on buttressing efforts to promote competitiveness by strengthening the government s capacity to identify and reduce barriers to businesses and private investors. Under Pillar 3, the WBG supported building the foundation for future competitiveness by investing in human capital and basic infrastructure. Under Pillar 4, the WBG support aimed at supporting Kazakhstan in ensuring future growth will not harm the environment and past liabilities are mitigated. These objectives were reaffirmed at the time of the CPSPR. Relevance of the WBG Strategy: 4. Congruence with Country Context and Program. The WBG strategy was broadly aligned with Kazakhstan s country context and own vision. Nevertheless, the WBG strategy was selective and mostly focused on the competitiveness aspects of macroeconomic management, private sector development, human capital, infrastructure and environmental sustainability. The CPSPR did not elaborate on how the strategy would be adjusted to support Kazakhstan s efforts to face the domestic banking system crisis, which had by then emerged. 5. Relevance of Design: The design of the strategy derived relevance from the fact that the main set of interventions of the WBG, analytical and advisory activities, was driven by the yearly agreements with Kazakhstan on the JERP. To enhance the relevance of these interventions, the FY05 CPS stated that relevance would be assessed against three criteria : (i) the demand for Bank s services, (ii) the value added of the development process and (iii) Kazakhstan s overall policy environment. Moreover, the CPS identified a set of indicators and related regional and world benchmarks that would guide the assessment, particularly as regards to progress in economic management, poverty abatement, and improvements in competitiveness and governance. The conditions and indicators for relevance were downplayed in the FY08 CPSPR and were not reported in the CPSCR. As anticipated in the FY05 CPS, the relevance of these interventions weakens when judged against the effective value added to the country and the progress (or lack of) in the policy framework, as appears to be the case in Kazakhstan. While country ownership is a necessary condition for relevance, it is certainly not a sufficient condition. 6. Relevance of Response to Market: Until 2007 IFC fostered private-sector-led growth focusing on non-extractive sectors and on the frontier regions of the country. These interventions responded to the country s market conditions, and covered the financial sector (mostly larger banks) as well as targeted investments in manufacturing and services, construction and construction materials, hotels, retail trade and leasing. Providing financing to overleveraged financial institutions at the time of the bubble was not sound from a macroeconomic standpoint and went against the objectives of the WBG strategy. After 2007, when the banking system went into a crisis because of the withdrawal of foreign financing to domestic banks, IFC adjusted its approach. IFC raised total country commitments more than five times focusing mostly on supporting ailing banks, providing them with equity, quasi-equity, senior debt and trade finance. IFC focused mostly on mid-cap and smaller niche banks, which had larger share of SME loans in their portfolio. In contrast with the pre-crisis interventions, these were relevant in that they helped palliate the rapid decline in foreign financing faced by domestic banks, while aiding financing flows to SMEs. 7. Risk Identification and Mitigation: The strategy identified as main risks the macroeconomic factors as well as the government s lack of commitment to policy reform. The CPS also recognized

4 For Official Use Only 4 the risk associated with the change in business model, as defined by the JERP. The WBG did not identify as a major risk, however, the foreign leveraging of Kazakhstan s banking system, which reached over 140 percent of domestic deposits in Moreover, the strategy was not adjusted at the time of the CPSPR to take this crisis into account. Overview of CAS/CPS Implementation: Lending and Investments: 8. IBRD approved financing for US$ 3.7 billion through 15 operations (investment and policy based lending) during the CPS. Under Pillar 1, which focused on managing the oil windfall and increasing public sector efficiency, IBRD delivered three projects, including Customs Development (CDP, FY08), Tax Administration Reform (TARP, FY10), Strengthening the National Statistical System (NSSP, FY11), as well as the Kazakhstan Development Policy Loan (KZ-DPL, FY10). Under Pillar 2, which focused on the business environment, IBRD delivered three projects, including Agricultural Post-Privatization Assistance APL 2 (APL2, FY05), Agricultural Competitiveness (ACP, FY05), and Technology Commercialization (TCP, FY08). Under Pillar 3, which aimed at supporting human capital and basic infrastructure, IBRD delivered six projects, including North-South Electricity Transmission (NSET, FY06), Health Sector Technology Transfer and Institutional Reform (HSTTP, FY08), South-West Roads: Western Europe-Western China International Transit Corridor (CARECP, FY09), Moinak Electricity Transmission (METP, FY10), Technical & Vocational Education Modernization (TVEMP, FY11), and Alma Transmission (ATP, FY11). Finally, under Pillar 4 on environment, IBRD delivered two projects, including Ust-Kamenogorsk Environmental Remediation (UKERP, FY06) and Forest Protection and Reforestation (FPRP, FY06). In addition, IBRD continued implementing seven projects that had been approved under the previous CPS. 9. IBRD s share of commitments at risk fluctuated during the strategy period, starting from nil, reaching 16.5 percent in 2007 and falling to 2.5 percent in About one fifth of the projects by number were at risk in While the share of commitments at risk is low by ECA standards, the share of operations by number is as high as the rest of the World Bank. Of seven projects exited during the CPS period and evaluated by IEG, six received a rating of satisfactory and one of moderately satisfactory. 10. IFC invested in 18 new projects, four of which closed during the CPS period. In addition, it saw 18 other projects from before exit during the CPS period. Out of the 18 new projects, 10 were green field projects and in the remaining ones were in existing enterprises. Total IFC commitments in Kazakhstan reached approximately US$ 750 million in 2011, three times as much as all pre-2005 commitments. For the most part due to the domestic banking crisis, 85 percent of these financings have been in the financial sector. The remaining 15 percent were in the manufacturing and services sectors. During FY05-FY11, IFC s net commitment activity in equity participations was 16 percent of the total with loans accounting for the remaining part. In contrast with the past, no new IFC investment commitment went to the oil, gas, mining, metal and steel industries. 11. On the IFC side, all four investments evaluated by IEG received a positive development outcomes rating. However, IFC did not seem to give enough attention to monitoring and evaluation in Kazakhstan. Out of 41 investment projects active during the review period, only 11 had Development Outcome Tracking System (DOTS) forms completed with seven evaluated internally as successful in their overall development outcome. In addition, one project was rated as mostly successful and another one as mostly unsuccessful on development outcomes. A number of projects were not subjected to DOTS exercise at all.

5 For Official Use Only 5 Analytic and Advisory Activities and Services 12. IBRD provided Kazakhstan with 129 pieces of AAA through the Joint Economic Research Program (JERP), covering issues in all four pillars of the strategy. A portion of the cost of the JERP is covered by Kazakhstan; that share stood at 30 percent at the beginning of the CPS period and increased gradually to an estimated 85 percent this fiscal year. 13. IFC had five advisory services (US$4 million) during the CPS period, two of which predated the CPS. Advisory activities covered corporate governance improvement closer to international standards in their corporate practices (including the national legal and regulatory framework), as well developing access to finance to MSMEs located near a restructured iron and still mill. Other smaller advisory services focused on facilitating SME certification and quality improvement, equipment leasing, and evaluation of a regional co-generation company s investment program. The advisory service on primary mortgage institutional support project was rated by IEG as unsuccessful in development effectiveness due to a weak project design, insufficient supervision, and understaffing. Partnerships and Development Partner Coordination 14. IBRD worked jointly with the IMF in preparing an FSAP, and with the OECD in a Higher Education Report. The WBG also worked in collaboration with the European Commission (EC), USAID, the Asian Development Bank (ADB), the European Bank for Reconstruction and Development (EBRD), and several United Nations agencies. Safeguards and Fiduciary Issues 15. There was one safeguard issue related to the South-West Roads project brought to the Inspection Panel s attention in June The issue related to loss of assets and livelihood from land acquisition for the project and potential adverse environmental impacts with regard to works on different sections of the road. In October 2011, the Panel concluded that IBRD was taking actions to resolve these concerns and did not recommend further action on its part. IEG is not aware of any INT investigations on Kazakhstan over the CPS period, but INT has opened an information item on the Kazak East West Highway. Overview of Achievement by Objective: Pillar 1: Reducing losses in competitiveness through prudent management of the oil revenues and increased public sector efficiency. 16. Under this pillar, the WBG strategy aimed at helping Kazakhstan in managing the oil revenues (including their transparency) and the government s medium term expenditure framework (MTEF), developing the local and central government capacity to implement public spending, and address corruption. 17. Managing the oil revenues and the government s MTEF. During the CPS period, Kazakhstan saved some of the revenue stemming from its extractive industries, including oil, in the National Fund of the Republic of Kazakhstan (NFRK), which was created in 2000 and is administered by the Central Bank. Accumulated savings, which amounted to US$38 billion in 2011, were drawn down to face the impact of the global financial crisis, as noted above. Oil revenues are substantial, accounting for about 50 percent of total government revenue. Kazakhstan has yet to participate in the International Forum of Sovereign Wealth Funds (IFSWF) and adopt the so-called Santiago Principles that are followed by 21 other sovereign wealth funds (SWFs) since October The NFRK scores an 8 (out of a possible10) in the Sovereign Wealth Fund Institute s Transparency rating, higher than

6 For Official Use Only 6 some countries in the region and at the benchmark rating of 8 that reflects adequate transparency arrangements. Nevertheless, the Peterson Institute for International Economics, which also rates SWFs, gave a rating of 65 over 100 to the NFRK in August 2011, lower than Chile, Azerbaijan, Trinidad and Tobago and Timor Leste. As for the Extractive Industries Transparency Initiative (EITI), Kazakhstan endorsed the principles in 2005 and was admitted as an EITI Candidate country on 27 September 2009, after having received a special price for its efforts to raise awareness of the EITI across the country. On 14 December 2010, the EITI Board designated Kazakhstan, as an EITI Candidate country that is Close to Compliant. The initial six-month grace period has been extended and now goes until August The Board granted the extension with the expectation that Kazakhstan will increase the coverage of material payments and revenues in its EITI reports. 18. On fiscal policy and the government s MTEF, the government approved a three year budget based on strategic plan at the national and regional level in However, the IMF has noted that the fiscal framework remains weak because of the sizeable quasi-fiscal and off-budget items. Indeed, the budget does not include all public and quasi-public enterprises, which remain involved in many off budget deficit transactions and have built-up external debt outside of the government s tally. Nevertheless, the government avoided a wholesale bailout approach in the wake of financial crisis, increasing instead public wages, pensions and public works. The effectiveness of this expenditure reallocation has yet to be assessed. 19. IBRD provided support through the Kazakhstan Development Policy Loan (KZ-DPL, FY10), the Strengthening the National Statistical System Project (NSSP, FY11), and several pieces of AAA. The KZ-DPL focused on the rules to access the NFRK and the reallocation of expenditures towards social spending. The improvements in the statistical system were expected to help, among other things, in public expenditure management. Chief among the AAA was the work in helping Kazakhstan with the extension of the grace period of the EITI, and the dialogue on Kazakhstan macroeconomic response to the global financial crisis and its own banking system crisis. A 2005 report made recommendations to strengthen the government s economic policy and contain overheating and preserve competitiveness during the boom years. IBRD also provided technical assistance on managing public investments and through a policy note on the financial sector, and an FSAP prepared jointly with the IMF. The FSAP and the policy note played a role in alerting the government about some of the causes of the crisis in domestic banks, but robust foreign leveraging limits were addressed clearly only in July 2011, in the wake of the domestic crisis and despite having been identified clearly in an IBRD policy note of Developing the local and central government capacity to implement public spending. Kazakhstan has a three level government structure (central, regional and local). Budget relations between these levels as defined in the Budget Code were revised in The 2008 amendment decentralized spending further and introduced elements of strategic and results-oriented budgeting at all levels of government. However implementation remains a challenge. Although some delegation and improved oversight is taking place within the central government, the strong centralization in decision making limits the degree to which subnational units can be held accountable for results. The link between the performance of subnational units and budget transfers is, thus, weak. 21. IBRD provided analytical support through the 2006 report on intergovernmental fiscal relations and the 2010 public financial management review of the system of budgetary relations and management of local budget expenditure for education and social protection. The CPSCR doesn t report on the implementation of the recommendations stemming from these reports. 22. Addressing corruption. As reported by the World Economic Forum (WEF), Kazakhstan made little progress in increasing the transparency of its public sector, and corruption remains the most problematic factor for doing business, according to the latest Global Competitiveness Report (GCR). The Wastefulness of Government Spending index improved marginally from 3.4 over 7.0 in to 3.5 in However, the index on the Diversion of Public Funds to Companies and Individuals or Groups due to Corruption worsened from 3.9 over 7.0 to 2.8, while that on the Favoritism in Decisions of Government Officials when Deciding upon Policies and Contracts

7 For Official Use Only 7 deteriorated somewhat from 3.1 over 7 to 2.6, and that on Institutions deteriorated slightly from 3.6 to 3.5 over the same period. The lack of progress is also reflected in the corruption indicators reported by different sources. Two of these sources show insignificant change. For the period , the Corruption Perception index of the International Property Right Index changed from 2.6 over 10.0 to 2.7, while the indicator on Control of Corruption of the World Bank Governance Indicators in terms of percentiles deteriorated slightly from 15.6 to In 2008 Kazakhstan adopted changes to its procurement law, strengthening the submission processes and the free standing complaint mechanisms, while introducing e-procurement. However, there is limited evidence on the effectiveness of this new law with e-procurement just beginning (0.07 percent of current expenditures). Efforts to improve customs and the customs union lowered the percentage of physical inspections from 70.0 percent in 2007 to 18.6 percent in 2009, but Kazakhstan ranks poorly on the GCR s indicator related to the burden of customs procedures (102 out of 142 in the 2011/12), particularly the large number of documents required for exporting or importing as well as the long time needed to export or import. As for tax efficiency, the IMF reports in its 2011 Article IV Consultation that non oil tax revenues remain below the pre-crisis level. 23. IBRD supported Kazakhstan through two projects whose central objective was to reduce corruption in customs and tax administration-- Customs Development Project (CDP, FY08) and Tax Administration Reform (TARP, FY10). In addition, the Kazakhstan Development Policy Loan (KZ-DPL, FY10) included actions to strengthen public financial management, based on a Public Finance Management Assessment (PEFA-based) report undertaken by IBRD in IEG rates the achievement of the WBG strategy under pillar 1 as moderately unsatisfactory. Kazakhstan saved part of its oil revenues in the National Fund of the Republic of Kazakhstan (NFRK) through the CPS period, and introduced a medium term expenditure framework in Progress in complying with the Extractive Industries Transparency Initiative (EITI) has been slower than expected, and the NFRK, albeit being ranked as transparent by some, does not rank well within the sovereign wealth funds in other ratings. Macroeconomic policy was thwarted by failures in macro-prudential regulations of the banking system and bank supervision. Kazakhstan s efforts to decentralize to lower levels of government through legal changes have yet to overcome the culture of strong centralization in decision making. Similarly, Kazakhstan s moves to improve governance, including through better customs and tax administration, have had little effect. Pillar 2: Promoting competitiveness by strengthening the government s capacity to identify and reduce barriers to businesses and private investors. 25. Under this pillar, the WBG strategy aimed at supporting Kazakhstan s efforts to (i) make progress in WTO accession and bilateral trade agreements; (ii) fostering technology transfer and commercialization of research and development; (iii) developing SMEs, including their linkages to the rest of the economy; and (iv) developing agricultural support policies, including quality and safety standards. 26. WTO accession and bilateral trade agreements. Kazakhstan has yet to complete its accession to the World Trade Organization (WTO). Accession, which was foreseen for 2009, was delayed because the country entered into a customs union with Belarus and raised its tariffs by an average of five percentage points. The customs union, which was formalized in 2010, led to an increase in physical controls at borders, except for those with Russia that were eliminated in July 2011 when Russia joined the customs union. Kazakhstan also embarked on a pilot program on joint customs control with China and Kyrgyz Republic in the context of the trade facilitation program under the Central Asian Regional Economic Cooperation Program (CAREC). 27. IBRD supported work in this area through two pieces of AAA. 28. Technology transfer and commercialization of research and development. Despite being a priority for the government, there has been scant progress in transferring technology and commercializing research and development. The share of innovative production in GDP decreased

8 For Official Use Only 8 from 1.27 percent in 2004 to 0.66 percent in 2010, according to the Statistical Agency of Kazakhstan. In addition, the country s expenses in research and development declined from 0.25 percent in 2004 to 0.16 percent in Moreover, the WEF index for innovation rated Kazakhstan with a 2.6 over 7.0 in 2011/12. Actual diversification towards non extractive tradables has been relatively limited. The economy remains highly resource dependent with agriculture accounting for 5 percent and manufacturing for 11 percent of GDP. Nevertheless, the number of organizations conducting research and development increased from 295 in 2004 to 412 in IBRD provided support through the Technology Commercialization Project (TCP, FY08), as well as three pieces of AAA. The project has been delayed considerably in its implementation, starting operations three years after Board approval. 30. SMEs development, including their linkages to the rest of the economy. The state continues to dominate the non-oil economy and large state companies have squeezed the SME sector. According to the Statistical Agency of Kazakhstan, the share of SMEs in GDP declined from 40.5 percent in 2005 to 31.2 percent in 2008 (latest data; the methodology has been revised and the new data is not consistent), while the share of SMEs in total employment increased marginally from 22.8 percent in 2005 to 32.4 percent in 2010 (latest data). IBRD reports that a steady decline in total factor productivity has subtracted 1.5 percentage points per annum from growth, with the decline being most noticeable in mining, construction and the processing industries. Despite efforts to improve competition legislation and ease public private partnerships (PPPs) in 2010, Kazakhstan continues to pursue policies on import substitution and local content requirements, as well as government support schemes which hinder productivity. 31. IBRD provided support through one piece of AAA. IFC had an enterprise restructuring project that focused on financing employees of large enterprises willing to receive student loans for retraining and start their own business. The outcomes of this intervention are not known at this stage. In addition, IFC provided assistance on leasing, corporate governance and SME development, including to those serving the oil and gas sector. There is no independent evaluation of development effectiveness of SME-support programs sponsored by IFC (e.g., number of SMEs created, amount of financing provided to MSMEs, number of loans extended to MSMEs, sustainability of MSMEs supported by these programs, etc.) A flagship advisory and investment project aimed at creating a fund to facilitate access to credit for SME-input suppliers resulted in a modest number of loans (only six active accounts out of 16 committed), poor performance of the fund after the largest borrower defaulted on its obligations in the midst of the crisis, and an impairment in the fund s capacity to repay at maturity the full loan to IFC. 32. Agricultural support policies, including quality and safety standards. The agriculture share in GDP declined from 6.8 percent in 2005 to 4.8 percent in 2010 and the agricultural raw materials exports as a percentage of exported merchandises decreased from 1 percent in 2004 to 0.2 percent in The agricultural sector s access to and usage of credit remains very limited. Kazakhstan introduced some regulations on quality and safety standards, but the extent of their implementation and its impact are not known at this stage. 33. IBRD supported Kazakhstan s efforts through two projects, the Agricultural Post-Privatization Assistance APL 2 (APL2, FY05) and the Agricultural Competitiveness Project (ACP, FY05). The ACP has experienced delays in its implementation. Additional support came from components in a pilot GEF project, the Drylands Management Project (GEF FY03, closed FY10). IBRD also delivered one piece of AAA. 34. IEG rates the achievements of outcomes under Pillar 2 as unsatisfactory. Kazakhstan has slowed its accession to the World Trade Organization (WTO) path by entering into trade agreements inconsistent with WTO principles. Similarly, the country has made scant progress in technology transfer, and innovative production has declined as a share of GDP. The share of SMEs in GDP seems to have declined, reflecting the contradictory policy framework that they face regarding competition on the one hand, and import substitution and government support schemes on the other.

9 For Official Use Only 9 The share of agriculture in GDP and in total exports also declined, and there is no information on the effectiveness of changes in agricultural quality and safety standards. Pillar 3: Building the foundation for future competitiveness by investing in human capital and basic infrastructure. 35. Under this pillar, the WBG strategy sought to support the government in improving (i) health; (ii) education; (iii) basic services; and (iv) transport. 36. Improving education. The 2011 UNESCO report shows Kazakhstan achieving universal primary education (99.1 percent), adult literacy (99.7 percent); gender parity (99.3 percent); and survival rate to grade 5 (99.5 percent). These figures show a slight improvement over those of 2007, which were also high. By contrast, the PISA 2009 results show that 59% of students score below the basic competency level in mathematics, 58% in reading and 55% in sciences, which are significantly below the OECD average. At the same time, the WEF indicator Quality of the Education System declined from 3.8 over 7.0 in to 3.0 in In higher education sector, the government has established a national accreditation center and joined the Bologna process in IBRD support was through the Technical & Vocational Education Modernization Project (TVEMP, FY11) and six pieces of AAA. 38. Improving health. Reflecting economic development and increases in public expenditure on health, Kazakhstan has seen improvements in key health indicators such as life expectancy (68.4 years in 2009 from 66.0 in 2005), under five mortality (39.4 per 1000 live births in 2004 to 33.2 in 2010) and tuberculosis incidence (223 in 2004 to 151 for in 2010). Maternal mortality remains high (36.9 per births). Non communicable diseases such as cardiovascular and cancers, other tobacco and alcohol related diseases remain the major cause of adult mortality. 39. IBRD provided support through the Health Sector Technology Transfer and Institutional Reform Project (HSTTP, FY08) and five pieces of AAA. As of FY11, IFC started to venture into regional health insurance on a small scale basis. 40. Improving basic services. While Kazakhstan has made strong progress in electricity transmission, but regressed in water supply and the quality of electricity service. In the electric grid the country refurbished transmission substations and modernized the dispatch control system, reducing the average deviation of system frequency from 500 mhz in 2005 to 20 mhz in Kazakhstan also increased the volume of electricity transferred from the northern to the southern parts of the country from 650 MW in 2005 to 1,350 MW in Transmission losses on the North- South interconnector declined from 8.8 percent in 2005 to 5.5 percent in However, the WEF s index on quality of electricity supply declined from 4.3 over 7.0 in to 3.9 in Moreover, the 2009 enterprise survey reported a higher number of power outages in a typical month (5) and higher value lost due to power outages (3.5% of sales) than in the ECA region or in upper middle income countries. In addition, the share of Kazakhstan s population with access to improved water declined from 96.0 percent in 2005 to 95.0 percent in Kazakhstan s progress in improving basic services has been slowed by the regulatory framework. Strong competition among the electricity generation companies in the wholesale market has resulted in relatively low prices and lack of sufficient resources to finance investments in new capacity. In addition, the government has set the retail tariff at a level that does not ensure the full recovery of expenses, leading companies to rely on obsolete technology and plants. In the water sector, the National Agency for Natural Monopolies enforces low tariffs which are among the lowest in the world (roughly 3-4 times below affordability level). 42. IBRD provided support to the electricity sector through three projects, the North-South Electricity Transmission Project (NSET, FY06), the Moinak Electricity Transmission (METP, FY10), and the Alma Transmission (ATP, FY11). In the water sector IBRD provided support through the Nura

10 For Official Use Only 10 River Clean-up Project (NRCP, FY03), which closed in FY11. In addition, IBRD delivered ten pieces of AAA. In IFC had a due diligence advisory project to assess the investment needs of a generation/heating company servicing regions in northeast Kazakhstan. 43. Improving transport. According to the WEF, the indicator on road quality of the Global Competitiveness Index improved slightly from 2.3 over 7.0 in to 2.5 in The total road network increased from 90,018km in 2004 to 93,612km in 2008, while the percentage of paved roads declined from 93.4 in 2004 to 88.5 in The WEF also reports that the indicator on the Quality of Railroad Infrastructure improved from 2.6 in to 3.9 in While major rehabilitation road works are carried out by private contractors, routine maintenance is still carried out by a state owned enterprise. Studies highlight that Kazakhstan s expenditures in roads have an imbalance between expenditures on maintenance and new construction (the latter being too high), and face corruption and high cost of civil works. 44. IBRD supported Kazakhstan s efforts through the South-West Roads: Western Europe- Western China International Transit Corridor Project (CARECP, FY09), as well as through the completion of the Road Transport Restructuring Project (FY99), which closed in IBRD also produced three pieces of AAA. 45. IEG rates the achievement of outcomes of the WBG strategy under pillar 3 as moderately satisfactory. Kazakhstan high coverage of education remains challenged by its quality, with students performing below OECD standards. Health indicators show the country making progress, backed in part by a large increase in public expenditure in health. In the provision of basic services, Kazakhstan has improved the electric grid, and provides its people with ample improved water. The quality of electric service, however, remains weak. Kazakhstan s progress in improving basic services has been slowed by the regulatory framework, which discourages investments in electricity generation and water treatment. Kazakhstan improved its road infrastructure, but remains challenged by their maintenance. Pillar 4: Ensuring future growth will not harm the environment and past liabilities are mitigated. 46. Under this pillar, the WBG strategy aimed at supporting Kazakhstan in (i) dealing with the consequences of growth on the environment; and (ii) addressing regional development issues. 47. Dealing with the consequences of growth on the environment. According to the Statistical Agency of Kazakhstan, the government s expenses on environment protection tripled from US$ 209 million in 2004 to US$ 682 million in 2010, while the air emissions from stationary sources declined from 3.0 million tons in 2004 to 2.3 million in Policy-wise, the government adopted a new environmental code in At the same time, however, the CO2 emissions increased from 172,158 kt in 2004 to 236,954 kt in Also, combustible renewables and waste increased from 44 metric tons of oil equivalent in 2004 to 157 in The higher expenses and regulatory changes have yet to yield results. 48. IBRD provided support in this area through the Ust-Kamenogorsk Environmental Remediation (UKERP, FY06) which included a component on water quality monitoring. In addition there were two pre-cps projects that were active. One was the Drylands Management Project (GEF, FY03, closed in FY10) and the other one the Uzen Oil Field Rehabilitation Project (FY97, closed in FY07). IBRD also provided one AAA. 49. Addressing regional development issues. IBRD provided support on a limited scale on reducing the salinity of the Aral Sea, dry lands management, forest protection, reforestation, recovery of some lands polluted by heavy metals, rehabilitating an oil field, and cleaning a river. In the absence of a clear definition of this objective, IEG interprets it by the general thrust of the WBG interventions. All the six projects have a local focus, and four of them received moderately satisfactory ratings or better in internal evaluations. Two of them, however, were rated moderately unsatisfactory in internal

11 For Official Use Only 11 evaluations. These projects are the Syr Darya Control and Northern Aral Sea Phase I project (FY01, closed FY11), the Drylands Management Project (GEF FY03, closed FY10), the Forest Protection and Reforestation Project (FY06, still active), the Ust Kamenogorsk Environmental Remediation Project (FY06, still active), the Uzen Oil Field Rehabilitation Project (FY97, closed FY07), and the Nura River Clean Up Project (FY03, closed FY11). 50. IEG rates the strategy s outcome under Pillar 4 as moderately unsatisfactory. Kazakhstan s efforts to move towards environmentally sustainable growth, as demonstrated by new legislation and higher expenditures, have yet to yield results despite having been in place for five years. The circumscribed interventions of the WBG in helping Kazakhstan address regional development issues have been mostly positive, but have been slow in bringing about their scaling up or replication in the country. Achievement of CPS Objectives Objectives Pillar 1: Reducing losses in competitiveness through prudent management of the oil windfall and increased public sector efficiency Pillar 2: Promoting competitiveness by strengthening the government s capacity to identify and reduce barriers to businesses and private investors Pillar 3: Building the foundation for future competitiveness by investing in human capital and basic infrastructure Pillar 4: Ensuring future growth will not harm the environment and past liabilities are mitigated IEG Rating Moderately Unsatisfactory Unsatisfactory Moderately Satisfactory Moderately Unsatisfactory 4. Overall IEG Assessment Overall Outcome: IBRD/IDA Performance: IFC Performance: Moderately Unsatisfactory Moderately Unsatisfactory Moderately Satisfactory Overall outcome: 51. IEG rates overall outcome of the WBG s strategy as moderately unsatisfactory. Kazakhstan saved part of its oil revenues in the National Fund of the Republic of Kazakhstan (NFRK) through the CPS period, and introduced a medium term expenditure framework in Progress in complying with the Extractive Industries Transparency Initiative (EITI) has been slower than expected, and the NFRK has mixed ratings within the sovereign wealth funds. Macroeconomic policy was thwarted by failures in macro-prudential regulations of the banking system and bank supervision. Kazakhstan s efforts to decentralize to lower levels of government through legal changes have yet to overcome the culture of strong centralization in decision making. Similarly, Kazakhstan s moves to improve governance, including through better customs and tax administration, have had little effect. Kazakhstan has slowed its accession to the World Trade Organization (WTO) path by entering into trade agreements inconsistent with WTO principles. Similarly, the country has made scant progress in technology transfer, and innovative production has declined as a share of GDP. The share of SMEs in GDP seems to have declined, reflecting the contradictory policy framework that they face regarding competition on the one hand, and import substitution and government support schemes on

12 For Official Use Only 12 the other. The share of agriculture in GDP and in total exports also declined, and there is no information on the effectiveness of the changes in agricultural quality and safety standards. Kazakhstan high coverage of education remains challenged by its quality, with students performing below OECD standards. Health indicators show the country making progress, backed in part by a large increase in public expenditure in health. In the provision of basic services, Kazakhstan has improved the electric grid, and provides its people with ample improved water. The quality of electric service, however, remains weak. Kazakhstan s progress in improving basic services has been slowed by the regulatory framework, which discourages investments in electricity generation and water treatment. Kazakhstan improved its road infrastructure, but remains challenged by their maintenance. Kazakhstan s efforts to move towards environmentally sustainable growth, as demonstrated by new legislation and higher expenditures, have yet to yield results despite having been in place for five years. The circumscribed interventions of the WBG in helping Kazakhstan address regional development issues have been mostly positive, but have failed to bring about their scaling up or replication in the country. IBRD Performance 52. IEG rates IBRD s performance as moderately unsatisfactory. The JERP provided a flexible framework for IBRD s AAA activities in Kazakhstan, and ensured the country s ownership of that program. IBRD also reacted in a timely manner to provide financial support to Kazakhstan at the time of the global financial crisis. However, the relevance of IBRD s approach to Kazakhstan waned due to the weak contributions of the interventions in terms of results (value added) and the policy framework. These two points, which were foreseen in the original CPS of FY05 and were intended to be monitored throughout the strategy period, were not followed in the CPSPR and in the CPSCR. The results framework of the strategy was ill defined and did not allow IBRD to have a working monitoring and evaluation of the strategy and its implementation. The opportunity to adjust the objectives at the time of the CPSPR was missed. Nevertheless, the supervision of the portfolio was in line with that of the rest of IBRD. Coordination with other development partners was included the IMF, the EC, USAID, ADB, EBRD, some UN agencies, and a report with OECD. Attention to safeguard and fiduciary issues was adequate. IFC Performance: 53. IEG rates IFC performance as moderately satisfactory. IFC reacted quickly to changing country conditions and provided support to a number of domestic banks when their foreign sources dried up. IFC also did a good job in implementing its interventions, and made an effort to contribute to the SME sector using different approaches. However, the investments and the financing of financial institutions before 2008, at a time when these institutions were overleveraged abroad was not sound from a macroeconomic standpoint and was counter to the objective of the WBG strategy of maintaining a stable macroeconomic environment. 5. Assessment of CAS/CPS Completion Report 54. The CPSCR provides a description of the WBG strategy implementation, including the finding and lessons learned. An important shortcoming is that the CPSCR deviated from the objectives set out under the CPS and the CPSCR, and chose instead to report on selected Government s core development goals. Reporting on CPS objectives and result would have been more effective if the 2008 CPSPR had redefined more specific objectives under each pillar to limit their scope, identify measurable outcomes and elaborate an M&E framework. While the need for a flexible, demanddriven strategy is understandable, that fact does not deter from the need to monitor and evaluate the strategy and its implementation. The CPSCR also failed to report on the conditions and indicators for relevance set out in the original strategy, which were central to evaluate the effectiveness of the business model used by the WBG in Kazakhstan.

13 For Official Use Only Findings and Lessons 55. IEG stresses five findings. First, full country ownership of an advisory agenda does not guarantee its effectiveness. While being a necessary condition, it is by no means a sufficient one as the Kazakhstan example shows. Second, a flexible approach such as the one embodied in the JERP calls for effective and regular monitoring of the two factors that were identified in the FY05 CPS as central to the relevance of the interventions, namely the contribution of these interventions in terms of results (value added) and the progress being made in the policy framework. Third, WBG s flexible approaches with middle income countries need to be coupled with results frameworks that allow valuable and operative monitoring and evaluation of the strategy and its implementation throughout its period. In the absence of monitoring, WBG efforts may turn out to be ineffective from a development perspective. Fourth, the CPSPR is the stage where the objectives of the strategies can be adjusted to what is feasible and to the WBG contributions. The Kazakhstan strategy could have been seen in a better light under a more realistic set of objectives. Fifth, careful constant monitoring of the macroeconomic conditions, including the financial sector, and a persuasive policy dialogue with the government about them would allow the WBG to be a more effective and opportune development partner. A more compelling early engagement by the WBG in dealing with the core issues may have lessened the effects and duration of the domestic banking crisis of Kazakhstan.