Actual Project Name : My Ozone Depleting Substances Phaseout Project L/C Number: Project Costs (US$M US$M):

Transcription

1 Public Disclosure Authorized IEG ICR Review Independent Evaluation Group Report Number : ICRR Project Data: Date Posted : 08/30/2013 Public Disclosure Authorized Public Disclosure Authorized Country: Malaysia Project ID : P Appraisal Actual Project Name : My Ozone Depleting Project Costs (US$M US$M): Substances Phaseout Project L/C Number: Loan/Credit (US$M US$M): Sector Board : Environment Cofinancing (US$M US$M): Cofinanciers : Board Approval Date : 08/15/1995 Closing Date : 06/30/ /31/2010 Sector(s): Other industry (100%) Theme(s): Pollution management and environmental health (100% - P) Prepared by : Reviewed by : ICR Review Group: Coordinator : Richard C. Worden Robert Mark Lacey Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: This was a stand-alone project supported by the Montreal Protocol Investment Fund. No World Bank Group financing was involved. The Memorandum and Recommendation of the Director for Country Department 1 of the East Asia and Pacific region to the Bank s regional Vice President of August 11, 1995 (hereafter referred to as the MRD) stated the original Global Environmental Objective (GEO) as: to assist the Government of Malaysia (GOM) in carrying out its Country Program to phase -out ozone depleting substances (ODS) by implementing cost-effective investment sub-projects to reduce consumption of ODS in Malaysia (on page 2 of both documents). Public Disclosure Authorized The wording of the GEO as stated in the Ozone Projects Trust Fund of the Montreal Protocol on Substances That Deplete the Ozone Layer ( Montreal Protocol ) is slightly different, but the meaning and effect are the same (OTG Grant Number MA, Schedule 2 (no page numbers), dated October 23, 1995). The original outcome target was about 30 ODS phase-out investment sub-projects by the end of The GEO was modified in an amendment to the Grant Agreement signed on May 9, 2002 by the Government of Malaysia and the Bank. It read as follows (with italics added to identify changes to the original Grant Agreement wording): The overall objective of the Project is to assist the Recipient in carrying out its Country Program aimed at phasing-out the use of ozone depleting substances within its territory through the implementation of cost-effective priority investments and the National CFC [Chlorofluorocarbon] Phase-out Plan. The original outcome target was to assist the Government phase -out 1,014 metric tons of consumption of ozone depleting substances (ODS). The revised outcome target was to assist the Government phase-out 2,147.5 metric tons of potential ODS by implementing priority investment sub-projects and the National CFC Phase-out Plan.

2 b.were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? No c. Components: The original Project design (estimated: US $20 M; actual: US $20 M) had one component to provide financial assistance to large ODS-consuming enterprises in five industrial sectors to convert their manufacturing processes to use non ozone-depleting substances. At the time the Project was appraised and approved, ODS consumption in Malaysia was dominated by two chlorofluorocarbon agents : CFC-11 and CFC-12. These were used as refrigerants in mobile air conditioners, as blowing agents in foam insulation production, and as propellant for aerosol products. Much smaller amounts of 1,1,1-trichloroethane (TCE) were used as a metal cleaning solvent, and even smaller amounts of carbon tetrachloride (CCl 4 ) were used primarily for laboratory testing. The revised Project was extended in May 2002 for eight more years until the end of 2010, and the Montreal Protocol grant was increased by US $5 M to assist the Government implement its National CFC Phase -out Plan, which had just been approved in December Part A: A Investment Activities (estimated: US $21 M; actual: US $17.62 M). The Project was redesigned and expanded to include technical and financial support to thousands of small and medium -sized enterprises in four of the five industrial manufacturing sectors (except for aerosol propellants, which were only used in the manufacturing processes of three large companies in Malaysia ) that were addressed under the original Project scope. These activities were carried out on an individualized sub -project basis for medium-sized companies. For small manufacturing firms in the foam and solvent cleaning sectors as well as servicing companies in the mobile air conditioner and refrigerant sectors, the Project adopted a group approach to implement sub -projects using a Group Coordinator and a voucher system for Project participants who had been certified to buy equipment to service air conditioning and refrigeration systems. Part B: B Institutional Development (estimated: US $3.0 M; actual: US $2.5 M). A new Project component was added to strengthen the implementation capabilities of relevant government agencies (Department of Environment, Ministry of International Trade and Industry, and customs authority ) to prepare, plan, implement, monitor and enforce new government regulations. These regulations established an ODS import permit system, banned the use of CFCs in manufacturing processes, and trained and certified technicians working in the refrigerant and mobile air conditioning servicing sectors. These activities were carried out under the direction of the Department of Environment through the provision of consultants services, training workshops, an awareness -building publications, public announcements, and events. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : Total Project cost was US $20.46 million; the estimated cost at appraisal was US $ 20 million. This was estimated to increase by US $5 million in 2002 following the formal amendment of the Project to assist the Government of Malaysia implement the NCFCP, in compliance with its national obligations under the Montreal Protocol. Of this new total of US $25 million, US $21 million was allocated to goods, works, and services associated with the sub-grants projects; US $3 million to the new Institutional Development component, and US $ 1 million was split between the local implementing agents fees and contingencies. However, of this additional US $5 million, only US $460,000 was expended due to savings obtained from the sub -grants projects of US $3.38 million that effectively opened up room for the expansion and extension of the Project to eliminate totally the consumption of CFCs in Malaysia by 2010 in accordance with its Montreal Protocol obligations (see Figure 1 of the ICR, p. 11). Financing : The Project was financed completely by the Ozone Projects Trust Fund of the Montreal Protocol (OTG Grant Number MA). There was a gap of nearly four years in disbursements made by the Multilateral Fund for the Implementation of the Montreal Protocol (MLF) between the 1 st of October 1996 and the end of June 2000 when US $6.02 million was disbursed. Then, only US $800,000 more was disbursed over the next two-and-half years until the Project was restructured at the end of This was due mainly to the fact that the original conceptualization of the Project was to address only large manufacturing companies in five key industries with sophisticated management capacity and large financial resources requiring little technical or financial assistance. Once it became apparent that the National CFC Phase-Out Plan (NCFCP) would be rolled out in 2002, the scope of the Project changed to include small and medium-size firms in the retail and service sectors of those industries. This required much greater technical and financial support from the Project. From 2003 onward, disbursements were made more smoothly and regularly until the Project closed at the end of 2010.

3 Borrower Contribution : None was planned nor materialized. Dates: The closing date of the original Project was extended three times to accommodate the implementation of on-going sub-[grant investment] projects (ICR, p. 3). First, the Project was extended from December 31, 1998 until June 30, 1999; a second time to December 31, 2001, and a third time to June 30, Following the Project s formal restructuring when the Grant Agreement was amended on May 9, 2002 to assist in the implementation of the NCFCP, the Project closing date was extended for eight years until December 31, 2010 for the reasons cited above. 3. Relevance of Objectives & Design: a. Relevance of Objectives: (i) Relevance of Original Objective : Substantial. The original objective of the Project was relevant to the country s priorities and obligations as well as to the 1993 and 1999 Country Assistance Strategies (there is no current Country Assistance Strategy between Malaysia and the Bank as the country has voluntarily stopped borrowing from the Bank ). The country's priority was to implement its legal obligations under the Montreal Protocol, to which it became a signatory in The demand for imported chlorofluorocarbons (CFC-11 and CFC-12 CFCs) had increased by 8.5% annually between 1989 and 1994 when the Project was prepared. The 1993 Country Assistance Strategy stressed a selective assistance program in line with the Government s priorities: including for the environment and private sector development (p. 19). The 1999 document mentioned the Montreal Protocol Project specifically (p. 18). (ii ii) Relevance of Revised Objective : High The Project s objective and focus was later expanded to include more medium -sized enterprises as well as hundreds of small firms in the refrigerant and mobile air conditioning servicing industries. It was only possible to reach the agreed-upon ODS reduction goals of the National CFC Phase -out Plan and the country s obligations under the Montreal Protocols by expanding the original design of the Project to encompass these priority industrial sectors. b. Relevance of Design: Relevance of Original Design : Substantial The original design of the Project was oriented exclusively toward implementing up to 30 cost-effective investment sub-projects with factories in five high-priority manufacturing sectors: foam insulation, refrigerants, solvents, aerosol propellants, and mobile air conditioners. These beneficiaries were large manufacturing enterprises with the internal technical capacity to manage and implement individual sub -projects, and were hence not expected to require technical assistance. Design involved a single Component and Outcome Target (to reduce the consumption of CFC-11 and -12, TCE, and CCl 4 by 1,014 ozone-depleting potential tons among this group of manufacturing companies by the end of 1998). Relevance of Revised Design : Substantial. The results framework of the revised project generally presented a clear and logical link between the activities to be financed and the expected outputs and outcomes. The addition of up to 50 additional medium-sized enterprises in four of the same five manufacturing sectors, and up to 3,000 mobile air conditioning and 600 refrigerant servicing technicians working in small service repair shops, required the project to support training, certification and equipment to convert those substances from ODS to more ozone -friendly substances. The design incorporated different modalities for delivering technical and financial support according to the size of the enterprise and its capabilities. Technical assistance in addition to financial subsidies was necessary due to the limited capabilities and resources of the smaller and medium-sized enterprises involved. Design also included an institutional strengthening component to address capacity limitations in the Department of the Environment and reinforce coordination with other government agencies involved in the enforcement of, initially, import licensing for ODS, and later of the prohibition of all ODS imports. One shortcoming in the results framework was the absence of targets for the outcomes of the institutional strengthening activities. 4. Achievement of Objectives (Efficacy): Achievement of Original GEO : to assist the Government phase -out 1,014 metric tons of consumption of ozone depleting substances (ODS ODS) by implementing up to 30 cost-effective investment sub -projects by the end of 1998: Modest. The target had still not been met when the project was restructured in 2002, four years later than the original date of By 2002, 862 metric tons had been reduced under the project, 85% of the target of 1,014 metric tons. According to information subsequently provided by the project team, the reason for this shortfall was that, of the 30 enterprises designated for investment sub -projects, thirteen had not required the project s assistance since they had

4 already converted their productive processes to ozone -friendly substances using their own resources, and a further company was ineligible due to majority foreign ownership. The project therefore benefited 16 of the 30 originally designated enterprises. Achievement of the Revised GEO : to assist the Government phase -out 2, metric tons of potential ODS by implementing priority investment sub -projects and the National CFC Phase -out Plan: High. The target of reducing 229 metric tons of ozone-depleting potential substances among an estimated 50 mid-sized firms in the four priority manufacturing sectors (foam production, solvents, refrigeration, and mobile air conditioners ) was exceeded by 20 metric tons, even though it was achieved by 42 of the original 50 firms identified by the National CFC Phase-out Plan. In terms of determining the reductions of ozone -depleting potential metric tons consumed by the two sectors servicing small-scale mobile air conditioners and refrigerants, this could only be measured indirectly by reviewing annual reports of ODS imports audited by the Government s own National Auditing Department. These reductions in ODS imports were compared to annual maximum allowable limits agreed to between the Government and the MLF. Figure 1 of the ICR (p. 11) depicts the actual decline in CFC consumption over the entire 16-year period of the Project as well as the reductions required by the Montreal Protocol. Throughout this entire period of time, the actual consumption of CFCs, as determined from these audited CFC import records, was below both the Montreal Protocol obligations and the maximum allowable limits agreed to between the Government of Malaysia and the MLF. By 2010, CFC imports (which represented 2,092 of the 2,148 metric tons of all three types of targeted ODS ) had been completely eliminated (the other types of ODS were small amounts of TCE and CCl.) The fact that imports were completely eliminated by 2010 and that the Project had set out to reduce consumption by between roughly three-quarter of this total, or 3,000 3,200 metric tons out of a total of nearly 3,900 metric tons of ODS indicates that the targets were exceeded. This is further supported by other corroborating evidence, such as the destruction of machinery that would be required to revert back to using ODS, the high cost of obtaining ODS due to their legal prohibitions, and the risk of legal consequences due to continued monitoring and enforcement efforts by the Government. With regard to attribution, the ICR (pages 10-11) states: "Without the NCFCP, it was likely that Malaysia would not be able to meet phase-out obligations of CFCs during Both investment and institutional development activities under the NCFCP assisted the Government of Malaysia to accelerate the phase -out of ODS consumption from 2003 onwards, which were additional to the Montreal Protocol phase -out schedules. Achievements from 2003 onwards were only attributed to the project because there was no new sub -project approved to Malaysia to phase -out consumption of these chemicals after approval of the NCFCP. About 2,092 ODP tones of CFCs, 51 ODP tons of 1,1,1-trichloroethane and 4.5 ODP tons carbon tetrachloride was phased out during this period." 5. Efficiency: No ex-ante or ex-post economic or financial analyses of the project were prepared. Eligibility requirements for the benefiting companies included a cost-effectiveness threshold established by the Executive Committee (ExCom) of the Montreal Protocols. Cost-effectiveness was measured in terms of the amount of grant funds spent to phase -out each unit (in kilograms) of ODS that were eliminated during both phases of the Project. Efficiency of the Project under the Original GEO ( ): Modest. During the first seven years of Project implementation, the efficiency with which Project investment activities was carried out was modest. According to the ICR, the Memorandum and Recommendation of the Director indicated that the Project would support about 30 ODS phase-out sub-projects by the end of 1998 (p. viii). However, only 16 were eventually implemented, and these took nearly four years longer to achieve than estimated at appraisal. In terms of the ratio of US dollars spent to reduce each unit of ODS, the actual cost -effectiveness of US $10.11 per kilogram of ODS reductions was 7% above the appraisal target of US $9.43. Implementation was delayed by the initial reluctance of the companies to disclose proprietary financial data to the Project during appraisal, lack of knowledge locally about manufacturing alternatives using more ozone -friendly technologies and products, slow government permitting and licensing procedures, delays in bidding processes and in transferring technologies to meet demand, and the financial crisis of which caused considerable difficulties for enterprises to acquire other funding sources to advance the payments for the equipment purchased before claims for reimbursement were made (ICR, p. 5). Efficiency of Project under the Revised GEO ( ): Substantial. After restructuring the Project, the efficiency with which Project investment activities were implemented improved markedly. The time period (eight years) taken to produce the project outcome was the time foreseen at the 2002 restructuring when the closing date was formally extended to Despite the increased difficulty and complexity of

5 the Project during this phase, a number of adjustments in the delivery of services, better coordination between different government agencies, and improved cost -effectiveness resulted in more efficient implementation and value for the money spent. The ozone-depleting potential reductions achieved by 42 mid-sized manufacturing firms surpassed the target set in the amendment to the Grant Agreement by 20 metric tons or nearly 10% above the target of 229 metric tons. It achieved this with a cost-effectiveness ratio of US $10.49/kilogram of ODS reduced, or roughly two-thirds of the cost of US $15.49/kilogram estimated at restructuring. In the mobile air-conditioning sector, the target numbers of training centers (30) was exceeded by 10, the target number (60) of sets of training equipment sold at subsidized prices was surpassed by 20, and the target (3,000) for technicians certified was exceeded by more than 1,750. In terms of financial subsidies provided, the number of recovery & recycling machines provided was increased by nearly 250% (from 350 to 858) while the number of basic service tools remained close to the target of 3,000 units. Unexpected cost savings for the manufacturing sectors paid for almost the entire cost of providing the additional 508 recovery & recycling machines. In the refrigeration servicing sector, the target number of training centers and training equipment provided was nearly met, while the more than 2,500 technicians certified was more than triple (330%) the target of 600, again without increasing the originally estimated cost. The financial subsidies provided in the form of vouchers issued and exercised (695) also exceeded the target of 600 at estimated cost. The voucher scheme allowed thousands of small enterprises to acquire equipment in an efficient and transparent manner, and resulted in a 20% reduction in the price of these machines in the market. The flexibility given to the implementing agencies to add beneficiaries or change tactics contributed toward achieving the objectives of the National CFC Phase -out Project s efficiency and ultimate success. a. If available, enter the Economic Rate of Return (ERR ERR)/Financial Rate of Return (FRR FRR) at appraisal and the re-estimated estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal ICR estimate No No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Outcome of Originally Designed Project : The relevance of objectives and of design was substantial. Efficacy (only just over half the originally estimated number of firms actually benefited from the project ) and efficiency are assessed as modest. Overall outcome of the original project is rated moderately unsatisfactory. This outcome rating corresponds to a 3 on the six-point rating scale. Outcome of Restructured Project : The relevance of objectives was rated high while the that of design was rated substantial. Efficacy was high -- the project substantially exceeded many of its output targets and achieved all expected outcomes. Efficiency of project implementation and cost -effectiveness improved significantly, and is assessed as substantial. Therefore, the outcome of the revised project is rated as highly satisfactory. This outcome rating corresponds to a 6 on the six-point rating scale. In accordance with the IEG/OPCS s Harmonization Guidelines, the score assigned to the initial phase of the Project s outcome is multiplied by the percentage of Project funds disbursed at the time of restructuring (43.7%) while the score for the revised Project Outcome is multiplied by the percentage of Project funds pertaining to its implementation, or the remaining 56.3%. Outcome Rating for Initial Project: Moderately Unsatisfactory (3) X 43.7% = 1.75 Outcome Rating for Revised Project: Highly Satisfactory (6) X 56.3% = 3.38 Total Score: = 4.69, which may be rounded up to an overall outcome rating of satisfactory. a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk that Malaysia will backtrack on its international obligations under the Montreal Protocols and allow the use of ODS again is low for a number of reasons. First, no ODS are produced in Malaysia, so they must be imported,

6 which is now illegal. This ban on ODS imports is enforced by the Royal Customs and Excise Department and the Ministry of International Trade and Industry. All import permits and records are verified by the Department of Environment, which maintains an on-going monitoring program, and by the National Auditing Department. In addition, the main regional suppliers (China and India) have stopped producing and exporting ODS. All factories producing goods using such substances have been converted to ozone -friendly processes and materials, and the equipment used to maintain those units has also been destroyed and replaced by nearly one thousand sets of CFC recovery & recycling machines in service centers around the country. Although there will continue to be some demand for servicing of existing ODS-based equipment, its ever-more limited availability as current stockpiles run out will force firms to retire that equipment early and convert to ozone -friendly systems. Finally, the high cost of ODS relative to other ozone-friendly substances and the credible threat of civil sanctions for their importation and consumption have reversed the monetary and legal incentives to use them in the future. a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: a. Quality at entry: The Bank assisted the Borrower in preparing the Project and appraised it in accordance with the requirements of the MLF of the Montreal Protocols based on the merits and cost -effectiveness of each investment sub -project. The Memorandum and Recommendation of the Director provided justification for the Bank s involvement in developing and supporting the Project, identified the key issues to address and the priority manufacturing sectors through a national survey conducted in It also contained assurances that adequate institutional capabilities were in place within the Department of Environment (as the Implementing Agency) and its local implementing agency (CETEC) to implement the Project. However, the estimate of 30 manufacturing enterprises identified to receive Project financial assistance, and the related target of ozone-depleting potential tons of reduction (1,014 metric tons) to be achieved by the end of 1998, turned out to be too large. According to the project team, this was because a number of enterprises used their own resources to convert their production processes and materials to more ozone -friendly means, while others were assisted in making the transition by multi -lateral donors such as the UNDP and UNIDO. More research among the beneficiary companies, and the Malaysian private sector generally, as well as among other development partners, might have led to the identification of these risks. In addition, the decision of some of the companies to undertake ODS phase-out using their own resources could be attributable to the long delay between project preparation (the Concept Note was reviewed by the Bank in early 1991) and appraisal (the project was approved in August, 1995). Quality-at at-entry Rating : Moderately Unsatisfactory b. Quality of supervision: The issues which resulted in the delays and shortfalls prior to restructuring -- the reluctance of enterprises to disclose information, the financial crisis affecting the region, cumbersome government regulations, and inadequate identification of firms' interest -- cannot be blamed on weak supervision. The restructuring embodying the extension of the closing date until 2010 did not take place until 2002 (seven years after approval); however, it could not have taken place earlier since the new National CFC Phase -out Plan, which it was intended to support, was only completed in 2001 and rolled out in The quality of the Bank s supervision after restructuring was exemplary in a number of ways, and there were no shortcomings identified. First, the restructured project was well designed and implemented. The flexibility and adaptability displayed by the Bank team allowed the project to achieve and exceed the goals established by the National Plan. The innovative implementation of the group sub-projects assisted by a Group Coordinator and supported by the voucher system to acquire subsidized servicing equipment and training in its proper use was adjusted and adapted over time to work out some of the logistical and technical glitches that were initially encountered. Coordination between various Government agencies in monitoring the importation of ODS through the permit application process, auditing its veracity in a credible manner, and enforcing new Government regulations and prohibitions ensured that illegal importation and consumption of ODS were completely eliminated in Malaysia. According to the ICR, Without the National CFC Phase-out Plan, it is likely that Malaysia would not have been able to meet the phase-out obligations of CFCs during (p. 10). Second, the Bank conducted regularly scheduled supervision missions to assist the Government and Implementing Agencies in working out many of the challenges and obstacles faced throughout the project and to

7 ensure its ultimate success. For example, the Bank recommended reallocating the savings achieved in some of the manufacturing sector activities to provide more resources to fund other incomplete activities in the servicing sectors or to make additional investments in the institutional strengthening component. US $400,000 in savings from the manufacturing sector were used to expand the number of recovery and recycling machines provided to the mobile air conditioner servicing sector, and US $ 450,000 was transferred to sustain import enforcement activities (ICR, p. 8). A Quality Assurance Group (QAG) assessment conducted in 2004 stated that the Bank s performance helped the Government improve project implementation in several areas e.g. Project Management Unit staffing, reorganization of the National Ozone Unit in the Department of Environment, practical advice on new implementation modalities (voucher, training program), regulations, and public awareness activities (ICR, p. 15). According to the ICR, the Bank provided adequate training and supervision to the implementing agencies to ensure their full compliance with the Bank s fiduciary and safeguard policy requirements. Quality of Supervision Rating : Highly Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government fulfilled its commitment under the Montreal Protocols to help achieve the project s objective to phase-out the consumption of ODS by initially controlling their importation (since there was no domestic production of ODS) through an import control permitting system, and then by banning their consumption in the manufacturing sector after Although there were some issues regarding inadequate enforcement, the Government finally took remedial actions to address this problem (ICR, p. 16) in The Government passed the Refrigerant Management Law, approved the National CFC Phase -out Plan, and agreed to restructure the project in a second phase from to extend the its reach to medium-sized manufacturing companies and to hundreds of small-scale refrigerant servicing centers. It ultimately allowed the close coordination of various public agencies to effectively enforce the ban on imports of ODS via a credible auditing process and interdiction efforts of customs controls. And it reformed the National Ozone Unit within the Department of Environment in accordance with the recommendations of the Bank s Mid-Term Review. Government Performance Rating Satisfactory b. Implementing Agency Performance: The Department of Environment (DOE) was the Implementing Agency of the Project, though it contracted out many hands-on functions to CETEC (the Center for Environmental Technologies) to act as the Local Implementing Agency and work directly with manufacturing and servicing firms. During the period prior to the restructuring ( ), the Implementing Agency worked with 16 large, technically sophisticated manufacturing firms in five key sectors16 large, technically sophisticated manufacturing firms in five key sectors, reducing the use of ozone-depleting substances among those firms. After restructuring, DOE and CETEC agreed a more decentralized and adaptive approach to implementing the project with the Bank and with the ExCom of the MLF of the Montreal Protocol. This shift allowed them to make adjustments in the voucher program to subsidize the purchase of new less damaging equipment from the project in a more efficient and transparent manner (reducing the cost of some equipment by 20%). They also reallocated project resources derived from savings achieved in the manufacturing sector to broaden and strengthen the enforcement and training efforts of DOE in all of its district offices. Both activities continued after project closure. They overcame a number of logistical challenges in providing equal access to subsidized equipment and meeting demand in a timely manner after some early trial-and-error experiences. According to the ICR, the Bank s and MLF s procurement procedures were followed satisfactorily, documentation and filing systems were systematically and well maintained (ICR, p. 16), and credibly audited financial information and progress reports were reported on -time and in accordance with all requirements, allowing for the uninterrupted disbursement of annual tranches of Project funds from the MLF (see Section 11). The adoption of a new approach to providing training workshops and supporting a number of individual sub-projects as a single group sub-project allowed implementation to be accelerated. A more assertive public outreach effort by DOE and CETEC increased participation among the two refrigerant servicing sectors. As a

8 result of the combined effect of all of these efforts, targets were exceeded by wide margins in most cases. No flaws in implementation appear to have gone unresolved. Implementing Agency Performance Rating : Highly Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: During the first phase of the Project, M&E design was simple and straightforward. It consisted of CETEC preparing and submitting appraisals for each sub -project to the DOE first, and then to the Bank for clearance. All sub-projects also had to be approved by the ExCom of the MLF for the Montreal Protocols before funds could be disbursed. These appraisal reports were to include information on the expected ozone -depleting potential reductions resulting from the Project s intervention, the alternative technology to be employed to obtain those reductions, and the disposal plan for destroying existing equipment consuming ODS. After restructuring, the M&E framework was made more flexible in order to respond to the more complex and difficult service delivery modality needed to reach out to medium -sized and small enterprises. Thus, prior review and Bank clearances were required for only the first three investment sub -projects in each of the four medium-sized manufacturing sectors. During this phase of the Project, CETEC was to prepare the sub -project appraisal reports with inputs from the group coordinators and consultants to present them to DOE and the Bank for clearance. The clearly stated objectives were linked to measurable indicators with the exception of the institutional strengthening component. It was not clear from the ICR how the targets were established for the two ODS servicing industries (refrigerants and mobile air conditioners). b. M&E Implementation: Implementation of the M&E system was conducted by the local implementing agencies (DOE and CETEC) with inputs provided by various stakeholders. It was supervised by the DOE, the Bank, and the ExCom of the MLF. Baseline values were established for indicators in the results framework, which were regularly monitored and measured in accordance with Bank and MLF requirements, and the data collected appear to have been reliable and of good quality. During both phases of the Project, annual work programs were prepared by CETEC and DOE, approved by the Bank, and submitted to ExCom for disbursements of funds. After restructuring, audited ODS consumption verification reports were prepared by the National Auditing Authority and presented to the Bank and the MLF in order to release annual Project disbursements. There is no indication that any weaknesses in the design of the M&E system were fixed during implementation because they were not identified as such by Project implementers and thus were unlikely to have been changed. c. M&E Utilization: The ICR reports that the approach taken by the Project s M&E system was adopted by the Government, and continues to be used to track the number of service technicians trained and certified in both the mobile air conditioning and refrigerant servicing industries. There was ample evidence in Project documents that feedback provided by the Project s M&E system was used by Project implementers and stakeholders to make a number of important changes in the way that technical assistance and public outreach were provided to beneficiaries. For example, the voucher scheme for subsidizing ozone -friendly equipment to servicing technicians was adjusted in several important ways to improve fairness and transparency. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The Project was classified as Category C and did not trigger any safeguards policies. b. Fiduciary Compliance:

9 The ICR reports that fiduciary compliance was addressed by training Project staff in both the Bank s and the Montreal Protocol s specific fiduciary procedures and requirements. Disbursements were initially very low between 1995 and mid-2000, reaching only US $2.12 million. After restructuring, disbursements accelerated and were made on regular intervals. Financial management reviews were conducted annually during Bank supervision missions, and were found to be consistently satisfactory. The Project team maintained adequate records and documentation as required. External audits conducted annually by the Office of the Auditor General were unqualified, with no major accountability and internal control issues. According to the ICR (page 16), the Bank s and MLF s procurement procedures were followed satisfactorily, although there were some minor shortcomings regarding (a) the need for closer compliance with agreed procurement procedures and process that the prior /post reviews should be strictly followed, (b) the need for due diligence when using shopping procedures that companies should be qualified and well established authorized suppliers and (c) the need for timely submission of annual procurement plan. No cases of misprocurement were recorded. c. Unintended Impacts (positive or negative): One unintended positive impact of the Project was the Government s decision to continue offering training and certification workshops to refrigerant servicing industries to help them remain compliant with the country s new Refrigerant Management Law. This effort by DOE occurred despite the fact that the PMU within DOE, which had been supervising these workshops organized by CETEC, was dismantled accordingly after completion of the project (ICR, p. 29). However, it was not stated how many technicians are currently being trained and certified by DOE, nor the extent to which that level of training is sufficient to meet the demand for them in Malaysia. d. Other: 12. Ratings: ICR IEG Review Reason for Disagreement /Comments Outcome: Highly Satisfactory Satisfactory The outcome target for the project as originally designed was not met, and efficiency was modest. The outcome of the restructured Project was highly satisfactory, leading to an overall outcome assessment of Satisfactory. Risk to Development Outcome: Negligible to Low Negligible to Low Bank Performance : Satisfactory Borrower Performance : Satisfactory Quality of ICR : Moderately Satisfactory Satisfactory Satisfactory Although supervision, notably in the post-restructuring phase, is rated highly satisfactory, there were significant shortcomings in quality at entry leading to an overall rating of moderately satisfactory for Bank performance. NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons:

10 The most important lesson to be learned from this Project is the importance and effectiveness of integrating the various aspects or components of the Project toward achieving its clear and simple objective. Technical and financial assistance from the MLF or the Bank could not have achieved that objective alone. It required the integration of an upgraded regulation framework, targeted public outreach efforts, institutional capacity building, and technical and financial assistance offered as incentives to encourage voluntary private sector participation, thereby reinforcing the commitment to phasing out the use of ODS in Malaysia. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR was well-organized, concise yet complete, and presented sufficient information and evidence to permit an evaluation of the Project. There were a few important instances where confusing text may have been caused by the lack of native English skills among those preparing the document. However, these were clarified in subsequent personal interviews and exchanges with the Project team. The ICR does not state the reasons for the removal of 14 of the original 30 firms preselected as eligible for MLF funding (although the Borrower, in its comments on the draft ICR Review, clarified that it had explained the reasons to the ICR team ). The four year delay between the concept review and appraisal/approval is not sufficiently explained. a.quality of ICR Rating : Satisfactory