Evaluation Approach Project Performance Evaluation Report for ADB Loans 1913/1914 Sri Lanka: Plantation Development Project July 2015 I.

Transcription

1 Asian Development Bank. 6 ADB Avenue, Mandaluyong City, 1550 Metro Manila, Philippines Tel ; Fax ; evaluation@adb.org; Evaluation Approach Project Performance Evaluation Report for ADB Loans 1913/1914 Sri Lanka: Plantation Development Project July 2015 Team Leader: Maya Vijayaraghavan, Senior Evaluation Specialist ( mvijayaraghavan@adb.org) Contact: evaluation@adb.org I. INTRODUCTION 1. The objective of this project performance evaluation report (PPER) is to assess the performance of Asian Development Bank s (ADB) loans provided to the Ministry of Plantation Industries (MPI) in Sri Lanka to ensure the plantation sector's long-term sustainability without further external assistance. 1 Findings and lessons drawn from this evaluation will feed into the country assistance program evaluation (CAPE) for Sri Lanka. 2. The PPER has been scheduled for preparation five and a half years after the loan closed in February 2010 and five years after circulation of the project completion report (PCR) in December II. BACKGROUND 3. Twenty Sri Lankan regional plantation companies (RPCs) were divested or privatized under the ADB supported Plantation Reform Project. 2 Consequently, yields, production, and profitability of the sector improved, but high production costs and low profitability persisted as a result of neglect and poor management during the period that the sector was nationalized. Further, due to regulations and restrictions on direct tea exports, the tea sector continued to produce primary commodities rather than diversify its production. As a result of poor living and working conditions of estate workers, the sector also faced a looming labor shortage stemming from out-migration and voluntary youth unemployment. Additional support was needed to transform producers of primary commodities to agribusiness entities capable of providing a competitive remuneration package to attract and retain talent. III. PROJECT DESIGN AND IMPLEMENTATION 4. The expected impact of the Plantation Development Project (PDP) was to ensure the plantation sector s long-term sustainability without further external assistance. The expected outcomes of the project were enhanced profitability of the plantation sector and improved living 1 ADB Report and Recommendation of the President to the Board of Directors: Proposed Loan to Sri Lanka for the Plantation Development Project. Manila. 2 ADB Report and Recommendation of the President to the Board of Directors: Proposed Loan to Sri Lanka for the Plantation Reform Project. Manila. This project was designed to transfer ownership of the plantation sector through divestment of the 23 regional plantation companies (RPCs) comprised of 443 estates. The RPCs consisted of tea, rubber, and coconut estates, and tea and rubber factories. The Plantation Development Project is follow-on to this project.

2 2 and working conditions of the estate work force to a level comparable with that in other sectors. 3 The project had four main components to achieve these objectives: (i) Investment: This component aimed to provide long-term financing to the RPCs in the form of a credit line through participating financial institutions (PFIs), and equity and quasi-equity instruments through a Plantation Fund; 4 crop and noncrop diversification; factory consolidation and automation; and effluent treatment plants and marketing ventures; (ii) Social and Environment Programs: This component focused on improving social well-being of estate workers who faced poor living and working conditions. Project sub-components were workers housing loans, workers amenities, social programs, and environmental initiatives; (iii) Marketing Initiatives: Marketing initiatives through product research and development, a marketing intelligence and resource center, support for automated tea auctions, and support for alternative marketing mechanisms; and (iv) Institutional Development and Project Management: Institutional strengthening through support for an industry umbrella body (the Tea Authority of Sri Lanka), consulting services, training, development of subleasing and outgrower models, and project management. 5. The total project cost at appraisal was estimated at $114.4 million equivalent, comprising $34.4 million (30%) in foreign exchange and $80.0 million in local currency equivalent. ADB support accounted for 26% of project financing, 5 with the balance of $84.4 million equivalent to be financed by the beneficiaries (34%), 6 the Government (33%), and the PFIs (7%). Actual cost at completion was $67.21 million, of which $23.65 million was in foreign exchange costs and $44.56 million was in local currency. A total of $18.97 million was disbursed under the ADB loan With a loan effectiveness date of August 2003, the project was scheduled to be implemented over a six-year period. Project costs covered civil works, equipment, vehicles, training, research and development, and consulting services. An initial environmental examination (IEE) was undertaken and a summary included. There were no major issues related to safeguards, or other social risks. IV. MAJOR FINDINGS OF THE PROJECT COMPLETION REPORT AND PROJECT COMPLETION VALIDATION REPORT 7. The project completion report (PCR) and project validation report (PVR) were circulated in December 2010 and January 2012 respectively. Based on the ADB criteria of relevance, effectiveness, efficiency, sustainability, and impact, the PCR and PVR rated the project 3 The project was expected to replant and infill 1,750 hectares (ha) and 2,370 ha of tea, replant 3,750 ha of rubber, plant 2,500 ha of oil palm and 10,000 ha of forest plantation, and establish five marketing alliances. The Project aimed to provide housing loans to 6,000 estate workers, reroof 11,000 line rooms and provide welfare facilities and amenities in the workplace. 4 For tea, rubber, and coconuts. 5 Ordinary Capital Resources (OCR) loan of $10 million equivalent and Asian Development Fund (ADF) loan of $20 million equivalent % for investment subprojects, 30% for workers housing loans, 50% for social infrastructure and training. 7 This excluded the reallocation of savings of $15 million equivalent in 2008 to the proposed Power Sector Development Project, which reduced the ADB contribution by nearly 50%.

3 3 successful. 8 The project was rated relevant for support provided to the plantation sector, which was the largest export industry, and the biggest employer. 9 The social development programs were relevant to the s poverty reduction goals. The PVR noted the housing loans program was less relevant due to availability of alternative funding sources. 8. The project was rated effective in achieving its outcome. The investment component was the only source of long-term financing available to the RPCs to develop productive assets. Field development of tea and rubber exceeded targets, as did crop diversification into oil palm; diversification into forestry was less effective. The social development initiatives were highly effective, with improved working and living conditions for estate workers. However, marketing initiatives were not effective, and automation of tea auctions and electronic trading was not implemented. 9. The project was rated efficient on account of the reevaluated economic internal rate of return (EIRR) of 19% compared to 16% at appraisal. 10 The PVR noted that loan effectiveness was delayed by 9 months because of late compliance with conditions, particularly the submission of a bill to the Parliament for reconstitution of boards of tea-related industries. Procurement and disbursement was slow from , especially for the non-investment components. 10. The project was rated likely sustainable based on the nature of the inputs. The PCR noted that cropping benefits were likely to lead to higher RPC revenues, but the ability to convert these into acceptable profit margins remained a low probability due to a host of factors. 11 Introduction of equipment improved the efficiency of key plantation tasks, and management system upgrades led to improved labor relations and productivity gains. Worker satisfaction with living and working conditions had tangible impacts on worker productivity, both in terms of output and decreased absenteeism; the social investments were likely to be highly sustainable. 11. The PCR assessed the impact of the project to be significant. This was based on achievements in land-use change of RPC land, mainly replanting and infilling of tea, rubber, and coconut that was expected to generate an incremental revenue stream to 16 RPCs. Impact on factory modernization was significant, as were improvements in social and health indicators of most estate workers. The overriding impact of the project was to change the culture of estate management and operations. The PVR noted that if the policy-related covenants had been complied with and the RPCs had been more inclined to contribute to the social programs, a rating of substantial impact would have been merited. 12. Lessons identified in the PCR were: (i) the expectation the RPCs would meet 50% of social development costs was unrealistic given the financial position of the companies; (ii) the project design was unduly complex; (iii) the inclusion of additional inputs for the development of the Tea Authority of Sri Lanka, marketing alliances, and demand-based public research were 8 ADB Completion Report: Plantation Development Project. Manila; ADB Validation Report: Plantation Development Project. Manila. 9 The plantation sector accounted for around 19% of foreign earnings, and employed about 265,000 workers. 10 This reflects larger areas of tea and rubber replanting and rapid increases in commodity prices, especially for rubber, tea, and palm oil. The PCR noted that many of the gains have been eroded by wage increases, and profitability of RPCs is contingent on controlling wage costs. Other constraints identified in the PCR were a relatively short remaining lease period (35 years), a unionized resident workforce, limited profitability, and an uncertain political environment. 11 For example, due to wage escalation, inflation-linked lease payments, and uncertainty over rights to forestry resources.

4 4 not client-driven, and they lacked appreciation of the commercial and proprietary rights attached to market relationships; and (iv) there was inadequate consideration of sector financial risks, including wage inflation, and political risks related to price escalation in the sector. V. MAIN ISSUES TO BE ADDRESSED BY THE EVALUATION 13. The proposed PPER will assess various aspects of project formulation, design, implementation and sustainability, and assess project performance taking into account impact, outcome, and output indicators realized by the project. The PPER will reevaluate the project in detail to confirm/update the findings of the PCR using the evaluation criteria of: (i) relevance, (ii) effectiveness, (iii) efficiency, (iv) sustainability of outputs and outcomes achieved to date, and (v) impact. 12 An indicative evaluation framework is presented in the Appendix. 14. Of the 16 RPCs that participated in the project, 12 achieved profitability in 2009 mainly due to record tea and rubber prices. Yield and cost of production benefits from the previous Plantation Reform Project and PDP were expected in five to seven years time. Five and a half years after loan closing, the evaluation aims to assess profitability of the RPCs that participated in the PDP. 15. At the time of loan closing, the ability of RPCs to finance adequate replanting, infrastructure maintenance, and social programs was still weak. Maintenance of physical assets constructed or rehabilitated and continued well-being of the estate workforce is crucial to sustain the benefits of the PDP. The evaluation aims to assess the adequacy and appropriateness of operations and maintenance of ADB project investments, and the current status of estate worker living and working conditions in a representative sample of RPC estates. VI. EVALUATION SCOPE AND APPROACH 16. The PPER will use a mix of the following methods to gather evidence for the evaluation: (i) desk review of project documents/reports; (ii) review of available RPC annual reports and financial statements; (iii) review of available project data and national statistics on socioeconomic indicators for estate workers (iv) discussions with project staff in Manila and Colombo; (v) discussions with agencies and other project stakeholders in Sri Lanka; and (vi) site visits to a representative sample of RPC estates. VII. RESOURCE REQUIREMENTS, SCHEDULE, AND KNOWLEDGE DISSEMINATION 17. The evaluation will be undertaken by a team led by a Senior Evaluation Specialist (IED Team Leader) and supported by an IED Senior Evaluation Officer, IED Senior Evaluation Assistant, an international consultant with expertise in the sector, and a national consultant. 18. The independent evaluation mission (IEM) is proposed to be undertaken for a maximum of 10 working days in September/October In addition to discussions with agencies and other stakeholders in Sri Lanka, the IEM will plan site visits to a representative sample of RPC estates. 19. The proposed schedule for the PPER is subject to clearance of the mission by the Government of Sri Lanka and the engagement of suitable consultants: 12 Independent Evaluation Department Guidelines for Preparing Performance Evaluation Reports for Public Sector Operations. (amended in March 2013). Manila:ADB.

5 5 Milestone Tentative Date Approval of evaluation approach paper I July 2015 Recruitment of consultants and desk review July-August 2015 ADB operations staff I II September 2015 Independent Evaluation Mission IV September I October 2015 Analysis and Report Writing October 2015 IED internal peer-review II November 2015 Draft PPER Interdepartmental Circulation III November 2015 Draft to Editor IV December 2015 Submission to Director, IED2 IV January 2016 Submission to Director General I February The PPER will be peer reviewed by Andrew Brubaker (Senior Evaluation Specialist, IED1) and Srinivasan Palle Venkata (Evaluation Specialist, IED2), and will be available to the public after approval by the Director-General, IED. The report will be uploaded on ADB s external and internal websites and will provide inputs to ADB s evaluation information system.

6 6 Appendix EVALUATION FRAMEWORK Evaluation Criteria Relevance Evaluation Questions Was the project relevant to national priorities in Sri Lanka? How does the project fit with similar projects implemented by the Government, bilateral, and multilateral organizations? Was the project aligned with ADB s policies and strategies? Were the project objectives of enhancing profitability of the plantation sector and improving living and working conditions of estate workers appropriate to address the development needs of Sri Lanka? Were the project components adequate to meet the objectives of the Indicators/Information Required Socioeconomic indicators for Sri Lanka Government development policies ADB s country operational strategy and sector strategy Project design and monitoring framework Source of Information Report and Recommendation of the President (RRP), project documents, project administration manual, Project Completion Report (PCR), and back to office reports (BTORs) Minutes of Staff review committee meetings, Management review meetings Relevant documents Relevant documents from other bilateral and multilateral organizations officials, and other key stakeholders Methods/Analysis to be used Desk review Analysis of program design indicators Was the project design appropriate to address the envisaged impact, outcomes, and outputs? Did the project inputs, outputs and outcomes follow the logical results chain to achieve the project s objective? Were the project outputs and outcomes measurable and achievable within the project

7 Appendix 7 Evaluation Criteria Evaluation Questions timeframe? Indicators/Information Required Source of Information Methods/Analysis to be used Were the relevant lessons learned from similar interventions earlier included in the design of the Effectiveness To what extent were project outputs and outcomes achieved as indicated in the DMF? What are the factors that contributed to the achievement or non-achievement of expected outputs and outcomes? What were the roles of stakeholders in the Realized project outputs and outcomes in relation to the targets set Project monitoring framework Implementation process Involvement and participation of stakeholders Issues and challenges related to achieving outputs and outcomes RRP, project documents, project administration manual, PCR, and BTORs officials and other key stakeholders Monitoring reports of outputs and outcomes RPC annual reports RPC financial statements Desk review Analysis of program design indicators Efficiency How well were the project s resources used in achieving the expected outcomes? How efficient were procurement and use of equipment and materials obtained during the Were there any delays in implementation or changes in scope? If yes, what were the reasons? Loan disbursement and fund utilization data Implementation and procurement arrangements Monitoring data on inputs and outputs EIRR National statistics RRP, project documents, project administration manual, PCR, and BTORs officials, and ADB procurement staff RPC annual reports RPC financial statements Desk review Sustainability What is the likelihood that crop and non-crop yields and productivity will be sustained beyond the life of the Project profitability FIRR officials, stakeholders RPC annual reports RPC financial

8 8 Appendix Evaluation Criteria Evaluation Questions What is the likelihood that reductions in cost of production, and increased labor productivity will be sustained? Indicators/Information Required Source of Information statements Published documents Methods/Analysis to be used What is the likelihood that the improvements in social well-being of estate workers will be sustained? Are there adequate incentives for continued stakeholder participation beyond the duration of the Impact Can the business operations and financial position of the RPCs be sustained? Are there any unintended or adverse project impacts on society or the economy? Government reports and statistics. officials, stakeholders Desk review Direct observation through field visits