Evaluation Approach Technical Assistance Performance Evaluation Report on ADB Support for Social Protection: Responding to Shocks and Risks July 2017

Transcription

1 Asian Development Bank. 6 ADB Avenue, Mandaluyong City 1550 Metro Manila, Philippines Tel ; Fax ; evaluation@adb.org; Evaluation Approach Technical Assistance Performance Evaluation Report on ADB Support for Social Protection: Responding to Shocks and Risks July 2017 Team Leader: Hyun H. Son, Principal Evaluation Specialist, IETC ( hhson@adb.org) Contact: evaluation@adb.org I. Introduction 1. The Independent Evaluation Department (IED) assesses the performance of clusters of technical assistance (TA) projects to contribute to knowledge solutions and improve the operations within the Asian Development Bank (ADB). 2. The proposed technical assistance performance evaluation report (TPER) will examine ADB s support for social protection in the context of responding to various shocks and risks. The TPER will evaluate four TA projects: (i) Social Protection of Poor Female Workers in the Garment Sector in the Context of Changing Trade Environments in Bangladesh (started 2004, completed 2007, approved for $420,000); 1 (ii) Strengthening the Government's Institutional Capacity for Improving Food Security in Bangladesh (started 2008, completed 2011, approved for $600,000); 2 (iii) Strategic Capacity Development for Social Sectors in Mongolia (started 2009, completed 2011, approved for $700,000); 3 and (iv) Supporting the Development of a Social Protection Framework for Nepal (started 2010, completed 2011, approved for $200,000) The findings of the TPER will help strengthen ADB s social protection support to foster inclusive growth and eradicate poverty in Asia and the Pacific. They will contribute to improving ADB s efforts in helping developing member countries provide social protection in times of shocks and risks and strengthen their social protection programs to better prepare for shocks. The findings will also feed into the ongoing corporate evaluation on the design, use, and results of policy-based lending in ADB. 1 ADB Technical Assistance Completion Report: Social Protection of Poor Female Workers in the Garment Sector in the Context of Changing Trade Environments Project in Bangladesh. Manila. 2 ADB Project Completion Report: Emergency Assistance for Food Security Project in Bangladesh. Manila. 3 ADB Project Completion Report: Social Sectors Support Program in Mongolia. Manila. 4 ADB Technical Assistance Completion Report: Supporting the Development of a Social Protection Framework for Nepal. Manila.

2 2 4. This paper discusses the rationale; details of the four selected TA projects; evaluation scope, approach, and data sources; tentative schedule and resource requirements; and dissemination of the findings of the proposed TPER. II. Rationale A. Sector Background 5. Shocks cover adverse events that lead to negative economic and social impacts affecting entire communities or individual households. There are two types of shocks depending on the extent of their impacts. Covariate shocks have an effect on a large part of a population and can be triggered by economic crises including price hikes, natural or climatic disasters, pandemics, political crises, or conflicts. On the other hand, idiosyncratic shocks are commonly experienced over the course of a life-cycle such as illness, old age, or loss of employment or income The intensity and frequency of shocks and risks are expected to increase in the future. Extreme weather and climate events are likely to be more frequent and intensify amid rapid urbanization and migration. Economic volatility and financial crises are also seen to persist given greater economic and financial interdependence among nations through globalization. 7. Asia and the Pacific is not immune from shocks and risks. The region has experienced the repercussions of the Asian financial crisis and the global financial crisis. The international food crisis in triggered food price hikes in Asia and the Pacific, with the region s overall food price index increasing by 54% from January 2007 to June The region is also highly vulnerable to climate change, with 6 of the 10 most vulnerable countries to climate change in 2013 located in Asia and the Pacific Social protection programs are important tools in preparing for and responding to shocks and risks. They can strengthen people s capacity to manage risks before shocks occur by, for instance, promoting human capital development, and diversifying assets and livelihoods. Social protection also provides immediate support during shocks. Relief and assistance prevent households from falling into poverty or deeper poverty, while cash and in-kind transfers and public works interventions help avoid human capital losses. Social protection also helps ensure recovery in the aftermath of shocks through pro-poor, countercyclical public investments, and risk diversification programs. 9. ADB defines social protection as the set of policies and programs designed to reduce poverty and vulnerability by promoting functioning labor markets, diminishing people s exposure to risks, and enhancing their capacity to protect themselves against hazards and interruption and loss of income. Social protection comprises three major categories: social assistance, social insurance, and labor market programs Different shocks and risks require different social protection interventions, which should be adapted to the coping strategies and local conditions of affected populations. Skills development and training programs, for instance, can help cushion the impact of job losses due 5 Bastagli, F Responding to a Crisis: The Design and Delivery of Social Protection. Overseas Development Institute Working Paper. London. 6 ADB Food Security in Asia and the Pacific. Manila. 7 These countries are Cambodia, India, Lao People s Democratic Republic, Pakistan, Philippines and Viet Nam. See ADB Climate Change in Asia and the Pacific. Manila. 8 ADB Social Protection Strategy of the Asian Development Bank. Manila.

3 3 to economic crisis. Humanitarian emergencies in turn require food distribution or health services. The TPER will evaluate how effectively ADB provides social protection support to mitigate the impacts of shocks and risks, and contribute to the broader agenda of inclusive growth and poverty reduction. It will assess the extent to which ADB tailors its TA support to various shocks and risks to effectively respond to or prepare for these. The TPER will examine the types of social protection interventions supported by the four TA projects and how well they are suited in addressing the shocks and risks they intend to respond to. 11. What determines the effectiveness of social protection in the context of responding to shocks and risks? Timeliness, adequacy, and adaptability are important features that ensure effective social protection measures. The TPER will examine the extent to which the selected TA projects supported timely, adequate and adaptable social protection programs. (i) (ii) (iii) Timeliness: Social protection programs need to be implemented quickly and at a scale that covers a large share of populations affected by shocks. Adequacy: Social protection measures must fulfill their goal of maintaining consumption and access to services in times of shocks. They must have sufficient financing to allow for rapid response and scaling up. Adaptability: Social protection interventions need to be tailored to specific risks and shocks and coping mechanisms of affected populations (footnote 4). 12. Various challenges are associated with implementing social protection measures to effectively respond to shocks and risks. First, existing social protection policies carried out during non-crisis situations may not necessarily coincide with providing timely and adequate response during shocks. Second, provision of transfers and services may be more difficult in the context of shocks. Third, government budgets become more strained during crisis situations as the need for social protection increases. Moreover, governments usually allocate limited resources to social protection programs, thereby requiring specific elements of social protection such as targeting of beneficiaries. Four, it can be very difficult to design social protection programs when there is no crisis and the programs can be used to provide funds for political reasons. The TPER will assess how the selected TA projects mitigated these challenges. B. Strategic Context 13. ADB adopted its Social Protection Strategy in The strategy seeks to help people break the cycle of poverty, enhance growth through investment in human capital, increase productivity, and reduce their vulnerability to risk. ADB s social protection strategy has three main elements: (i) (ii) (iii) Labor market policies and programs aimed at promoting employment, efficient functioning of labor markets, and worker protection. Social insurance programs to cushion the risks associated with unemployment, catastrophic out-of-pocket health costs, disability, work injury, and the growing ranks of the elderly. Social assistance and welfare service programs that target the most vulnerable with no means of support, including single mothers, the homeless, or physically or mentally challenged people (footnote 4). 14. To provide strategic directions for strengthening social protection systems in Asia and the Pacific, ADB adopted its Social Protection Operational Plan, The plan has five priority areas: (i) continued development of ADB-financed social protection projects, (ii) active

4 4 identification of opportunities to integrate social protection into projects, particularly in the education, finance, health, and public management sectors, (iii) support to capacity building and policy, and knowledge sharing, (iv) cultivation of partnerships and South South cooperation, and (v) monitoring and reporting on social protection programs and trends in Asia and the Pacific The TPER will assess how relevant the social protection interventions of the four TA projects are to the objectives of the social protection strategy and strategic directions of the social protection operational plan. 16. ADB has supported social protection programs and policies to respond to various risks and shocks. For instance, in the aftermath of the global financial crisis, ADB bolstered social protection measures in Mongolia, the Philippines, the Peoples' Republic of China and Viet Nam, with loans amounting to more than $1.1 billion. Bangladesh also received ADB support amounting $645 million to help expand social safety net programs for populations affected by the financial crisis such as households reliant on remittances, laid off factory workers, women forced into low-paying and insecure jobs, and young people joining the labor market at a time of limited job opportunities. 10 III. Selected TA Projects: Interventions, Achievements, and Challenges 17. The four TA projects were approved and implemented over (see Appendix 1). They were selected to examine how well ADB s TA activities support social protection measures in the event of various shocks and risks such as economic shocks, food price spikes, unemployment risks, and fragile and conflict-affected situations. Table 1 identifies the intended social protection response to various shocks and risks of the four TA projects. Table 1: Technical Assistance Projects Covered by the Evaluation TA No. Country Project Title Shock or Risk Addressed 4320 Bangladesh Social Protection Unemployment Risk: of Poor Female The phase out of the Workers in the MFA quota restrictions in Garment Sector in Bangladesh resulted in the Context of the shutting down of Changing Trade some garment factories, Environments in which mainly employ Bangladesh women Bangladesh Strengthening the Government's Institutional Capacity for Improving Food Food Crisis: Bangladesh suffered from food price hikes in 2008, which pushed an additional 2.5 million households or 12 million people into Intended Social Protection Response (i) produce a strategic report on possible approaches for post-mfa human-capacity development, employment, and social protection measures, (ii) pilot projects to benefit female garment workers, (iii) support exchange of experiences, and (iv) identify and document lessons learned from the pilot projects, and scaling up of potentials. (i) improve the institutional capacity of the Ministry of Food and Disaster Management to administer, manage, and monitor safety net programs, and 9 ADB Social Protection Operational Plan, Manila. 10 ADB. ADB and Social Protection. Accessed 5 June

5 5 TA No. Country Project Title Shock or Risk Addressed Security in poverty. The country is Bangladesh also vulnerable to climate risks and extreme weather events that may disrupt agricultural output Mongolia Strategic Capacity Development for Social Sectors in Mongolia 7530 Nepal Supporting the Development of a Social Protection Framework for Nepal Economic Shock: Mongolia experienced the adverse impacts of the global financial crisis and economic downturn. The crisis triggered spikes in food and oil prices and high inflation in 2008 in the country. Fragile and Conflict Situation: Nepal is emerging from a decade long armed conflict, which ended in Socioeconomic and political inequalities were deemed the root cause of the conflict in Nepal. Intended Social Protection Response (ii) strengthen the institutional capacity of Ministry of Agriculture s policy, planning, and coordination unit for planning of food production, input requirement, and procurement and distribution of inputs. (i) support policy reforms on targeting assistance and improving social sector efficiency, (ii) strengthen statistical analysis and data collection, (iii) build capacity for the management of a national inter sector database of the beneficiaries of social assistance and other government subsidies, (iv) develop a consensus on targeting social assistance among national and local government officials and the public, and (v) develop environment and social safeguard capacity for urban development stakeholders in Ulaanbaatar (i) develop a consolidated Social Protection Framework, (ii) enhance knowledge and skills of the participating agencies in social protection, (iii) define organizational roles to implement social protection framework, and (iv) identify financial options for social protection TA = technical assistance, MFA = Multi-Fiber Agreement, No = number. Source: Independent Evaluation Department. 18. The four TA projects were self-rated successful by the operations departments involved (see Appendix 2). They were reported to have made important contributions to piloting social protection interventions that could be replicated or scaled up. The Social Protection of Poor Female Workers in the Garment Sector in the Context of Changing Trade Environments in Bangladesh piloted social protection measures in a changing industry environment as social protection for garment workers was not in place. It helped about 1,500 women workers (not only retrenched) access housing services, skills training, and health care, and to redeploy about 500 out of about 1,000 retrenched women workers back into the sector. The labor market and social

6 6 protection pilot programs supported by the TA project were said to have provided a model that could be replicated in other sectors or other countries undergoing industry restructuring. Similarly, the TA project on Strengthening the Government's Institutional Capacity for Improving Food Security in Bangladesh introduced a web-based monitoring system for food-stock management and delivery in selected upazilas. The system was said to be amenable to replication nationwide. This will be checked by the evaluation. 19. The selected TA projects also supported policy discourse on social protection. The Strategic Capacity Development for Social Sectors in Mongolia encouraged policy dialogue and work to raise awareness and build consensus on targeted social assistance. It contributed to the passage of the amended Social Welfare Law in January The TA project on Supporting the Development of a Social Protection Framework for Nepal also led the way for a vibrant policy discourse on social protection in the country. It produced a draft social protection framework, which includes a proposed institutional structure to implement the government s social protection policy. These reported achievements will be checked in the field and their status will be updated. 20. Complex government procedures and a lack of evidence on the impacts of social protection measures were among the challenges faced by the selected TA projects. The Strengthening the Government's Institutional Capacity for Improving Food Security in Bangladesh TA project experienced delays due to procedural complexity at the ministries and departments. On the other hand, the draft framework produced through the Supporting the Development of a Social Protection Framework for Nepal TA project would supposedly have benefitted from evidence on the coverage and impacts of the country s cash transfers and public works programs. Hence, decisions made in the draft framework, such as the lowering of the eligibility for receiving the old age allowance from 70 years to 65 years, were not informed with such evidence. IV. Evaluation Scope, Approach, and Data Sources 21. The proposed TPER will examine how ADB s social protection support contributes to its goals of promoting inclusive growth and eradicating poverty in Asia and the Pacific. It aims to assess whether ADB delivers timely, adequate and adaptable social protection support through its TA projects to effectively respond to various shocks and risks. It will examine the four TA projects based on the evaluation criteria of: (i) relevance, (ii) effectiveness, (iii) efficiency, (iv) sustainability, and (v) impact. The TPER will follow the IED guidelines for TA evaluations The TPER will use primary and secondary data and will be prepared in accordance to IED s evaluation guidelines for public sector operations. The methodologies will include: (i) desk review of relevant project documents, (ii) key informant interviews with ADB project officers and staff of executing and implementing agencies in Bangladesh, Mongolia and Nepal, and (iii) focus group discussions with TA projects beneficiaries and other stakeholders in these countries. Table 2 lists the evaluation questions under these criteria and the data sources. Table 2: Evaluation Questions by Criteria Criteria Evaluation Questions Data Sources Relevance (i) Did the TA projects support social protection measures relevant to the identified shocks and risks, and their root causes? (ii) To what extent were the designs of the TA projects informed by sound diagnostics of the: Review of technical assistance completion reports, and other project documents; key informant interviews with ADB project 11 IED Guidelines for the Evaluation of Public Sector Operations. Manila: ADB. Appendix 3.

7 7 Criteria Evaluation Questions Data Sources (a) root causes of shocks, (b) associated risks faced by affected populations and their coping mechanisms, (c) designs and objectives of existing social protection measures, and (d) funding for and political commitment to social protection programs? (iii) (iv) (v) (vi) (vii) To what extent did the TA projects provide timely and adaptable social protection response to the identified shocks and risks? Were the TA projects aligned with the social protection policies of the government and ADB, and ADB s goals of inclusive growth and poverty reduction? Were the targets and indicators identified in the DMFs adequate to capture shock- and community-specific challenges? Were they feasible to be achieved given the identified time frame? Were communities affected by shocks and risks properly targeted? To what extent were affected communities engaged in project design, implementation and monitoring? Effectiveness (i) To what extent were the intended outputs and outcomes identified in the DMFs achieved? (ii) What factors contributed to the achievement (or lack of) of the outputs and outcomes? (iii) To what extent were the social protection programs adequate, i.e. did social protection programs achieve their objectives and mitigate the adverse impacts of the identified shocks and risks? (iv) What wider policy or institutional factors by governments and ADB facilitated or hindered the achievement of the outputs and outcomes? Efficiency (i) How efficiently were project resources such as time and funding used to achieve the intended outputs and outcomes? (ii) (iii) (iv) How flexible were the project designs to allow for midterm corrections, if any? To what extent did ADB and government processes, and socioeconomic practices and conditions of affected populations hamper or facilitated the efficient implementation of the projects? Did executing and implementing agencies have adequate staffing and skills to efficiently implement the project? (v) What could have ADB done differently to prevent project delays, if any? Sustainability (i) To what extent were the achieved outputs and outcomes sustained? (ii) Do executing and implementing agencies have sufficient human, institutional, and officers, staff of executing and implementing agencies, and other stakeholders. Review of technical assistance completion reports, impact or beneficiary assessments (if any) and other project documents; key informant interviews with ADB project officers, staff of executing and implementing agencies, and other stakeholders; focus group discussions with project beneficiaries. Review of technical assistance completion reports, and other project documents; key informant interviews with ADB project officers, staff of executing and implementing agencies, and other stakeholders. Review of technical assistance completion reports, and other project documents; key informant

8 8 Criteria Evaluation Questions Data Sources financial resources to maintain the project s outputs and outcomes? (iii) Are policy, legal and regulatory frameworks in place to sustain the reforms and achievements made by the TA projects? (iv) Do governments have sufficient fiscal resources that can be quickly mobilized for social protection response in times of shocks and risks? Impact (i) To what extent have the intended impacts of the TA projects been achieved? (ii) How did the TA projects strengthen social protection systems in responding to shocks and risks? (iii) To what extent did the projects help provide timely, adequate, and adaptable social protection support to help shock-affected communities? (iv) What innovative social protection interventions can be replicated or scaled up in other countries? interviews with ADB project officers, staff of executing and implementing agencies, and other stakeholders; focus group discussions with project beneficiaries. Review of technical assistance completion reports, and other project documents; key informant interviews with ADB project officers, staff of executing and implementing agencies, and other stakeholders; focus group discussions with project beneficiaries. ADB = Asian Development Bank, Design and monitoring framework = DMF, TA = technical assistance. Source: Independent Evaluation Department. V. Tentative Schedule and Resource Requirements 23. The evaluation will be carried out according to the following schedule: Milestone Target Date Desk research and preparation of approach paper Q Approval of approach paper Q Consultation with headquarters-based ADB staff Q Recruitment of consultants Q Evaluation mission Q Preparation of the draft TPER Q3 Q Internal comments and external peer review Q Interdepartmental review Q OSEC Editing Q Submission to IED Director General for approval Q An evaluation mission will be conducted for two weeks during the third quarter of Primary data will be collected during the mission through key informant interviews with staff in ADB s resident missions and staff of executing and implementing agencies, as well as discussions with project beneficiaries and other stakeholders. Hyun H. Son will lead the evaluation team composed of Ma. Juana Dimayuga, Jennifer Llaneta, an international expert, and a national consultant. The peer reviewer will be Joanne Asquith, Principal Evaluation Specialist at ADB. The terms of reference for the consultant to be recruited is provided in Appendix. 25. The consultancy cost estimate for the preparation of the evaluation study is in Supplementary Appendix (available on request).

9 9 VI. Dissemination of Findings 26. Upon approval of the report, the findings will be disseminated through IED s website and relevant knowledge sharing events within and outside ADB including a seminar with ADB s Social Development Thematic Group. Attachments: Appendix (public): 1) Technical Assistance Projects Supporting Social Protection Covered by the Evaluation 2) Intended Impacts, Outcomes, and Outputs, Lessons, and Recommendations of the Technical Assistance Projects 3) Terms of Reference of Consultants Supplementary Appendix (not public): A) Cost Estimates

10 10 Appendix 1 Technical Assistance Projects Supporting Social Protection Covered by the Evaluation TA Number Project Title Approved Amount Executing Agency Date Approved Date Closed a 4320 Social Protection of Poor Female Workers in the Garment Sector in the Context of Changing Trade Environments in Bangladesh 7101 Strengthening the Government s Institutional Capacity for Improving Food Security in Bangladesh 7300 Strategic Capacity Development for Social Sectors in Mongolia 7530 Supporting the Development of a Social Protection Framework for Nepal a = actual TA completion date TA = technical assistance Source: Asian Development Bank Database. (% disbursed) $420,000 (76.5%) $600,000 (70.1%) $700,000 (86.5%) $200,000 (70.6%) Ministry of Women and Children Affairs Ministry of Food and Disaster Management Ministry of Finance National Planning Commission Secretariat 16 March December July May June December May April 2011

11 Appendix 2 11 Intended Impacts, Outcomes, and Outputs, Lessons and Recommendations of the Technical Assistance Projects TA 4320: Social Protection of Poor Female Workers in the Garment Sector in the Context of Changing Trade Environments in Bangladesh TA 7101: Strengthening the Government's Institutional Capacity for Improving Food Security in Bangladesh TA 7300: Strategic Capacity Development for Social Sectors in Mongolia TA 7530: Supporting the Development of a Social Protection Framework for Nepal Impact Impact Impact Impact Reduced poverty of garment workers in a sustainable manner Improved capacity of the government for planning and undertaking mid- and long-term interventions to improve food security, including strengthening monitoring for increasing the efficiency of ongoing safety net programs Built firm foundations for change through strategic capacity development and a consensusbuilding program on the role of social assistance and the classification of recipients Reduced income inequality in Nepal Outcome Outcome Outcome Outcome Reduced risks and improved job opportunities for garment workers Improved monitoring system and better planning capability of the ministries of agriculture, and food and disaster management Ensured reform targets of the Social Sectors Support Program were met for the prioritization of the poor, and the provision of targeted social assistance Strengthened capacity of the government to plan and manage social protection schemes Output Output Output Output 1. Produce a strategic report on possible approaches for post- MFA human-capacity development, employment, and social protection measures for garment workers 1. Provide training on efficient food-stock management, use of information technology in increasing efficiency, and webbased monitoring systems and operations 1. Support policy reforms on targeting assistance and improving social sector efficiency 2. Strengthen statistical analysis and data collection 1. Develop a consolidated social protection framework 2. Enhance knowledge and skills of the participating agencies in social protection 2. Pilot women-friendly labor market programs to facilitate reemployment and social 2. Provide training on advance production planning, technology, 3. Build capacity for the management of an intersectoral national database of beneficiaries 3. Define organizational roles to implement the social protection framework

12 12 Appendix 2 TA 4320: Social Protection of Poor Female Workers in the Garment Sector in the Context of Changing Trade Environments in Bangladesh protection of retrenched RMG workers TA 7101: Strengthening the Government's Institutional Capacity for Improving Food Security in Bangladesh and digital communication in relation to food security TA 7300: Strategic Capacity Development for Social Sectors in Mongolia of social assistance and other government subsidies TA 7530: Supporting the Development of a Social Protection Framework for Nepal 4. Identify financial options for social protection 3. Pilot different types of social assistance that could be scaled up such as shelter support and micro and group insurance or saving, discretionary fund 4. Identify lessons learned and potential for scale up of programs in reemployment and social protection of female garment workers 3. Pilot operation of computerization of food-stock management 4. Develop consensus on targeting social assistance with national and local government officials and the public 5. Develop environment and social safeguard capacity for urban development stakeholders in Ulaanbaatar 5. Share lessons learned with the key stakeholders in a national workshop Overall Rating and Assessment Overall Rating and Assessment Overall Rating and Assessment Overall Rating and Assessment The TA is rated successful. All elements of the design and monitoring framework have been achieved. A strategy has been developed, pilot projects have been successfully implemented, a final workshop has been organized, and a final report with recommendations and findings has been elaborated. The TA is rated successful. Awareness raising, capacity development initiatives and training, and information technology hardware and software have laid good foundations for increasing efficiency in food-stock management and undertaking advanced crop production planning to ensure food security in the future. The TA is rated successful. It helped ensure that reform targets were met, which promoted efficiency and the provision of targeted social assistance. The TA contributed to the eventual passage of the amended Social Welfare Law in January 2012 and the government s adoption of the proxy means test as a targeting tool. It also helped advance propoor urban development in the Ulaanbaatar City Master Plan The TA is rated successful. It has facilitated, for the first time, a vibrant policy discourse on social protection in Nepal. This is expected to lead to important policy reforms that will contribute to more effective planning and execution of Nepal s social protection programs, which will contribute to more equitable poverty reduction.

13 Appendix 2 13 TA 4320: Social Protection of Poor Female Workers in the Garment Sector in the Context of Changing Trade Environments in Bangladesh Major Lessons TA 7101: Strengthening the Government's Institutional Capacity for Improving Food Security in Bangladesh Major Lessons TA 7300: Strategic Capacity Development for Social Sectors in Mongolia Major Lessons TA 7530: Supporting the Development of a Social Protection Framework for Nepal Major Lessons 1. There is a large unmet need for support and social protection among the workers in Bangladesh. 2. The involvement of NGOs and the target groups were crucial in ensuring the success of the project. Recommendations and Follow- Up Actions 1. Use of information technology knowledge and skills has high potential to improve efficiency, transparency, and governance. 2. Intensive facilitation and effective capacity development training can change staff attitudes. Recommendations and Follow- Up Actions 1. Creating the enabling environment to sustain reforms can be achieved by attaching TA to loans. Recommendations and Follow- Up Actions 1. One year is not sufficient to complete policy dialogue and reform processes in a sector where little work has been done previously. 2. TA design would have benefitted from an additional component to carry out some key evidence studies. This would have facilitated more informed decision-making by the sectoral ministries. Recommendations and Follow- Up Actions 1. Retain the lessons learned for future activities in other countries in the region. 1. The web-based system should be replicated nationwide. 2. The ministries of agriculture, and food and disaster management are urged to take measures as recommended by the specialists immediately to enhance efficiency of the respective departments. 3. Adequate operation and maintenance budget and qualified staff should be in place for effectively operating the systems. 1. The attached TA grant demonstrated that program loans need to be supported with a strong TA project to achieve intended policy reforms. Sources: Technical assistance reports, technical assistance completion reports, project completion reports, project validation reports 1. Government needs to generate more evidence on Nepal s ongoing cash transfers and other social protection programs in terms of their (i) impact on poverty, income inequality, and addressing social inclusion, and (ii) fiscal sustainability. 2. Support is also needed for institutional strengthening and modernization of registration, data management, payment methods, and monitoring and evaluation of social protection programs.

14 14 Supplementary Appendix A Terms of Reference of Consultants Background of Assignment 1. The Independent Evaluation Department (IED) assesses the performance of clusters of technical assistance (TA) projects to contribute to knowledge solutions and improve the operations within the Asian Development Bank (ADB). The findings of the TPER will help strengthen ADB s social protection support to foster inclusive growth and eradicate poverty in Asia and the Pacific. The TPER will contribute to improving ADB s efforts in helping developing member countries provide social protection in times of shocks and risks and strengthen their social protection programs to better prepare for shocks. The findings will also feed into the ongoing corporate evaluation on the design, use, and results of policy-based lending in ADB. A. International Consultant: Development Economist I. Scope of Work 2. The consultant s work will entail an assessment of the social protection measures supported in selected TA projects, with an emphasis on social protection support for mitigating the impacts of employment loss and food crisis. He/she should have knowledge of social protection programs, and familiarity with ADB s social protection strategy and operational plan. The engagement will be for 15 days. II. Detailed Tasks or Expected Outputs (i) (ii) (iii) (iv) (v) (vi) (vii) (viii) (ix) Assess the timeliness, adequacy and adaptability of the social protection measures in selected TA projects. Review the relevance and effectiveness of the social protection programs to the identified shocks and risks they were designed to address, particularly on loss of employment among women and food price hikes. Identify challenges in the efficient implementation and sustainability of ADB-supported social protection interventions. Participate in an evaluation mission to gather data and feedback on the selected TA projects. Prepare a background paper based on information gathered during the evaluation mission and a desk review of available information on the TA projects to assess the design, implementation and monitoring of the supported social protection schemes. Revise and finalize the background paper based on comments by IED and other peer reviewers. Assist the team in identifying good practices for effective social protection responses to various shocks and risks. Assist the team in addressing comments on and revising the draft final report accordingly after IED review and interdepartmental review stages; and Perform other related tasks requested by the Team Leader as deemed necessary.

15 Appendix 2 13 B. National Consultant: Research Associate I. Scope of Work 3. The consultant will assist in data analysis and research-related work on social protection. She/He must possess (i) knowledge of ADB s databases and information systems; and (ii) experience in synthesizing from various documentation sources. The engagement will be for 66 days. II. Detailed Tasks or Expected Outputs (i) (ii) (iii) (iv) (v) Collect, compile, and analyze relevant information and data related to the assigned evaluation for the drafting of the report. Conduct a literature review on social protection issues for the 3 countries (Bangladesh, Mongolia, and Nepal), and provide inputs to the background paper to be produced by the international consultant. Provide information for mission preparation on the selected TA projects including datasheets and facts on social protection. Assist in the quality control of the report by proof reading, checking the consistency of the main text and the appendixes, and ensuring the accuracy of the figures used. Assist in other related tasks that may be required by the Team Leader.