IWEco Project. Monitoring and Evaluation Plan and Budget. Appendix 07

Transcription

1 IWEco Project Monitoring and Evaluation Plan and Budget Appendix 07 COVER SHEET Name of Lead Partner Organizations: a) Caribbean Public Health Agency Environmental Health & Sustainable Development Department b) UNEP Caribbean Regional Coordinating Unit Contact person: a) Dr. C. James Hospedales, Executive Director, CARPHA; b) Nelson Andrade, Coordinator, UNEP Caribbean Regional Coordinating Unit; IWEco Project focus: Multi-focal Area International Waters, Biodiversity, Land Degradation, Sustainable Forest Management Duration of component: 60 months Amount of GEF grant: 685,474 USD ($ 535,473 USD (PM) & 150,000 USD (Evaluation costs)) Amount of Co-financing: 2,084,000USD Total funding: 2,769,473 USD 1

2 CONTENTS BACKGROUND MONITORING RESPONSIBILITIES AND EVENTS... 4 A. INCEPTION REPORT AND PROJECT SUPERVISION PLAN... 5 B. HALF-YEARLY PROGRESS REPORT, ANNUAL PROJECT REPORT AND PROJECT IMPLEMENTATION REVIEW (PIR) 6 C. PROJECT TERMINAL REPORT... 7 D. UNEP DEPI GEF IW SUPERVISION MISSION... ERROR! BOOKMARK NOT DEFINED. E. TECHNICAL REPORTS... 7 F. PROJECT PUBLICATIONS INDEPENDENT EVALUATION... 7 A. MID-TERM EVALUATION... 8 B. FINAL EVALUATION... 8 AUDIT CLAUSE LEARNING AND KNOWLEDGE SHARING KEY PERFORMANCE INDICATORS AND MONITORING PLAN SUMMARY OF INDICATIVE M&E FINANCING MONITORING AND EVALUATION SUMMARY BUDGET INDICATIVE MONITORING AND EVALUATION ACTIVITY LIST, RESPONSIBLE PARTIES AND CORRESPONDING BUDGET NONE* NONE*...ERROR! BOOKMARK NOT DEFINED. ANNEX A: TERMINAL EVALUTION TERMS OF REFERENCE PROJECT BACKGROUND AND OVERVIEW PROJECT OBJECTIVES AND ACTIVITIES RELEVANCE TO GEF PROGRAMMES EXECUTING ARRANGEMENTS BUDGET TERMS OF REFERENCE FOR THE EVALUATION OBJECTIVE AND SCOPE OF THE EVALUATION METHODS PROJECT RATINGS EVALUATION REPORT FORMAT AND REVIEW PROCEDURES SUBMISSION OF FINAL TERMINAL EVALUATION REPORTS RESOURCES AND SCHEDULE OF THE EVALUATION SCHEDULE OF PAYMENT ANNEX 1 TO APPENDIX 9: OVERALL RATINGS TABLE TOTALS GEF MINIMUM REQUIREMENTS FOR M&E MINIMUM REQUIREMENT 1: PROJECT DESIGN OF M&E LIST OF INTENDED ADDITIONAL RECIPIENTS FOR THE TERMINAL EVALUATION (TO BE COMPLETED BY THE IA TASK MANAGER) Supplemental Annex: Supervisory Plan 43 2

3 BACKGROUND The project will follow UNEP standard monitoring, reporting and evaluation processes and procedures. Substantive and financial project reporting requirements are summarized in this document. Reporting requirements and templates are an integral part of the UNEP legal instruments to be signed by the executing agencies and UNEP. The project M&E plan is consistent with the GEF Monitoring and Evaluation policy. The Project Results Framework presented in Appendix 4 of the ProDoc includes Specific, Measurable, Achievable, Relevant and Time-bound (SMART) for each expected outcome as well as mid-term and end-ofproject targets. These indicators, along with the key deliverables and benchmarks included in Appendix 6 will be the main tools for assessing project implementation progress and whether project results are being achieved. The means of verification and the costs associated with obtaining the information to track the indicators are summarized in Sections 4 and 5 of this appendix. Other M&E related costs are also presented in this Costed M&E Plan and are fully integrated in the overall project budget (see summary budget in section 7 below). As a result of the inception phase, a detailed M&E plan will be presented to the first meeting of the Project Steering Committee ( PSC) to ensure project stakeholders understand their roles and responsibilities vis-à-vis project monitoring and evaluation. The PSC will be responsible for proposing to UNEP management any necessary amendments to the M&E plan during project implementation. Indicators and their means of verification may also be fine-tuned by the PSC. Day-to-day project monitoring is the responsibility of the PCU but other project partners will have responsibilities to collect specific information to track the indicators. It is the responsibility of the Regional Project Coordinator (RPC) to inform UNEP of any delays or difficulties faced during implementation so that the appropriate support or corrective measures can be adopted in a timely fashion. The Project Steering Committee (PSC) will receive periodic reports on progress and will make recommendations to UNEP concerning the need to revise any aspects of the Results Framework or the M&E plan. Project oversight to ensure that the project meets UNEP and GEF policies and procedures is the responsibility to the Task Manager. The Task Manager will also review the quality of draft project outputs, provide feedback to the project partners, and establish peer review procedures to ensure adequate quality of scientific and technical outputs and publications. Project supervision will take an adaptive management approach. The Regional Project Coordinator (RPC) and the Task Manager will develop a detailed project supervision plan at the inception of the project that will be communicated to the project partners during the first meeting of the PSC. With the support of the UNEP Fund Management Officer backstopping this project, the Regional Project Coordinator will also be responsible for initial screening of the financial and administrative reports from the core partners prior to their submission to the Finance and Management Divisions of the United Nations Office at Nairobi. Progress vis-à-vis the delivery of agreed project outputs will be assessed by the PSC at least annually. Project risks and assumptions will be regularly reviewed both by project partners and the PCU on behalf of UNEP. Risk assessment and rating is an integral part of the annual Project Implementation Review (PIR), preparation of which will be the responsibility of the Regional Project Coordinator. The quality of project monitoring and evaluation will also be reviewed and rated as part of the PIR and the PSC shall clear the PIR prior to its final submission. Key financial parameters will be monitored quarterly to ensure cost-effective use of financial resources. 3

4 A Mid-term Review (MTR) or Mid -term Evaluation (MTE) will be organized by the UNEP Evaluation Office or the Task Manager in consultation with the Regional Project Coordinator and the outcomes reported to the Project Steering Committee. An independent terminal evaluation will take place at the end of project implementation and will be managed by the Evaluation Office of UNEP. A review of the quality of the evaluation report will be done by the Evaluation Office and submitted along with the report to the GEF Evaluation Office no later than 6 months after the completion of the evaluation. The standard terms of reference for the terminal evaluation are found in Annex A. These will be further adjusted to the special needs of the project. The GEF tracking tools are attached as Appendix 14 though to 30 of the ProDoc. These will be updated at mid-term and at the end of the project and will be made available to the GEF Secretariat along with the project PIR report. The mid-term review/evaluation and terminal evaluation will verify the information of the tracking tool. 1. Monitoring Responsibilities and Events Project oversight to ensure that the project meets UNEP and GEF policies and procedures is the responsibility to the Task Manager in UNEP-GEF. The Task Manager will also review the quality of draft project outputs, provide feedback to the project partners, and establish peer review procedures to ensure adequate quality of scientific and technical outputs and publications. At the first meeting of the PSC the Regional Project Coordinator (RPC) will present the detailed Monitoring and Evaluation Plan, as well as a full 60 month Project Supervision Plan and Gantt chart that will include: (i) tentative time frames for Project Steering Committee Meetings and meetings of the UNDP Regional Technical Advisor and (ii) p roject and sub-project related Monitoring and Evaluation activities and timelines. Day to day monitoring of the overall project implementation progress will be the responsibility of the Project Coordination Unit (PCU) based on the Project's Annual Work Plan and its indicators. The Regional Project Coordinator will inform UNEP of any delays or difficulties faced during implementation so that the appropriate support or corrective measures can be adopted in a timely and remedial fashion. The Regional Project Coordinator will fine-tune the progress and performance/impact indicators of the Project in consultation with the full Project Coordination Unit and with support from UNEP. The PCU will oversee monitoring of the entire project, while UNEP Car/RCU will focus on national sub-projects and CARPHA will focus on regional components of the project. The established indicators will be used to assess whether implementation is proceeding at the intended pace and in the right direction and will form part of the Annual Work Plan. Targets and indicators at the half-way point will be revised as part of the mid-term evaluation. All amendments and planning processes undertaken by the PCU will be subject to review and approval by the Project Steering Committee. Periodic monitoring of implementation progress will be undertaken by UNEP through the provision of quarterly reports, half-yearly reports submitted by each partner to the PCU. The Project Steering Committee will receive periodic reports on progress and will make recommendations to UNEP concerning the need to revise any aspects of the Results Framework or the M&E plan. Furthermore, specific meetings can be scheduled between the Project Team, UNEP, UNDP, CARPHA, Component Coordination Units, and other pertinent stakeholders as deemed appropriate and relevant (e.g. Steering Committee members, Co-funding partners, etc. Such meetings will allow parties to troubleshoot any problems pertaining to the Project in a timely fashion and to ensure smooth 4

5 implementation of project activities. A Report will be prepared by the PCU in coordination with UNEP Task Manager, UNDP RTA and CARPHA, and circulated within the PCU, all PSC members, CARPHA, UNDP, National Project Implementation Units (NPIU), and any accompanying stakeholders. Sub-project monitoring of implementation progress will be undertaken by National Project Implementation Units (NPIU) at the national level for Component 1 of the project, through the provision of quarterly reports. The reports are the responsibility of the National Project Coordinator at the national level, and will be developed with the close support of the PCU and RPC. For the remaining components relevant to regional level activities, they will be monitored and reported on by UNEP CAR/RCU, UNDP, and CARPHA and will integrate national level progress reports. Progress reports will be submitted to the PSC for review and reflection on challenges and opportunities. These reports will be the principle tools of regular project monitoring, form the basis for adaptive management. Each report will minimally contain: An account of implementation activities undertaken during the reporting period, and an update and assessment of progress against the implementation plan; Identification of (known and/or new) barriers to project implementation and corresponding recommendations for corrective actions during the following period (including revisions to the implementation plan, as necessary); A detailed and costed work plan for the subsequent reporting period, including status of funds held locally and, when necessary, a request for further cash transfers to the sub-project; An updated inventory of non-expendable equipment and items procured for the project; and Copies of project meeting reports or minutes and participant lists, as well as other technical outputs submitted to the project PCU. The monitoring information will then be compiled and submitted as part of the reporting for the project, and integrated in the periodic reporting by the executing agencies to the PCU. The collection of information at the national and regional levels will be part of the project s knowledge management efforts. Project Monitoring Reporting The Regional Project Coordinator in conjunction with the PCU team, UNEP Task manager, and UNDP RTA will be responsible for the preparation and submission of the following reports that form part of the monitoring process. Items (a) through (d) are mandatory and strictly related to monitoring, while (e) through (f) have a broader function and the frequency and nature is project specific to be defined throughout implementation. a. Inception Report and Project Supervision Plan A detailed M&E plan will be presented to the first meeting of the PSC and the RTAG to ensure project stakeholders understand their roles and responsibilities vis-à-vis project monitoring and evaluation. The PSC will be responsible for proposing to UNEP management any necessary amendments to the M&E plan during project implementation, including in relation to supervision arrangements. Indicators and their means of verification may also be fine-tuned by the PSC, supporting an adaptive management approach. Following the first PSC meeting and the RTAG meeting, an Inception Report (IR) will be presented to project partners and will include a more detailed narrative on the institutional roles, responsibilities, coordinating actions and feedback mechanisms of project related partners. In addition, a section will be included on progress to date in relation to project establishment and start-up activities, as well as an update of any changed external conditions that may affect project implementation such as unforeseen risks or newly arisen constraints. When finalized, the report will be circulated to project 5

6 counterparts who will be given a period of one calendar month in which to respond with comments or queries. Prior to this circulation of the IR, both UNEP and partner executing agencies will review the document. A Project Supervision Plan will also be communicated to the project partners at the inception. It will include a detailed First Year Work Plan divided in quarterly time-frames, detailing the activities and progress indicators that will guide implementation during the first year of the Project. This Work Plan will include the proposed dates for any visits and/or support missions from UNEP/UNDP, EAs or consultants, as well as time-frames for meetings of the PSC and Project Level Advisory Panels. The Report will also include the detailed project budget for the first full year of implementation, prepared on the basis of the Annual Work Plan, and including any monitoring and evaluation requirements to effectively measure project performance during the targeted 12 months time-frame. The Regional Project Coordinator will also be responsible for initial screening of the financial and administrative reports from the core partners prior to their submission to the Finance and Management Divisions of the United Nations Office at Nairobi. Progress vis-à-vis the delivery of agreed project outputs will be assessed by the PSC at least annually. Project risks and assumptions will be regularly reviewed both by project partners and the PCU on behalf of UNEP. Risk assessment and rating is an integral part of the annual Project Implementation Review (PIR), preparation of which will be the responsibility of the Regional Project Coordinator. The quality of project monitoring and evaluation will also be reviewed and rated as part of the PIR and the PSC shall clear the PIR prior to its final submission. Key financial parameters will be monitored quarterly to ensure cost-effective use of financial resources. b. Half-yearly Progress Report, Annual Project Report and Project Implementation Review (PIR) The Half-yearly Progress Report is a self-assessment report by project management to the UNEP Office and provides input to the reporting process as well as forming a key input to the Project Review undertaken by the Project Steering Committee. The PIR is an annual monitoring process mandated by the GEF, to be conducted by UNEP/UNDP and the Regional Project Coordinator in consultation with the partner executing agencies. It is an essential monitoring tool for IAs/EAs and the Regional Project Coordinator and offers the main vehicle for extracting lessons from on-going projects. An Annual Project Report is prepared on a yearly basis and its purpose is to reflect progress achieved in meeting the project's Annual Work Plan and assess performance of the project in contributing to intended outcomes through outputs and partnership work. Prior to submission, the Annual Project Report and Project Implementation Review (PIR) are discussed in the Project Steering Committee so that the resultant report accurately represents the project and is agreed upon by primary stakeholders. The sub-components will contribute to the above reports by providing national level assessment on outcome and impact indicators as outlined in the logical framework and GEF tracking tools. Monitoring will follow UNEP-GEF requirements, and the data will be submitted by the Component Coordination Units and reported to PSC and PCU. As per the current PIR template, the items in the PIR to be provided by the UNEP GEF Task Manager include the following: 6

7 An analysis of project performance over the reporting period, including outputs produced and, where possible, information on the status of outcomes; An analysis of any major constraints to achievement of results and the corresponding reasons for these constraints; An analysis of the annual Work Plans and related expenditure; An analysis of lessons learned; and Clear recommendations for future orientation in addressing key problems, especially in relation to noted key constraints. UNEP will analyse the Annual Project Reports and Project Implementation Reviews for results and lessons. The Reports are also valuable for the Independent Evaluators who can utilize them to identify any changes in project structure, indicators, workplan, etc. and view a past history of delivery and assessment. c. Project Terminal Report During the last three months of the project, the project team will prepare the Project Terminal Report. This comprehensive report will summarize all activities, achievements and outputs of the Project, lessons learnt, objectives (un)met, structures and systems implemented, etc. and it will be the definitive statement of the Project s activities during its lifetime. It will also outline recommendations for any further steps that may need to be taken to ensure sustainability and replicability of project activities and/or results. d. Technical Reports Technical Reports are detailed documents covering specific areas of analysis or scientific specializations within the overall project. As part of the Inception Report, the project team will prepare a draft Reports List, detailing the technical reports that are expected to be prepared on key areas of activity during the course of the Project, and tentative due dates. Where necessary this Reports List will be revised and updated, and included in subsequent Annual Project Reports. e. Project Publications Project Publications will form a key method of crystallizing and disseminating the results and achievements of the Project. These publications may be scientific or informational texts on the activities and achievements of the Project, in the form of journal articles, multimedia publications, etc. These publications can be based on Technical Reports, depending upon the relevance, scientific worth, etc., or they may be summaries or compilations of a series of Technical Reports and other analyses. The project team will determine if any of the Technical Reports merit formal publication, and will also, in consultation with UNEP, the partner executing agencies and other relevant stakeholder groups, plan and produce these publications in a consistent and recognizable format. Any publications need prior clearance from UNEP and the partner executing agencies. Project resources will need to be defined and allocated for these activities as appropriate and in a manner commensurate with the project's budget. 2. Independent Evaluation The project will be subject to two external evaluations as follows:- 7

8 a. Mid-Term Evaluation A Mid-term Review (MTR) or Mid -term Evaluation (MTE) will take place half -way after project implementation commences (30 months after project start). The purpose of the Mid-Term Review (MTR) or Mid-Term Evaluation (MTE) is to provide an independent assessment of project performance at mid-term, to analyze whether the project is on track, what problems and challenges the project is encountering, and which corrective actions are required so that the project can achieve its intended outcomes by project completion in the most efficient and sustainable way. In addition, it will verify information gathered through the GEF tracking tools. The review will be carried out using a participatory approach whereby parties that may benefit or be affected by the project will be consulted. Such parties were identified during the stakeholder analysis (see section 2.5 of the project document).the project Steering Committee will participate in the MTR or MTE and develop a management response to the evaluation recommendations along with an implementation plan. It is the responsibility of the UNEP Task Manager to monitor whether the agreed recommendations are being implemented. An MTR would be managed by the UNEP Task Manager at DEPI, and an MTE would be managed by the Evaluation Office of UNEP. The Evaluation Office will determine whether an MTE is required or whether an MTR is sufficient. b. Final Evaluation An independent Terminal Evaluation (TE) will take place at the end of project implementation. The Evaluation Office of UNEP will be responsible for the TE and liaise with the UNEP Task Manager at DEPI throughout the process. The TE will provide an independent assessment of project performance (in terms of relevance, effectiveness and efficiency), and determine the likelihood of impact and sustainability. It will have two primary purposes: (i) to provide evidence of results to meet accountability requirements, and (ii) to promote learning, feedback, and knowledge sharing through results and lessons learned among UNEP, UNDP and core partners ( Secretariat to the Cartagena Convention, UNEP Caribbean Regional Coordinating Unit (Car/RCU) and the Environmental Health and Sustainable Development (EHSD) Department of CARPHA). The direct costs of the evaluation will be charged against the project evaluation budget. The TE report will be sent to project stakeholders for comments. Formal comments on the report will be shared by the Evaluation Office in an open and transparent manner. The project performance will be assessed against standard evaluation criteria using a six point rating scheme. The final determination of project ratings will be made by the Evaluation Office when the report is finalized. The evaluation report will be publicly disclosed and will be followed by a recommendation compliance process. Audit Clause The partner executing agencies and national partners will provide UNEP with quarterly financial reports as well as certified annual financial statements, with an audit of the financial statements relating to the status of UNEP (including GEF) funds according to the established procedures to be set out in the project document for each Project Coordinating Agreement. The Audit will be conducted by the legally recognized auditor, or by a commercial auditor. 3. Learning and Knowledge Sharing 8

9 Results from the project will be disseminated within and beyond the project intervention zone through a number of existing information sharing networks including IW:LEARN and other fora. In addition: the project will participate, as relevant and appropriate, in UNEP/GEF sponsored networks, organized for Senior Personnel working on projects that share common characteristics; and the project will identify and participate, as relevant and appropriate, in scientific, policybased and/or any other networks, which may be of benefit to project implementation though lessons learned. The project will identify, analyze, and share lessons learned that might be beneficial in the design and implementation of similar future projects. Identifying and analyzing lessons learned is an ongoing process, and the need to gather and communicate knowledge is one of the project's central contributions. Lessons learned will be monitored for on a quarterly basis and shared at least once every 12 months. UNEP shall provide a format and assist the project team in categorizing, documenting and reporting on lessons learned. Objectively verifiable indicators as shown in the logical framework will be utilized in all evaluations. A biennial IWEco Partnership Forum will be convened after the first two years of project implementation followed by another towards the final year of the project. The Partnership Forum will be formulated like a conference to be attended by both representatives of the PSC and the full RTAG, along with donors, private sector collaborators and other regional and international governmental and non-governmental agencies. The main objective will be to review the contributions of the project, uptake the lessons learned, knowledge gained and seek out opportunities for replication, forge new collaborations and initiatives and consider policy and implementation mainstreaming at the SIDS regional and global levels. 4. Key Performance Indicators and Monitoring Plan The table below present the key performance indicators for each component and their outcomes, along with a baseline, mid-term and project end targets, and sources of verification. A full version, including indicators for outputs is found in Appendix 4 (Results Framework) and Appendix 6 (Key deliverables and benchmarks). Project Goal: to contribute to improved quality of the water, land and biological resources in the Caribbean through application of appropriate solutions that maintains the flow of ecosystem benefits and their contribution to long-term socioeconomic development in the Caribbean Indicators Baseline Mid-term & project end targets Sources of verification Objective: Implementation of an integrated approach to water, land and ecosystems services management, supported by policy, institutional and legislative reforms, and implementation of effective appropriate technologies to accelerate contribution to global targets on access to safe and reliable water supplies and improved sanitation, and contributing to improved ecosystem functioning in the Caribbean Suite of installed innovative solutions for improved water, land and biodiversity resources management Installed capacity to measure change in environmental and related socio-economic status indicators Mainstreamed policies and upgraded regulatory and fiscal incentive instruments for sustainable resources Fragmented and marginally effective interventions in addressing negative socioeconomic and ecosystem impacts associated with water, land and biodiversity degradation due to relatively weak institutional and regulatory environments. Effective, cost-effective onground technical and policy solutions that are widely disseminated through knowledge networks and replicated on-ground in Caribbean SIDS and global SIDS by end of project Project reports; websites and public awareness resources; scientific publications; country state of environment reports; convention/treaty/mea agreements 9

10 Project Goal: to contribute to improved quality of the water, land and biological resources in the Caribbean through application of appropriate solutions that maintains the flow of ecosystem benefits and their contribution to long-term socioeconomic development in the Caribbean Indicators Baseline Mid-term & project end targets Sources of verification management Trained stakeholders with built capacity Component 1 Objective: To develop and foster the implementation of targeted Innovative, climate-change resilient approaches to Sustainable Land Management (SLM), Integrated Water Resources Management (IWRM) (including Water Use Efficiency (WUE)), Integrated Coastal Zone management (ICZM) and enhanced management and maintenance of ecosystem services Outcome C1.1. Measurable stress reduction at project sites through appropriate sustainable water, land and ecosystems management interventions that account for climate change. Deepened analytical understanding of ecosystem quality, ecosystems services in target areas Biodiversity conservation ((index; ha; % change) Improved ambient soil and water quality including from reduction of pollution Direct and indirect emissions reductions or avoidances (ha; tonnes CO2eq) Area of land under improved management practices that also integrate climate change Limited use of targeted interventions to prove measurable benefits across the main focal areas; Critical areas are not being directly addressed by current interventions 8 Feasibility studies and other analysis providing baseline data and/or options for further actions (mid-term) 30-40% Improvement in species richness (30-40%, end of project), at least 2,735 ha landscape directly covered by project end (1,085 ha at midterm and 2,735 ha at projectend), forest area protected by project end, Restored habitat (Wetlands 200 ha, mid-term; 415 ha, project-end ; Forests under direct interventions: 1,510 ha mid-term and 2,320 ha project-end) Water quality, sediment and biomass index reduction over baseline in targeted sites and coastal zones 100% Water quality area protected by project end (50%, mid-term) (100%, project-end) sediment reduction (50%, midterm) (100%, project-end) 180,327 tco2 sequestered carbon through restoration and reforestation and 408,103 tco2 in avoided carbon emissions through in-situ conservation and sustainable forest management (estimate to be validated at inception) Forest Area (ha) under direct interventions with improved management practices (1,085 ha, mid-term) (2,735 ha, project-end); in-situ conservation over wider/adjacent 46,000 ha by project end) internal agency reports such as from respective country management unit(s) national reporting to conventions and other international and regional frameworks Outcome C1.2. Enhanced livelihood opportunities and socio-economic co-benefits for targeted communities from improved ecosystem services functioning. Communities and businesses Community-based Implementation of small Agency-community MOUs 10

11 Project Goal: to contribute to improved quality of the water, land and biological resources in the Caribbean through application of appropriate solutions that maintains the flow of ecosystem benefits and their contribution to long-term socioeconomic development in the Caribbean Indicators Baseline Mid-term & project end targets Sources of verification are engaged in activities that bring socio-economic and livelihoods benefits from land, water and ecosystems management Number of people with alternative livelihoods opportunities of targeted communities, especially the least advantaged groups Average annual income of environmental services, especially the least advantaged Number of persons, communities, with improved health, well-being Ecosystem services provision investments in natural resources management remains relatively low -keyed and few in number in consideration of the potential that exists; limited capacity within community groups to tap into resources Limited understanding of how investments in land, water and ecosystems resources management contributes to improved economies and sustainable livelihoods Relatively low engagement of community groups and CSOs in active natural resources management grants community-based projects integrated into national initiatives (5 or more communities, 2017) Number of people provided alternative livelihoods (in 5 communities, 2017) At least 20 % increase in annual income per capita from crop and livestock, and from forest and tree products (in 5 communities, 2017) Health indices of target communities (20% fewer reports of water-related illnesses, 2017) % species richness improvement and/or co-management arrangements; project documentation; community surveys (including beneficiary economic analysis) SGP implementation reviews; National grantee reporting; National planning and development reporting; National reporting to conventions and other international and regional frameworks Component 2 Objective: To strengthen the water, land and ecosystems resources Monitoring, and Indicators frameworks Outcome C2.1. Strengthened national and regional systems for monitoring of environmental status with respect to key international agreements. Strengthened indicators framework adopted at regional level becoming mainstreamed and utilized in national socio-economic, planning and development and environmental status assessments; State and non-state stakeholders demonstrate competency in application of indicators to enhance decision making Indicators are not applied in mainstream decision making low level of awareness amongst decision-makers of utility of environmental indicators observation platforms and decision support systems (DSS) for monitoring are generally weak Weak capacities in relevant state and non-state agencies for monitoring indicators MT - Caribbean countries endorsed suite of indicators by end of Year 4 technical personnel applying accepted methods and techniques in making IWRM, SLM, BD and SFM assessments that support decision-making by Year 4 community-based organizations, schools and other NGO groups are engaged in supportive assessments at the local community level by Year 4 state of environment reports at national and regional levels UN convention and other international and regional reporting outputs strengthened observation platforms strengthened training/capacity building programmes project reports Component 3 Objective: To strengthen policy, legislative and institutional reforms and capacity building to support Sustainable Land Management (SLM), Integrating Water Resources Management (IWRM)/Water Use Efficiency (WUE) and ecosystem services management taking into consideration climate change resilience building. Outcome C3.1. Strengthened policy and legislation for the effective management of water, land and ecosystems resources that account for climate change Improved compliance based on strong and effective policy and legislative instruments Demonstrated integration of water, land and ecosystems management in mainstream socioeconomic development National socio-economic development do not adequately factor in environmental management policy environmental policy is weakly articulated at both national and regional levels absence of regional-level environmental policy framework out-of-date legislation regulations not sufficiently developed to make laws countries commence process to policy and legislative reforms by mid-year 3 national development planning strategies and decision making frameworks broadening incorporation of elements of water, land and ecosystems resources management by mid-year 3 cooperation amongst regional support agencies strengthened through establishment of joint gazetted legislative amendments new policy statements reporting to UN conventions and other regional and international frameworks published policy statements publication and dissemination and access to information regional cooperation frameworks in effect; 11

12 Project Goal: to contribute to improved quality of the water, land and biological resources in the Caribbean through application of appropriate solutions that maintains the flow of ecosystem benefits and their contribution to long-term socioeconomic development in the Caribbean Indicators Baseline Mid-term & project end targets Sources of verification effective international and regional treaties not integrated into national law limited awareness and buy-in from stakeholders incentive measures to accompany legislative provisions weak approaches are more command-and-control rather than participatory cooperation agreements by end of Year 4 Outcome C3.2. Strengthened capacity of national and regional institutions and other stakeholders for water, land, and ecosystems management that accounts for climate change Decision making improved decision making at policy level through enhanced supported by improved coordination amongst stakeholder engagement relevant national improved level of technical support from national agencies stakeholders to stakeholder interests for institutional response from environmental management national state and non-state improved planning and agencies and regional coordination emerging agencies effective in between regional support addressing implementation organizations of water, land and stakeholders demonstrating ecosystems management empowerment through enhanced capacity through provision of skills and information Poor decision outcomes from fragmented institutional responses Limited engagement of stakeholders due to weakly constituted consultative processes on environmental management agency effectiveness to implement water, land and ecosystems management compromised by limited human resource and financial capacity low awareness at policy making level to gain support for investment in institutional strengthening poor coordination amongst regional agencies in harmonizing approaches and advisory support to countries meeting records and inputs to policy processes from intersectional committee meetings regional inter-agency cooperation frameworks in effect Component 4 Objective: To enhance knowledge exchange amongst practitioners to improve integrated and effective water, land and ecosystems resource management through the promotion of best-practices, replication of lessons learnt and experiences gained Outcome C4.1. Improved engagement and information access for practitioners and other stakeholders through targeted knowledge sharing networks Enhanced stakeholder networking and knowledge sharing towards implementation of solutions across the Caribbean and other SIDS regions Expanded, strengthened community of practices with shared experiences in successfully implementing solutions relative isolation of practitioners with limited opportunities for interactive experience-based learning limited dissemination and access to resources to practitioners in appropriate formats many web-based platforms but translation of knowledge to implementation is not well understood; translation of expert and traditional knowledge to application at community level is limited evidence of stakeholders applying knowledge, tools and methods generated by project technical support agencies providing enhanced support in sharing information to improve design making; strengthened linkages amongst practitioners in various fields "community of practices" across SIDS regions project stakeholders and other resource users accessing project website project contribute to other information clearinghouse Formal and informal communities of practices and associated knowledge platforms functional; reporting to UN conventions and other regional and international frameworks project on-line resources frequented by user community evidenced by interface diagnostics 12

13 Project Goal: to contribute to improved quality of the water, land and biological resources in the Caribbean through application of appropriate solutions that maintains the flow of ecosystem benefits and their contribution to long-term socioeconomic development in the Caribbean Indicators Baseline Mid-term & project end targets Sources of verification mechanisms 13

14 5. Summary of Indicative M&E Financing M&E financing will come from Components 5 and 6; Project Management and Evaluation respectively. Cofinancing for M&E of the GEF IWEco project is $2,084,000. This amount represents funds dispersed on IWEco baseline activities, data and information gathering in support of the various reports and funds expected to be dispersed on M&E related activities during the course of the project. Component 5: Project Management GEF Funding (US$) Co- Financing (US$) Total component Cost (US$) Sub-component 5.1: Day-to-day Project Management (through the PCU including Regional Project Coordinator, a Technical Assistant, a Financial and Administrative Assistants) Activity 5.1.1: Networking and communication on substantive matters with partner agencies 36, , ,541 Activity 5.1.2: Preparation and processing of contracts and agreements 25, , ,541 Activity 5.1.3: PSC and RTAG meetings Sub-activity : Travel - PCU participation and support to PSC 129, , ,541 Sub-activity : PSC and RTAG meetings (virtual, as part of PSC, or called upon by Components) 320, , ,014 Activity 5.1.4: Periodical financial and substantive Reporting Sub-activity : Reporting to UNEP, UNDP and GEF (inception report, QERs, QORs, PIRs & Mid-term Management Review, etc.) 25, , ,541 TOTAL component COST ($) 535,473 1,927,703 2,463,176 Component 6: Project Mid-term and Terminal Evaluation Sub-component 6.1: Project Mid-term and Terminal evaluations GEF Funding (US$) Co- Financing (US$) Total component Cost (US$) Activity Conduct Mid-term evaluation (MTE) 50,000 52, ,100 Activity Conduct terminal evaluation (TE) 100, , ,197 TOTAL component COST ($) 150, , ,297 14

15 6. Monitoring and Evaluation summary budget The budget for monitoring and evaluation of the project is summarized in the table below and comprises the costs allocated to M&E for each component in the project budget. The budget is derived from Components 5 and 6. Budget Item Total GEF Funding Monitoring and Evaluation Year 1 Year 2 Year 3 Year 4 Year 5 Cofinancing/ Counterpart Networking and communication on 36,000 7,200 7,200 7,200 7,200 7, , ,541 substantive matters with partner agencies Preparation and processing of 25,000 5,000 5,000 5,000 5,000 5, , ,541 contracts and agreements Consultant (M&E: Midterm and 150,000 50, , , ,297 Terminal Evaluations) Sub-contract (Periodic Status 25,000 5,000 5,000 5,000 5,000 5, , ,541 Reports from interventions and national input reviews (PIRs) ) PSC/RTAG Meetings and Inception 320,473 64,095 64,095 64,095 64,095 64, , ,014 Workshop Travel 129,000 25,800 25,800 25,800 25,800 25, , ,541 Total 685, , , , , ,095 2,084,000 2,769,473 Total 7. Indicative Monitoring and Evaluation activity list, responsible parties and corresponding budget Type of M&E activity Monitoring Project Steering Committee Meetings and Inception workshop Responsible Parties GEF Budget US$ Time frame Regional Project Coordinator (PCU) UNEP Task Manager UNDP RTA CARPHA UNEP CAR/RCU Component Coordination Unit 1 st PSC Meeting will be convened as an inception workshop. USD 110,000 (built in Comp. 5) Held within first four months of project start up. Networking and management of partnership agreements and incountry support Regional Project Coordinator (PCU) UNEP Task Manager UNDP RTA CARPHA UNEP CAR/RCU Component Coordination Unit USD 36,000 for partner agency networking and communication USD 25,000 preparation and processing/management of contracts and agreements USD 129,000 to support travel to countries (all built into Comp. 5) Over course of project Meetings of the RTAG Regional Project Coordinator (PCU) UNEP Task Manager UNDP RTA CARPHA UNEP CAR/RCU The 1 st RTAG Meeting will be in conjunction with the first PSC Meeting as an Inception workshop. `Other RTAG meetings will be held once/year Held within first four months of project start up; at least one meeting per annum 15

16 Type of M&E activity Inception Report and Project Regional Project Coordinator (PCU) Supervision Plan UNEP Task Manager UNDP RTA CARPHA UNEP CAR/RCU APR and PIR Regional Project Coordinator Lessons Learned (PCU) UNEP Task Manager CARPHA UNEP CAR/RCU Terminal Report Project team (PCU)* UNEP Task Manager CARPHA UNEP CAR/RCU Component Coordination Units External Consultant* Evaluation Mid-Term Review UNEP EOU/UNDP EO Regional Project Coordinator (PCU) UNEP Task Manager UNDP RTA CARPHA Regional Coordination Units Participating Institutions and stakeholders External consultant Final External UNEP EOU/UNDP EO PO Evaluation Regional Project Coordinator (PCU) UNEP Task Manager CARPHA Regional Coordinating Units Participating Institutions and stakeholders External Consultants (i.e. evaluation team) Audit UNEP Task Manager Regional Project Coordinator (PCU) * External Auditor TOTAL indicative COST Responsible Parties GEF Budget US$ Time frame USD 210,475 (built in Comp. 5) None* USD 25,000 None* USD 50,000 GEF resources in Component 6 and USD 52,100 as cofinancing from CAR/RCU and CARPHA USD 100,000 GEF resources in Component 6 and USD 104,197 as co-financing from CAR/RCU and CARPHA None * USD 685,473 in GEF resources (USD 535,473 from Comp 5 and USD 150,000 from Comp 6) (Total including CF is USD 2,769,473, of which USD 2,084,000 is CF) Immediately following inception workshop Annually At least one month before the end of the project Halfway through project cycle at around 30 months. At the end of project implementation Yearly 16

17 ANNEX A: TERMINAL EVALUTION TERMS OF REFERENCE TERMINAL EVALUATION TERMS OF REFERENCE (Based on the EOU Standard Format) Terminal Evaluation of the UNEP GEF project entitled: Integrating Water, Land and Ecosystems Management in Caribbean Small Island Developing States PROJECT BACKGROUND AND OVERVIEW The challenges Caribbean SIDS face on account of small landmasses, vulnerable economies, heavy dependence on external energy resources, with rising populations means that concerted efforts must be made in translating the concept of sustainable development to reality in practice. It is accepted that climate change will affect SIDS more directly that other regions of the globe in terms of scope and intensity of impact, hence governments must continue to hasten the process to safeguard the natural resource base so as to make communities and ecosystems resilient to the impacts of climate change. In spite of progress made on many fronts, many barriers remain in respect of water, land and biodiversity resource management. The Integrating Water, Land and Ecosystems Management in Caribbean Small Island Developing States (GEF -IWEco Project) is a five-year multi-focal area regional project that financed from (i) country GEF STAR allocations under Land Degradation and Biodiversity portfolios for national interventions and supported by (ii) funding from the GEF International Waters and Sustainable Forest Management portfolios for regional-level actions. The IWEco Project is a continuation of the work of prior initiatives, and most directly the GEF-IWCAM Project is addressing removal of these barriers. The aim of the Project is to lay the basis for continued investment in environmental resource management in Caribbean SIDS that is anchored in tried and tested innovative approaches. A total of ten Caribbean countries participated in the project. The project s overall objective is to contribute to maintenance of the integrity of Caribbean ecosystems that are of global significance and the sustainability of livelihoods through the application of existing proven technologies and approaches that are appropriate for small island developing states through improved fresh and coastal water resources management, sustainable land management and sustainable forest management that also seek to enhance resilience to the impacts of climate change. Project Objectives and Activities The project has four components; (1) Development and Implementation of Integrated Targeted Innovative, climate-change resilient approaches in SLM, IWRM (including WUE), ICZM and maintenance of ecosystem services; (2) Strengthening of the SLM, IWRM (and WUE) and ecosystems Monitoring, and Indicators framework; (3) Strengthening of the Policy, legislative and institutional reforms and capacity building for SLM, IWRM/WUE and ecosystem services management taking into consideration climate change resilience building and (4) Enhancing knowledge exchange, bestpractices, replication and stakeholder involvement. 17

18 Global environment objective: to apply appropriate methodologies and approaches for the sustainable management of water, land and biodiversity resources through replication and upscaling of innovative solutions to mitigate further degradation of these resources that will have practical application across small island developing states across the globe. Relevance to GEF Programmes The project is consistent with GEF-5 International Waters Focal Area Strategy and responds to Strategic Priorities 1,2, and 4 of the International Waters Strategy, as well as the Strategic Goals of the GEF-5 Programming Document, by undertaking a global assessment of transboundary water bodies, through a formalised consortium of partners, to support informed investments by the GEF and other international organizations, and to be sustained through a periodic process in partnership with key institutions, aimed at incorporating transboundary considerations into regular assessment programmes. The project is consistent with the GEF Focal Area Strategies for International Waters (IW), Land Degradation (LD) and Biodiversity (BD) including Sustainable Forest Management (SFM) with each focal area actions complementing one another as to promote a truly integrated approach to the management of natural resources. With regards to the GEF International Waters (IW) Strategy, the project is consistent with Strategic Objective 1 by seeking to enhance the management of watershed areas, groundwater aquifers, and coastal and marine ecosystems within the Caribbean Sea basin. Also, in terms of the GEF Land Degradation (LD) Strategy the project will address Strategic Objective 3 by contributing towards arresting and reversing current trends in land degradation which in the Caribbean, is aggravated by deforestation and unsustainable land management particularly in the more mountainous areas and other landscapes with fragile soils that are vulnerable to degradation. The project is consistent with the GEF Biodiversity (BD) Strategy, Strategic Objective 2 by promoting the conservation and sustainable use of biodiversity and the maintenance of ecosystem goods and services through the improved management of ecologically sensitive areas of interest (particularly those outside protected area systems) towards long-term positive impacts in representation of terrestrial and marine ecosystems, and threatened species. And finally, the GEF Sustainable Forest Management / REDD+ Strategy, Strategic Objective 1 by enhancing the enabling environment within the forest sector and across sectors, foster application of good management practices in existing forests with carbon sequestration benefits and promote the adoption of good management practices by relevant economic actors. Executing Arrangements The project is implemented through a network of international, regional and national partners along thematic lines, in accordance with capacity advantage based on institutional mandates of these partners. The Implementing Agencies are UNEP (lead) and UNDP. The project is co-executed by the Secretariat to the Cartagena Convention, UNEP Caribbean Regional Coordinating Unit (Car/RCU) and the Environmental Health and Sustainable Development (EHSD) Department of CARPHA, with the Secretariat assuming the role of lead Executing Agency. While at the national level, each participating country has designated a National Project Focal Point (NPFP) for the project and will further foster the establishment as relevant a National Intersectoral Committee (NIC). 18

19 Regional co-ordination and collaboration is facilitated through a Regional Project Co-ordination Unit (PCU), consisting of professional and support staff that provide technical assistance and advice to the participating countries. The Regional Project Steering Committee (PSC) meet annually to monitor progress in project execution, to provide strategic and policy guidance, and to review and approve annual work plans and budgets. The Committee is chaired by a national representative (on a rotational basis) and consists of the national project focal points (NPFPs) from all participating countries, representatives of the two GEF Implementing Agencies, the two co-executing Agencies comprising the core group, along with observers from the CARICOM Secretariat, the OECS Secretariat and other project partners. For each project country a there is a National Project Steering Committee (NPSC) comprised of the direct technical contributors and project beneficiaries, representing both state and non-state organizations and is administered by a National Regional Project Coordinator (NRPC). There is also a Regional Technical Advisory Group (RTAG) that advises the Steering Committee and the PCU on project technical issues at the local project level and at the regional level. Each country has a technical representative to the RTAG, along with designates from the IAs and EAs will from the core group. The lead national agencies in each country are as follows: Antigua & Barbuda: Ministry of Agriculture, Lands, Housing and the Environment Barbados: Ministry of Environment, water Resources and Drainage Cuba: Ministry of Science, Technology and the Environment (CITMA), the Ministry of Planning and the Ministry of Agriculture 1 Dominican Republic: Ministry of Environment and Natural Resources Grenada: Land Use Division and Forestry Department of the Ministry of Agriculture Jamaica: National Environment and Planning Agency (NEPA), supported by Forestry Department for biodiversity and Water Resources Agency for water resources management Saint Kitts & Nevis: Ministry of Environment and Ministry of Agriculture on Saint Kitts; Nevis Island Administration. Saint Lucia: The Forestry Department and Water Resources Management Agency from the Ministry of Agriculture Saint Vincent & the Grenadines: Ministry of Agriculture and the Forestry Department Trinidad & Tobago: Ministry of the Environment and Water Resources Budget The total project funding at the time of CEO clearance and signature by UNEP was US$ 94,644,334 million, with US$ 20,718,598 million funded by the GEF Trust Fund, US$ 3.0 million is funded from UNEP, US$ 1.0 million from UNDP, and US$ 69,925,736 million from other sources in-kind. The project also received US$125,000 GEF project preparation funds. 1 Ministerio de Ciencia, Tecnología y Medio Ambiente (CITMA) y Centro de Estudios Ambientales de Cienfuegos (CEAC) 19

20 APPENDIX 8 TERMS OF REFERENCE FOR THE EVALUATION 1. Objective and Scope of the Evaluation The objective of this terminal evaluation is to examine the extent and magnitude of any project impacts to date and determine the likelihood of future impacts. The evaluation will also assess project performance and the implementation of planned project activities and planned outputs against actual results. The evaluation will focus on the following main questions: 1. Primary effects: Did the project achieve the intended outcomes under each of the respective project components 1 through 4? In doing so, did the interventions raise the level of awareness and capacity for target groups to implement integrated approaches to water, land and ecosystems services management (international conventions and initiatives, national level policy-makers, regional and local policy-makers, resource managers and practitioners)? What is the likelihood of sustaining the project activities and/or their corresponding results beyond implementation? How has the project affected or influenced local economies or private sector actors? Is there evidence of socio-economic and livelihoods benefits? 2. Secondary effects: Was the project influential to policy makers and other key audiences, decision-makers and/or institutions as a result of the scientific authority and credibility of outputs produced? Did the outputs of the project articulate options and recommendations for priority actions and future interventions to the targeted audiences? How these options and recommendations were disbursed and is there any evidence of their uptake and use? If so by whom, and at what levels (local, national, regional international)? To what degree did the project support new or strengthened partnerships or networks formed around the issues of water and land management, and ecosystems services? 3. Management and Oversight: How did the project incorporate and use principles of results-based management? In what ways did the managers use information gathered during implementation to adjust the approach or strategies? To make decisions, foster learning among project partners, and/or ensure accountability? To what extent were stakeholders and partners involved in setting up and/or using monitoring and reporting systems? If so, how did this affect them or influence them? Were management and M&E systems set up and/or managed to enable efficiency and/or cost-effectiveness? What mechanisms were in place for coordination and communication? 2. Methods This terminal evaluation will be conducted as an in-depth evaluation using a participatory approach whereby the UNEP Task Manager, key representatives of the executing agencies and other relevant staff are kept informed and consulted throughout the evaluation. The consultant will liaise with the UNEP/EOU and the UNEP Task Manager on any logistic and/or methodological issues to properly conduct the review in as independent a way as possible, given the circumstances and resources offered. The draft report will be circulated to UNEP Task Manager, key representatives of the executing agencies and the UNEP/EOU. Any comments or responses to the draft report will be sent to UNEP/EOU for collation and the consultant will be advised of any necessary or suggested revisions. The findings of the evaluation will be based on the following: 20

21 1. A desk review of project documents including, but not limited to: (a) The project documents, outputs, monitoring reports (such as progress and financial reports to UNEP and GEF annual Project Implementation Review reports) and relevant correspondence. (b) Reports from the Steering Committee and Scientific and Technical Advisory Committee meetings. (c) Other project-related material produced by the project staff or partners. (d) Relevant material published on the project web-site. 2. In-field and teleconference interviews with project management and technical support including staff of the Project Coordinating Unit; project focal points and staff of the coordinating Units in each of the lead national agencies for each component and other partners as deemed necessary. 3. Interviews and Telephone interviews with intended users for the project outputs and other stakeholders involved with this project, including in the participating countries and international bodies. The Consultant shall determine whether to seek additional information and opinions from representatives of donor agencies and other organizations. As appropriate, these interviews could be combined with an questionnaire. 4. Interviews with the UNEP Task Manager and Fund Management Officer, and other relevant staff in UNEP dealing with all relevant Focal Areas (IW, BD, SFM, LD and SGP) related activities as necessary. The Consultant shall also gain broader perspectives from discussions with relevant GEF Secretariat staff. Key Evaluation principles. In attempting to evaluate any outcomes and impacts that the project may have achieved, evaluators should remember that the project s performance should be assessed by considering the difference between the answers to two simple questions what happened? and what would have happened anyway?. These questions imply that there should be consideration of the baseline conditions and trends in relation to the intended project outcomes and impacts. In addition it implies that there should be plausible evidence to attribute such outcomes and impacts to the actions of the project. Sometimes, adequate information on baseline conditions and trends is lacking. In such cases this should be clearly highlighted by the evaluator, along with any simplifying assumptions that were taken to enable the evaluator to make informed judgements about project performance. 3. Project Ratings The success of project implementation will be rated on a scale from highly unsatisfactory to highly satisfactory. In particular the evaluation shall assess and rate the project with respect to the eleven categories defined below: 2 A. Attainment of objectives and planned results: The evaluation should assess the extent to which the project's major relevant objectives were effectively and efficiently achieved or are expected to be achieved and their relevance. 2 However, the views and comments expressed by the evaluator need not be restricted to these items. 21

22 Effectiveness: Evaluate how, and to what extent, the stated project objectives have been met, taking into account the achievement indicators. The analysis of outcomes achieved should include, inter alia, an assessment of the extent to which the project has directly or indirectly assisted policy and decision-makers to apply information supplied by focal area indicators (i.e. international waters, biodiversity, climate change, land degradation, sustainable forest management) in their national planning and decision-making. In particular: Evaluate the immediate impact of the project on all relevant Focal Areas monitoring and in regional and national planning and decision-making and international understanding and use of transboundary waters indicators As far as possible, also assess the potential longer-term impacts considering that the evaluation is taking place upon completion of the project and that longer term impact is expected to be seen in a few years time. Frame recommendations to enhance future project impact in this context. Which will be the major channels for longer term impact from the project at the national and international scales? Relevance: In retrospect, were the project s outcomes consistent with the focal areas/operational program strategies? Ascertain the nature and significance of the contribution of the project outcomes to the Cartagena Convention (the LBS and SPAW Protocols), the St. Georges Declaration for Sustainable Development (of the OECS Sub-region), the Revised Treaty of Chaguaramas (CARICOM), the CARICOM Implementation Plan for Climate Change, the National Action Plans (NAPs) and National Biodiversity Strategy and Action Plans (NBSAPs), as well as the UN Conventions on Biological Diversity (CBD), Desertification and Land Degradation (UNCCD) and Climate Change (UNFCCC), and the wider portfolio of the GEF. Efficiency: Was the project cost effective? Was the project the least cost option? Was the project implementation delayed and if it was, then did that affect costeffectiveness? Assess the contribution of cash and in-kind co-financing to project implementation and to what extent the project leveraged additional resources. Did the project build on earlier initiatives, did it make effective use of available scientific and / or technical information. Wherever possible, the evaluator should also compare the cost-time vs. outcomes relationship of the project with that of other similar projects. B. Sustainability: Sustainability is understood as the probability of continued long-term project-derived outcomes and impacts after the GEF project funding ends. The evaluation will identify and assess the key conditions or factors that are likely to contribute or undermine the persistence of benefits after the project ends. Some of these factors might be outcomes of the project, e.g. stronger institutional capacities or better informed decision-making. Other factors will include contextual circumstances or developments that are not outcomes of the project but that are relevant to the sustainability of outcomes. The evaluation should ascertain to what extent follow-up work has been initiated and how project outcomes will be sustained and enhanced over time. Five aspects of sustainability should be addressed: financial, socio-political, institutional frameworks and governance, environmental (if applicable). The following questions provide guidance on the assessment of these aspects: Financial resources. Are there any financial risks that may jeopardize sustenance of project outcomes? What is the likelihood that financial and economic resources will not be available once the GEF assistance ends (resources can be from multiple sources, such as the public and private sectors, income generating activities, and trends that may indicate that it is likely that in future there will be adequate financial resources for 22

23 sustaining project s outcomes)? To what extent are the outcomes of the project dependent on continued financial support? Socio-political: Are there any social or political risks that may jeopardize sustenance of project outcomes? What is the risk that the level of stakeholder ownership will be insufficient to allow for the project outcomes to be sustained? Do the various key stakeholders see that it is in their interest that the project benefits continue to flow? Is there sufficient public / stakeholder awareness in support of the long term objectives of the project? Institutional framework and governance. To what extent is the sustenance of the outcomes of the project dependent on issues relating to institutional frameworks and governance? What is the likelihood that institutional and technical achievements, legal frameworks, policies and governance structures and processes will allow for, the project outcomes/benefits to be sustained? While responding to these questions consider if the required systems for accountability and transparency and the required technical knowhow are in place. Environmental. Are there any environmental risks that can undermine the future flow of project environmental benefits? The TE should assess whether certain activities in the project area will pose a threat to the sustainability of the project outcomes. For example; construction of dam in a protected area could inundate a sizable area and thereby neutralize the biodiversity-related gains made by the project; or, a newly established pulp mill might jeopardise the viability of nearby protected forest areas by increasing logging pressures; or a vector control intervention may be made less effective by changes in climate and consequent alterations to the incidence and distribution of malarial mosquitoes. C. Achievement of outputs and activities: Delivered outputs: Assessment of the project s success in producing each of the programmed outputs, both in quantity and quality as well as usefulness and timeliness. Assess the soundness and effectiveness of the methodologies used for developing the technical documents and related management options in the participating countries. Assess to what extent the project outputs produced have the weight of scientific authority / credibility, necessary to influence policy and decision-makers, particularly at the national level. D. Catalytic Role Replication and catalysis. What examples are there of replication and catalytic outcomes? Replication approach, in the context of GEF projects, is defined as lessons and experiences coming out of the project that are replicated or scaled up in the design and implementation of other projects. Replication can have two aspects, replication proper (lessons and experiences are replicated in different geographic area) or scaling up (lessons and experiences are replicated within the same geographic area but funded by other sources). Specifically: Do the recommendations for management of IWEco coming from the country studies have the potential for application in other countries and locations? If no effects are identified, the evaluation will describe the catalytic or replication actions that the project carried out. E. Assessment monitoring and evaluation systems. The evaluation shall include an assessment of the quality, application and effectiveness of project monitoring and evaluation plans and tools, including an assessment of risk management based on the assumptions and risks identified in the project document. The Terminal Evaluation will assess whether the project met the minimum requirements for project design of M&E and the application of the Project M&E plan (see minimum requirements 1&2 in Annex 4 to this 23

24 Appendix). GEF projects must budget adequately for execution of the M&E plan, and provide adequate resources during implementation of the M&E plan. Regional Project Coordinators are also expected to use the information generated by the M&E system during project implementation to adapt and improve the project. M&E during project implementation M&E design. Projects should have sound M&E plans to monitor results and track progress towards achieving project objectives. An M&E plan should include a baseline (including data, methodology, etc.), SMART indicators (see Annex 4) and data analysis systems, and evaluation studies at specific times to assess results. The time frame for various M&E activities and standards for outputs should have been specified. M&E plan implementation. A Terminal Evaluation should verify that: an M&E system was in place and facilitated timely tracking of results and progress towards projects objectives throughout the project implementation period (perhaps through use of a log frame or similar); annual project reports and Progress Implementation Review (PIR) reports were complete, accurate and with well justified ratings; that the information provided by the M&E system was used during the project to improve project performance and to adapt to changing needs; and that projects had an M&E system in place with proper training for parties responsible for M&E activities. Budgeting and Funding for M&E activities. The terminal evaluation should determine whether support for M&E was budgeted adequately and was funded in a timely fashion during implementation. F. Preparation and Readiness Were the project s objectives and components clear, practicable and feasible within its timeframe? Were the capacities of executing institution and counterparts properly considered when the project was designed? Were lessons from other relevant projects properly incorporated in the project design? Were the partnership arrangements properly identified and the roles and responsibilities negotiated prior to project implementation? Were counterpart resources (funding, staff, and facilities), enabling legislation, and adequate project management arrangements in place? G. Country ownership / driveness: This is the relevance of the project to regional and national development and environmental agendas, recipient country commitment, and regional and international agreements. The evaluation will: Assess the level of regional and country ownership. Specifically, the evaluator should assess whether the project was effective in providing and communicating information on focal areas that catalyzed action in participating countries to improve decisions relating to natural resource management of in each country. Assess the level of regional and country commitment to the generation and use of focal area indicators for decision-making during and after the project, including in regional and international fora. H. Stakeholder participation / public awareness: This consists of three related and often overlapping processes: information dissemination, consultation, and stakeholder participation. Stakeholders are the individuals, groups, institutions, or other bodies that have an interest or stake in the outcome of the GEF- financed project. The term also applies to those potentially adversely affected by a project. The evaluation will specifically: 24

25 Assess the mechanisms put in place by the project for identification and engagement of stakeholders in each participating country and establish, in consultation with the stakeholders, whether this mechanism was successful, and identify its strengths and weaknesses. Assess the degree and effectiveness of collaboration/interactions between the various project partners and institutions during the course of implementation of the project. Assess the degree and effectiveness of any various public awareness activities that were undertaken during the course of implementation of the project. I. Financial Planning Evaluation of financial planning requires assessment of the quality and effectiveness of financial planning and control of financial resources throughout the project s lifetime. Evaluation includes actual project costs by activities compared to budget (variances), financial management (including disbursement issues), and co- financing. The evaluation should: Assess the strength and utility of financial controls, including reporting, and planning to allow the project management to make informed decisions regarding the budget and allow for a proper and timely flow of funds for the payment of satisfactory project deliverables. Present the major findings from the financial audit if one has been conducted. Identify and verify the sources of co- financing as well as leveraged and associated financing (in co-operation with the IA and EA). Assess whether the project has applied appropriate standards of due diligence in the management of funds and financial audits. The evaluation should also include a breakdown of final actual costs and co-financing for the project prepared in consultation with the relevant UNEP Fund Management Officer of the project (table attached in Annex 2 to this Appendix Co -financing and leveraged resources). J. Implementation approach: This includes an analysis of the project s management framework, adaptation to changing conditions (adaptive management), partnerships in implementation arrangements, changes in project design, and overall project management. The evaluation will: K. Ascertain to what extent the project implementation mechanisms outlined in the project document have been closely followed. In particular, assess the role of the various committees established and whether the project document was clear and realistic to enable effective and efficient implementation, whether the project was executed according to the plan and how well the management was able to adapt to changes during the life of the project to enable the implementation of the project. L. Evaluate the effectiveness and efficiency and adaptability of project management and the supervision of project activities / project execution arrangements at all levels (1) policy decisions: National Coordination Committee; (2) day to day project management in each of the national executing agencies, UNEP Car/RCU and CARPHA s Environmental Health and Sustainable Development Department. M. UNEP Supervision and Backstopping Assess the effectiveness of supervision and administrative and financial support provided by UNEP/DGEF/ Identify administrative, operational and/or technical problems and constraints that influenced the effective implementation of the project. 25

26 The ratings will be presented in the form of a table. Each of the eleven categories should be rated separately with brief justifications based on the findings of the main analysis. An overall rating for the project should also be given. The following rating system is to be applied: HS S MS MU U HU = Highly Satisfactory = Satisfactory = Moderately Satisfactory = Moderately Unsatisfactory = Unsatisfactory = Highly Unsatisfactory 4. Evaluation report format and review procedures The report should be brief, to the point and easy to understand. It must explain; the purpose of the evaluation, exactly what was evaluated and the methods used. The report must highlight any methodological limitations, identify key concerns and present evidence-based findings, consequent conclusions, recommendations and lessons. The report should be presented in a way that makes the information accessible and comprehensible and include an executive summary that encapsulates the essence of the information contained in the report to facilitate dissemination and distillation of lessons. The evaluation will rate the overall implementation success of the project and provide individual ratings of the eleven implementation aspects as described in Section 1 of this TOR. The ratings will be presented in the format of a table with brief justifications based on the findings of the main analysis. Evidence, findings, conclusions and recommendations should be presented in a complete and balanced manner. Any dissident views in response to evaluation findings will be appended in an annex. The evaluation report shall be written in English, be of no more than 50 pages (excluding annexes), use numbered paragraphs and include: i) An executive summary (no more than 3 pages) providing a brief overview of the main conclusions and recommendations of the evaluation; ii) Introduction and background giving a brief overview of the evaluated project, for example, the objective and status of activities; The GEF Monitoring and Evaluation Policy, 2006, requires that a TE report will provide summary information on when the evaluation took place; places visited; who was involved; the key questions; and, the methodology. iii) Scope, objective and methods presenting the evaluation s purpose, the evaluation criteria used and questions to be addressed; iv) Project Performance and Impact providing factual evidence relevant to the questions asked by the evaluator and interpretations of such evidence. This is the main substantive section of the report. The evaluator should provide a commentary and analysis on all eleven evaluation aspects (A K above). v) Conclusions and rating of project implementation success giving the evaluator s concluding assessments and ratings of the project against given evaluation criteria and standards of performance. The conclusions should provide answers to questions about whether the project is considered good or bad, and whether the results are considered positive or negative. The ratings should be provided with a brief narrative comment in a table (see Annex 1 to this Appendix); vi) Lessons (to be) learned presenting general conclusions from the standpoint of the design and implementation of the project, based on good practices and successes or 26

27 vii) viii) problems and mistakes. Lessons should have the potential for wider application and use. All lessons should stand alone and should: Briefly describe the context from which they are derived State or imply some prescriptive action; Specify the contexts in which they may be applied (if possible, who when and where) Recommendations suggesting actionable proposals for improvement of the current project. In general, Terminal Evaluations are likely to have very few (perhaps two or three) actionable recommendations. Prior to each recommendation, the issue(s) or problem(s) to be addressed by the recommendation should be clearly stated. A high quality recommendation is an actionable proposal that is: 1. Feasible to implement within the timeframe and resources available 2. Commensurate with the available capacities of project team and partners 3. Specific in terms of who would do what and when 4. Contains results-based language (i.e. a measurable performance target) 5. Includes a trade-off analysis, when its implementation may require utilizing significant resources that would otherwise be used for other project purposes. Annexes may include additional material deemed relevant by the evaluator but must include: 1. The Evaluation Terms of Reference, 2. A list of interviewees, and evaluation timeline 3. A list of documents reviewed / consulted 4. Summary co-finance information and a statement of project expenditure by activity 5. The expertise of the evaluation team. (brief CV). TE reports will also include any response / comments from the project management team and/or the country focal point regarding the evaluation findings or conclusions as an annex to the report, however, such will be appended to the report by UNEP EOU. Examples of UNEP GEF Terminal Evaluation Reports are available at Review of the Draft Evaluation Report Draft reports submitted to UNEP EOU are shared with the corresponding Programme or Project Officer and his or her supervisor for initial review and consultation. The IWEco staff and senior Executing Agency staff are allowed to comment on the draft evaluation report. They may provide feedback on any errors of fact and may highlight the significance of such errors in any conclusions. The consultation also seeks feedback on the proposed recommendations. UNEP EOU collates all review comments and provides them to the evaluators for their consideration in preparing the final version of the report. 5. Submission of Final Terminal Evaluation Reports. The final report shall be submitted in electronic form in MS Word format and should be sent to the following persons: Michael Spilsbury, Chief, UNEP Evaluation and Oversight Unit 27

28 With a copy to: P.O. Box Nairobi, Kenya Tel.: +(254-20) Fax: +(254-20) micheal.spilsbury@unep.org Brennan Vandyke, Director, Director, GEF Coordination Office Block 2, North Wing, Ground Floor, UNEP P.O. Box Nairobi, Kenya Tel: +(254-20) Fax: +( ) brennan.vandyke@unep.org Isabelle Vanderbeck Task Manager UNEP/DEPI GEF International Waters Unit th Street, N.W. Washington, D.C Tel: : isabelle.vanderbeck@unep.org The Final evaluation will also be copied to the following GEF National Focal Points. N/A The final evaluation report will be published on the Evaluation and Oversight Unit s web-site and may be printed in hard copy. Subsequently, the report will be sent to the GEF Office of Evaluation for their review, appraisal and inclusion on the GEF website. 28

29 6. Resources and schedule of the evaluation 1. Resources and schedule of the evaluation This final evaluation will be undertaken by an international evaluator contracted by the Evaluation and Oversight Unit, UNEP. The contract for the evaluator will begin on ddmmyyy and end on ddmmyyyy (# days) spread over # weeks (# days of travel, to {country(ies)}, and # days desk study). The evaluator will submit a draft report on ddmmyyyy to UNEP/EOU, the UNEP/DGEF Task Manager, and key representatives of the executing agencies. Any comments or responses to the draft report will be sent to UNEP / EOU for collation and the consultant will be advised of any necessary revisions. Comments to the final draft report will be sent to the consultant by ddmmyyyy after which, the consultant will submit the final report no later than ddmmyyyy. After an initial telephone briefing with EOU and UNEP/GEF, the evaluator will conduct initial desk review work and later travel to meet with project staff in St Lucia at the beginning of the evaluation. Furthermore, the evaluator is expected to travel to a minimum of 4 project countries and meet with representatives of the project executing agencies, project partners, and the intended users of project s outputs. In accordance with UNEP/GEF policy, all GEF projects are evaluated by independent evaluators contracted as consultants by the EOU. The evaluator should have the following qualifications: The evaluator should not have been associated with the design and implementation of the project in a paid capacity. The evaluator will work under the overall supervision of the Chief, Evaluation and Oversight Unit, UNEP. The evaluator should be an international expert with working knowledge of each of the respective focal areas (IW, BD, LD, SFM), and a sound understanding of land and water management and sustainable forestry issues preferably in the context of SIDs. The consultant should have the following minimum qualifications: (i) experience in global environment and development issues faced by small island states; (ii) experience with management and implementat ion global and/or regional projects and in particular with those interventions targeted at developing long-term functional learning partnerships that target behavior and policy-influence and decision-making; (iii) experience with (full sized) project evaluation. Knowledge of UNEP programmes and GEF activities is desirable. Knowledge of the Caribbean is an advantage. Fluency in oral and written English is a must. 7. Schedule of Payment The consultant shall select one of the following two contract options: Lump-Sum Option The evaluator will receive an initial payment of 30% of the total amount due upon signature of the contract. A further 30% will be paid upon submission of the draft report. A final payment of 40% will be made upon satisfactory completion of work. The fee is payable under the individual Special Service Agreement (SSA) of the evaluator and is inclusive of all expenses such as travel, accommodation and incidental expenses. Fee-only Option The evaluator will receive an initial payment of 40% of the total amount due upon signature of the contract. Final payment of 60% will be made upon satisfactory completion of work. The fee is 29

30 payable under the individual SSAs of the evaluator and is NOT inclusive of all expenses such as travel, accommodation and incidental expenses. Ticket and DSA will be paid separately. In case, the evaluator cannot provide the products in accordance with the TORs, the timeframe agreed, or his products are substandard, the payment to the evaluator could be withheld, until such a time the products are modified to meet UNEP's standard. In case the evaluator fails to submit a satisfactory final product to UNEP, the product prepared by the evaluator may not constitute the evaluation report. 30

31 Annex 1 to Appendix 9: OVERALL RATINGS TABLE Criterion Evaluator s Summary Comments Evaluator s Rating A. Attainment of project objectives and results (overall rating) Sub criteria (below) A. 1. Effectiveness A. 2. Relevance A. 3. Efficiency B. Sustainability of Project outcomes (overall rating) Sub criteria (below) B. 1. Financial B. 2. Socio Political B. 3. Institutional framework and governance B. 4. Ecological C. Achievement of outputs and activities D. Monitoring and Evaluation (overall rating) Sub criteria (below) D. 1. M&E Design D. 2. M&E Plan Implementation (use for adaptive management) D. 3. Budgeting and Funding for M&E activities E. Catalytic Role F. Preparation and readiness G. Country ownership / drivenness H. Stakeholders involvement I. Financial planning J. Implementation approach K. UNEP Supervision and backstopping RATING OF PROJECT OBJECTIVES AND RESULTS Highly Satisfactory (HS): The project had no shortcomings in the achievement of its objectives, in terms of relevance, effectiveness or efficiency. Satisfactory (S): The project had minor shortcomings in the achievement of its objectives, in terms of relevance, effectiveness or efficiency. Moderately Satisfactory (MS): The project had moderate shortcomings in the achievement of its objectives, in terms of relevance, effectiveness or efficiency. Moderately Unsatisfactory (MU): The project had significant shortcomings in the achievement of its objectives, in terms of relevance, effectiveness or efficiency. Unsatisfactory (U) The project had major shortcomings in the achievement of its objectives, in terms of relevance, effectiveness or efficiency. Highly Unsatisfactory (HU): The project had severe shortcomings in the achievement of its objectives, in terms of relevance, effectiveness or efficiency. Please note: Relevance and effectiveness will be considered as critical criteria. The overall rating of the project for achievement of objectives and results may not be higher than the lowest rating on 31

32 either of these two criteria. Thus, to have an overall satisfactory rating for outcomes a project must have at least satisfactory ratings on both relevance and effectiveness. RATINGS ON SUSTAINABILITY A. Sustainability will be understood as the probability of continued long-term outcomes and impacts after the GEF project funding ends. The Terminal evaluation will identify and assess the key conditions or factors that are likely to contribute or undermine the persistence of benefits after the project ends. Some of these factors might be outcomes of the project, i.e. stronger institutional capacities, legal frameworks, socio-economic incentives /or public awareness. Other factors will include contextual circumstances or developments that are not outcomes of the project but that are relevant to the sustainability of outcomes. Rating system for sustainability sub-criteria On each of the dimensions of sustainability of the project outcomes will be rated as follows. Likely (L): There are no risks affecting this dimension of sustainability. Moderately Likely (ML). There are moderate risks that affect this dimension of sustainability. Moderately Unlikely (MU): There are significant risks that affect this dimension of sustainability Unlikely (U): There are severe risks that affect this dimension of sustainability. According to the GEF Office of Evaluation, all the risk dimensions of sustainability are deemed critical. Therefore, overall rating for sustainability will not be higher than the rating of the dimension with lowest ratings. For example, if a project has an Unlikely rating in any of the dimensions then its overall rating cannot be higher than Unlikely, regardless of whether higher ratings in other dimensions of sustainability produce a higher average. RATINGS OF PROJECT M&E Monitoring is a continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing project with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds. Evaluation is the systematic and objective assessment of an on-going or completed project, its design, implementation and results. Project evaluation may involve the definition of appropriate standards, the examination of performance against those standards, and an assessment of actual and expected results. The Project monitoring and evaluation system will be rated on M&E Design, M&E Plan Implementation and Budgeting and Funding for M&E activities as follows: Highly Satisfactory (HS): There were no shortcomings in the project M&E system. Satisfactory(S): There were minor shortcomings in the project M&E system. Moderately Satisfactory (MS): There were moderate shortcomings in the project M&E system. Moderately Unsatisfactory (MU): There were significant shortcomings in the project M&E system. Unsatisfactory (U): There were major shortcomings in the project M&E system. Highly Unsatisfactory (HU): The Project had no M&E system. M&E plan implementation will be considered a critical parameter for the overall assessment of the M&E system. The overall rating for the M&E systems will not be higher than the rating on M&E plan implementation. All other ratings will be on the GEF six point scale. 32

33 GEF Performance Description HS = Highly Satisfactory Excellent Alternative description on the same scale S = Satisfactory Well above average MS = Moderately Satisfactory Average MU = Moderately Unsatisfactory Below Average U = Unsatisfactory Poor HU = Highly Unsatisfactory Very poor (Appalling) 33

34 Annex 1 Annex 2 to Appendix 9: Co-financing and Leveraged Resources Co financing (Type/Source) Grants Loans/Concessional (compared to market rate) Credits Equity investments In-kind support Other (*) IA own Financing (mill US$) Government (mill US$) Other* (mill US$) Total (mill US$) Total Disbursement (mill US$) Planned Actual Planned Actual Planned Actual Planned Actual Planned Actual Totals Co-financing (basic data to be supplied to the consultant for verification) * Other is referred to contributions mobilized for the project from other multilateral agencies, bilateral development cooperation agencies, NGOs, the private sector and beneficiaries. 34

35 Annex 1 Leveraged Resources Leveraged resources are additional resources beyond those committed to the project itself at the time of approval that are mobilized later as a direct result of the project. Leveraged resources can be financial or in-kind and they may be from other donors, NGO s, foundations, governments, communities or the private sector. Please briefly describe the resources the project has leveraged since inception and indicate how these resources are contributing to the project s ultimate objective. Table showing final actual project expenditure by activity to be supplied by the UNEP Fund management Officer. (insert here) 35

36 IWEco Regional Sub-Integrating Water, Land and Ecosystems Management in Caribbean Small Island Developing States Annex 3 to Appendix 9 Review of the Draft Report Draft reports submitted to UNEP EOU are shared with the corresponding Programme or Project Officer and his or her supervisor for initial review and consultation. The DGEF staff and senior Executing Agency staff provide comments on the draft evaluation report. They may provide feedback on any errors of fact and may highlight the significance of such errors in any conclusions. The consultation also seeks agreement on the findings and recommendations. UNEP EOU collates the review comments and provides them to the evaluators for their consideration in preparing the final version of the report. General comments on the draft report with respect to compliance with these TOR are shared with the reviewer. Quality Assessment of the Evaluation Report All UNEP GEF Mid Term Reports are subject to quality assessments by UNEP EOU. These apply GEF Office of Evaluation quality assessment and are used as a tool for providing structured feedback to the evaluator. The quality of the draft evaluation report is assessed and rated against the following criteria: GEF Report Quality Criteria UNEP EOU Assessment A. Did the report present an assessment of relevant outcomes and achievement of project objectives in the context of the focal area program indicators if applicable? B. Was the report consistent and the evidence complete and convincing and were the ratings substantiated when used? C. Did the report present a sound assessment of sustainability of outcomes? D. Were the lessons and recommendations supported by the evidence presented? E. Did the report include the actual project costs (total and per activity) and actual co-financing used? F. Did the report include an assessment of the quality of the project M&E system and its use for project management? UNEP EOU additional Report Quality Criteria G. Quality of the lessons: Were lessons readily applicable in other contexts? Did they suggest prescriptive action? H. Quality of the recommendations: Did recommendations specify the actions necessary to correct existing conditions or improve operations ( who? what? where? when?). Can they be implemented? Did the recommendations specify a goal and an associated performance indicator? I. Was the report well written? (clear English language and grammar) J. Did the report structure follow EOU guidelines, were all requested Annexes included? K. Were all evaluation aspects specified in the TORs adequately addressed? L. Was the report delivered in a timely manner GEF Quality of the MTE report = 0.3*(A + B) + 0.1*(C+D+E+F) EOU assessment of MTE report = 0.3*(G + H) + 0.1*(I+J+K+L) Combined quality Rating = (2* GEF EO rating + EOU rating)/3 The Totals are rounded and converted to the scale of HS to HU 36 UNEP EOU Assessment Rating Rating

37 IWEco Regional Sub-Integrating Water, Land and Ecosystems Management in Caribbean Small Island Developing States Rating system for quality of terminal evaluation reports A number rating 1-6 is used for each criterion: Highly Satisfactory = 6, Satisfactory = 5, Moderately Satisfactory = 4, Moderately Unsatisfactory = 3, Unsatisfactory = 2, Highly Unsatisfactory = 1, and unable to assess = 0. 37

38 Annex 4 to Appendix 9 IWEco Regional Sub-Integrating Water, Land and Ecosystems Management in Caribbean Small Island Developing States GEF Minimum requirements for M&E Minimum Requirement 1: Project Design of M&E 3 All projects must include a concrete and fully budgeted monitoring and evaluation plan by the time of Work Program entry (full-sized projects) or CEO approval (medium-sized projects). This plan must contain at a minimum: SMART (see below) indicators for project implementation, or, if no indicators are identified, an alternative plan for monitoring that will deliver reliable and valid information to management SMART indicators for results (outcomes and, if applicable, impacts), and, where appropriate, corporate-level indicators A project baseline, with: a description of the problem to address indicator data or, if major baseline indicators are not identified, an alternative plan for addressing this within one year of implementation An M&E Plan with identification of reviews and evaluations which will be undertaken, such as mid-term reviews or evaluations of activities An organizational setup and budgets for monitoring and evaluation. Minimum Requirement 2: Application of Project M&E Project monitoring and supervision will include implementation of the M&E plan, comprising: Use of SMART indicators for implementation (or provision of a reasonable explanation if not used) Use of SMART indicators for results (or provision of a reasonable explanation if not used) Fully established baseline for the project and data compiled to review progress Evaluations are undertaken as planned Operational organizational setup for M&E and budgets spent as planned. SMART INDICATORS GEF projects and programs should monitor using relevant performance indicators. The monitoring system should be SMART : 1. Specific: The system captures the essence of the desired result by clearly and directly relating to achieving an objective, and only that objective. 2. Measurable: The monitoring system and its indicators are unambiguously specified so that all parties agree on what the system covers and there are practical ways to measure the indicators and results. 3. Achievable and Attributable: The system identifies what changes are anticipated as a result of the intervention and whether the result(s) are realistic. Attribution requires that changes in the targeted developmental issue can be linked to the intervention. 4. Relevant and Realistic: The system establishes levels of performance that are likely to be achieved in a practical manner, and that reflect the expectations of stakeholders. 5. Time-bound, Timely, Trackable, and Targeted: The system allows progress to be tracked in a cost-effective manner at desired frequency for a set period, with clear identification of the particular stakeholder group to be impacted by the project or program

39 Annex 5 to Appendix 9 IWEco Regional Sub-Integrating Water, Land and Ecosystems Management in Caribbean Small Island Developing States List of intended additional recipients for the Terminal Evaluation (to be completed by the IA Task Manager) Name Affiliation Government Officials GEF Focal Point(s) Executing Agency Implementing Agency 39

40 IWEco Regional Sub-Integrating Water, Land and Ecosystems Management in Caribbean Small Island Developing States Supplemental Annex: Supervisory Plan 40