One and Half Year Review

Transcription

1 Governments of the Eastern Caribbean/ UNICEF CPAP One and Half Year Review M&E Section Barbados July 2013

2 Situation Analysis - Social Statistics for evidence-based decision making From Data assessment in the Eastern Caribbean (2010) The OECS countries and Barbados have incomplete reporting for most of the indicators in the 2009 UNICEF World Fit for Children reports. While reasonably complete (though uneven across time) data exists for several of the MDG indicators, data is extremely sparse for other indicators of child health, child protection and women s health. Little information appears to be regularly collected or reported on measures of nutrition for young children, such as exclusive breastfeeding duration or feeding practices, and micronutrients. All countries in the region have reported recent immunization rates either in the UNICEF State of the World s Children, or the most recent regional MDG report. But reproductive health indicators are incomplete and unevenly reported throughout the region. Surprisingly, literacy and education indicators are also unevenly available across the Caribbean countries. Almost no information appears to exist for indicators related to child protection. Furthermore, almost no data exists or is reported on adolescent development, or life satisfaction

3 Situation Analysis Determinants to barriers and bottlenecks Example 1 : Culture of M&E (Post-IDEAS regional meeting) Environment Lack of Policy/legal framework for M&E function and work Insufficient or lack of funds for M&E function and work (no M&E unit, vacant positions, budget for M&E activities, etc.) No consequence for low/no performance in Govt programs Supply side Weak or missing M&E component in Govt programmes Lack of dedicated funds for M&E activities of Govt programmes Focus is more on funds utilization vs outcomes Demand side Lack of incentives to manage performance Insufficient coordinated and/or sector approach to M&E Quality Capacity gaps (skills, M&E systems, etc.) for M&E work No clear programming cycle (Planning-Monitoring-Evaluation process and tools)

4 Situation Analysis Determinants to barriers and bottlenecks Example 2: Administrative data for the Day-care, ECD and Education Sectors (Grenada / St Lucia workshops) Environment Supply side Demand side Quality Obsolete Statistical Act (CSO and Ministry s responsibilities, data production and access, etc.) Insufficient or lack of funds for data production and analysis within CSO as well as line-ministries (budget, staffing, equipment, etc.) Priority given to economic and financial data (more frequent, more sectors, etc.) over to social data Low response rates and delays in response to questionnaires Lack of dedicated staff for timely processing of collected data Ad-hoc production of some key social data Lack of internal demand for social data / Donor s driven demand for reporting Insufficient linkages between data production and data use for decision-making Insufficient sharing of data among Divisions and Ministries to (i) inform decision-making and (ii) feedback to stakeholders Capacity gaps ( skills, equipment)for data production and processing Challenges in presenting a complete picture (all islands, private and public sector, etc.)

5 Situation Analysis - Update Working with counterparts, the M&E section could use the first 1,5 years of the current MCP to: 1- have a clearer understanding of the determinants to barriers and bottlenecks to M&E culture and production, sharing and use of social data for decision-making basis for country specific assistance 2 collect updated and new data/evidence on key Children s rights issues more available data/evidence 3- identify persisting gaps and issues (ongoing/pending work for ): - Continued data gaps (nutrition, adolescents, disability, etc.) - Persisting need to harmonize and standardize definitions (OECS, CARICOM) and methodology (CPA) - Need to analysis observed discrepancy between national data and International estimates

6 Situation Analysis Update As a RESULT of the above, It is now possible to work on the following (in selected countries): Analysis of insufficiently documented issues - child mortality, discipline at home, teacher s absenteeism and violence among children at school, etc. Equity-focused analysis Trends Analysis - rural/urban disaggregation for St Lucia and Barbados for the first time but also, by age, ethnicity, income, education level, stratum, etc. - In T&T, MICS 3 and 4 trends analysis Child poverty study Country Equity Profiles - In T&T, using most recent Census, HBS and MICS4 data (led by Social Policy) - For 12 countries (using DevInfo as a platform, STATA for further analysis) + roll out of Devinfo with sufficient mass of information in T&T, St Lucia, and Barbados and other countries (Montserrat, Anguilla, St Vicent&Grenadines)

7 Strategic Focus Focus Countries Data collection / Survey St Lucia, Barbados and T&T (MICS4) Data collection / Admin. data systems Equity-focused SITANS / Country profiles Knowledge Management / Devinfo Evidence-based National Planning/ Visioning using data (MICS4, etc.) Country-level approach: St Lucia, Dominica, Grenada and Antigua (Day-care, ECD and Education sectors) Dominica and St Lucia (Child protection sector) Multi-country approach: BoTs (ECD and C.P. sectors - DFID proposal) Regional level approach: Child Protection (Juvenile Justice) with OECS Secretariat/ UNODC All 12 SIDS 5 pilot countries : Barbados, St Lucia, Grenada, Anguilla, Montserrat + OECS Secretariat (CARICOM/CDB/UNICEF initiative) T&T (CTT-info linked to MICS4 data dissemination) Trinidad&Tobago St Lucia Equity-focused Evaluation: St Lucia (Koudmen St Lisi social assistance prorgram, ) St. Kitts & Nevis and Antigua & Barbuda (HighScope) St.Vincent & the Grenadines (ECHO) - top 3 countries in 2012/13: St Lucia, Trinidad, Barbados - Many countries, therefore many priorities / Many priorities, therefore many countries

8 Strategic Focus Strategies in Middle Income Countries 1/ At the Country level: Advocacy and technical assistance at higher level (PS, DPS levels) Facilitating/convening and tech. assistance Partnerships and alliances with new partners MoST/ RBM-Equity training, Performance Management/Office of the PM in T&T, Admin Data / Grenada Increased participation/coordination with key line Ministries (MoH, MoSD, MoSW, MoE) in each country (MICS work, Admin. data systems, etc.) Ministry as well as with THA in T&T; Private sector for ECD administrative data in Grenada, Ministry of Planning for M&E work in T&T, etc. --- Key to ensure political commitment, technical support, cross-sectoral work and sustainability 2/ With OECS Secretariat, CDB, CARICOM, ECLAC as well as with UNCTs (2): Build alliances and resource leveraging Knowledge management UNDAF create a critical mass of partners working for the same objectives with the same strategy: OECS Secretariat/UNODC and UNICEF for admin. data for Juvenile Justice, CDB/UNDP/UNICEF for data collection/mpi as well as Evaluation Network, CDB/CARICOM/UNICEF for Devinfo CARICOM/CDB/UNICEF DevInfo initiative in 5 pilot countries joint activities (post IDEAS work to promote M&E Culture, new survey tool for the SIDS, etc.) --- Key to avoid competition among Development partner s approaches, to avoid redundant and duplicated efforts as well as extra-burden on counterparts

9 Strategic Focus Strategies in Middle Income Countries 3/ With other partners (UWI, possible donors, etc.) Fundraising / resource leveraging systematic efforts to fundraise (DFID proposal) and leverage resources (UNFPA, CUSO) in support to IR1.1 Partnership building strengthen (CDB and UNDP) and create (UWI for M&E master course ) longer-term and results-oriented partnerships. New opportunities for MICS report launch and child-friendly version of MICS reports, SMS-reporting children participation - key to ensure stronger and more direct relationships with all groups of the society 4/ At the regional level (4 Unicef offices, with TACRO) Knowledge Management assessment of 3 possible knowledge centers to be part of a virtual hub (children s rights observatory) Fundraising regional proposal for M&E (2012) Facilitating/convening between Sept 2012 and May 2013 (9 months), 3 regional workshops organized by the M&E section in Barbados (MORES workshop, MICS4 data dissemination and post-ideas meeting) - Key to support a regional approach to national problems and South-South Cooperation

10 2012 Review of section IR / Indicators/Analytical Statement IR 1.1 Statement: By 2016, sex and age disaggregated data on children is available, integrated into national monitoring and evaluation systems and the evidence produced is used for policy analysis and decision-making by Governments in Eastern Caribbean countries (OT 501/502) IR Indicator(s) No of countries in which MICS (Round 4 or 5) is completed Baseline: zero / Target 2016: 6 Progress: Full Draft report for Barbados, Few draft chapters for St Lucia. MICS4 Data for T&T getting validated. all 3 final reports are expected for Sept 2013 Draft Analytical Statement Each MICS4 experience has a been a completely different one with issues ranging from a substantial delays in completing the field work (Barbados), to significant internal delays (T&T), to loss of a significant number of key staff members (Barbados, T&T) to strong CSO leadership throughout the process (St Lucia). dependence on partner s timeline (all 3 countries) No. of countries with administrative data system in Education, Health and Protection, that produce up-to date sex and age disaggregated data on girls and boys Baseline: t.b.c in 2013 / Target by 2016: 6 Progress: -Day-care/ECD/Education: Progress in 3 countries (Dominica, St Lucia and Grenada) - Child Protection : OECS/UNODC/UNICEF regional workshop for Juvenile Justice Data Workshops critical to get (i) getting PS level support, (ii) widening participation to the private sector and sub-national partners, (iii) increasing buy-in from all stakeholders, and (iv) promoting inter-division and cross-sector work that is key to establishing an effective admin. data system. Important milestones in Grenada / Education sector - revision of monthly and annual questionnaires, - real-time monitoring of teacher s absenteism and violence against children at school - more systematic use of Admin. data for management decisions and planning work. Important milestones reached in St Lucia / Education sector : - increased rates of returns as a results of increased mobilisation (101 principals, 96 ECD pratictioners) Very good progress in Grenada / Day-care sector: - simplification of the M&E tools and process, - draft bill for private centers to register, comply with quality standards and report data to MOSD and (iii) cross-sector work between MOE and MoSD (tools, children tracking, etc.) - dependence on partner s timeline (St Lucia, Grenada) - Challenges with How to scale-up within the country and replicate good practices in other countries

11 2012 Review of section IR / Indicators/Analytical Statement IR 1.1 Statement: By 2016, sex and age disaggregated data on children is available, integrated into national monitoring and evaluation systems and the evidence produced is used for policy analysis and decision-making by Governments in Eastern Caribbean countries (OT 501/502) No. of countries with national and sectoral ( Health, Education and Protection ) policies based on evidence provided by disaggregated data on girls and boys Baseline: TBC by dec / Target by 2016: five No. of countries with Health, Education and Protection programs based on evidence (analysis) provided by disaggregated data on girls and boys Baseline: t.b.c by dec2013 / Target by 2016: twelve Progress: t.b.c by dec 2013 Using data for evidence-based work requires institutional change New data (MICS4, etc.) is already supporting Office advocacy and evidence-based policy work (St Lucia) will do so (Barbados, T&T) Knowledge by partners (Gvt, UN Agencies, etc.) of the existence of MICS4 data leads to a high demand for them as well as capacity building request (data analysis and RBM) in all three countries (St Lucia, Barbados and Trinidad) Ambitious result and targets: 3 sectors No of country-led evaluations conducted in 5 countries (four priority countries and T&T) Baseline: zero Target by 2016: five Progress: 1 - Evaluation of Koudmen-St.Lisi programme in St.Lucia completed - 2 evaluations (ECHO and HIGHSCOPE) to start Evaluation of Koudmen was conducted in a participatory way which has permitted to : - build the M&E capacity of SSDF, - engage a large range of stakeholders (former board and new board members, PM Office, families and children) into the process - increase the demand for a strong M&E system (MoST, SSDF) Findings of the Koudmen evaluation are already used by (i) SSDF to support Koudmen phase 2 (recruitment of an M&E Officer) and (ii) by Social Policy Specialist to inform Social Protection Reform Debate. * Koudmen evaluation was presented at IDEAS Global conference (May 2013) and will be part of IDEAS publication + UNICEF Research Office

12 2012 Review of section IR / Indicators/Analytical Statement IR 1.1 Statement: By 2016, sex and age disaggregated data on children is available, integrated into national monitoring and evaluation systems and the evidence produced is used for policy analysis and decision-making by Governments in Eastern Caribbean countries (OT 501/502) IR Indicator(s) No of countries with updated situation analysis on the realization of the rights of children and women. Baseline: eleven Target by 2016:twelve Progress: 12 Country Equity Profiles will be available by Dec. 2013, using most recent data Draft Analytical Statement In 2012, priority was on gathering updated data and rolling-out MORES (after Sept. 2013). 4 out of 5 analysis to determinants to barriers and bottlenecks planned for 2013 (IMEP) are completed there is a still a strong need to build partners capacity for equity-focused work No of countries that produce research and position papers or thematic analysis on child poverty and disparities Baseline: 0 Target by 2016: 12 countries Progress: t.b.c by Dec No of regional bodies/institutions (UWI, CARICOM, OECS, CDB, ECLAC) that produce research and position papers or thematic analysis on child poverty and disparities Baseline: 0 Target by 2016: 5 sub-regional Progress: t.b.c by dec Not a priority for this year, as efforts still focus on increasing data production for equity-focused analysis MICS4 data and other national data ( like census, SLC, etc.) create new opportunities for 2013/14 Ongoing discussions in all 3 MICS4 countries for MICS4 data further analysis and Child Poverty study (T&T) UNFPA funds available (around USD 15000) for MICS4 data analysis in T&T, St Lucia and Barbados Production of equity focused publications may first require capacity building on Equity

13 Challenges The IR 1.1 looks like a PCR in itself. By design, its implementation is constrained as some expected results are very ambitious given (i) the actual situation at country level and (ii) the limited resources and timeframe. At times, misunderstanding (ex: only 51 centers ) and over-simplification of the work to be done (ex: MOE/Admin. system) Promoting M&E culture, use of data for decision-making or strengthening administrative systems, is not just a technical issue. It is also and equally, a political and institutional one. Observed delays in the IR1.1 implementation are due to internal and external factors. M&E section s constraints (competing priorities, simultaneous requests, Implementing a program and monitoring the Country Program, etc.) Office internal constraints (internal processes, delays, etc.) Partners c constraints (national agenda vs UNICEF timeline, internal delays, high staff turnover, coordination among ministries, etc.)

14 Challenges Promoting equity (disparity reduction, decentralized monitoring, etc.) is a change for for Partners and UNICEF staff as well. it is neither simple nor can be made in a simplistic way It requires stakeholder s buy-in and time M&E section work is to comply with very strict Quality Standards (MICS manual, Evaluation Quality Checklist, Equity-focused Sitan guidelines, etc.) this requires a lot of additional time and efforts to work with partners and consultants to ensure compliance and quality of the final reports ( 3 MICS and 3 evaluations for ) Examples of : CFS evaluation, Koudmen evaluation

15 Lessons Learned in 2012/13 1- Equity is a sensitive issue and needs to be approached in the context of : Small states : additional work / extra workload, capacity gaps (skills, staffing, budget, etc.) small islands : anonymous data, trust, targeting, etc. 2 - Promoting equity-focused analysis of social data requires to invest in building interest, buy-in and national capacity for it. Capacity gaps observed (staff, budget, skills, etc.) versus realistic timeframe for 12 countries and 4 large sectors Who (if not UNICEF for now) will tacks disparities issues that are central for MICOs? / Which counterparts can become a champion in using decentralized monotoriting data to inform Gvt program and policy work? Will Equity be like the MDGs? 3 - Given the expected results for IR1.1, it is important to be both strategic and opportunistic. 4 - To save time and money, and avail appropriate technical support on time, it is important to identify skilled experts (including SSC, TACRO roster, etc.) and partners for equity focused work and sign MOUs or LTAs if the best option.

16 Planning for Mid-Term Review 1 - To document: - Promotion of evidence-based decision making (MICS4 work in 3 countries, Admin. data work in 3 sectors, promotion of evaluation culture with CDB/UNDP/UNICEF, etc.) 2 - To Evaluate: - UNICEF (ECA or 4 COs?) capacity building strategy in the sub-region in support to public reform, increase public performance and capacity to respond to emerging issues - Evaluation of Gvt s partnerships with CSOs to implement national policies and programs and deliver social services to the poorest

17 Planning for Mid-Term Review 3 - To Review : - In-depth analysis of Working in middle-income SIDS : what does this means for UNICEF ECA? : UNICEF modalities of work in 12 SIDS such as country focal points, counterpart s participation in the P,M&E process, effective model for upstream work, etc. Part of the answer can be found by analyzing the cumulative effects of working in small states and islands and in a multi-country environment Small state Same sovereignty as for larger States (importance of ownership, leadership, national process, etc.) Same Ministry structure as a in larger State but less staff and less specialized skills Significant negative effects of staff turn-over on the institutional capacity Etc. Island Specific development challenges (poverty, limited resources, high external debt, etc. ) Important staff turnover due to few good job opportunities in the country Small pool of consultants Etc. Multi-country / Regional dimension Need for increased coordinated approach at the regional level But also, need to respond to individual country requests How best to action from the regional level to the national level? How best to scale-up at the regional level? National timeline vs UNICEF timeline vs ideal timeline vs only possible timeline for implementation and to ensure adequate representation and participation from all key regional stakeholders Etc.