M&E in Practice: Micro-hydro Projects in Nepal

Transcription

1 M&E in Practice: Micro-hydro Projects in Nepal African Electrification Initiative Practitioner Workshop Maputo, Mozambique June 9-11, 2009 Hussain Samad The World Bank

2 Presentation Context Based on an ongoing study Assessing the Social Impact of Rural Energy South Asia Energy Department (SASDE) studies MH projects. More on this study can be found in Banerjee, Singh and Samad (forthcoming). First of its kind in WB to implement M&E framework in the context of rural electrification (RE) and investigate development impacts of RE, which has so far been limited to anecdotal evidences. 2

3 Rural Electrification in Nepal: Context Wide disparity in electricity access between rural (35%) and urban areas (90%). Extension of grid is not viable in remote areas, because of terrain, altitude. MH is a viable alternative. MH projects are implemented under Rural Energy Development Program (REDP) by Alternative Energy Promotion Center (AEPC), an agency under the Ministry of Environment Science and Technology (MoEST). Started in 1996 in 5 districts with funding from UNDP. WB joined the effort in 2 nd phase in Currently in 3 rd phase. As of Dec. 2008, about 235 MH installations (3.6 MW capacity) and 30,000 beneficiary HHs in 40 districts. 3

4 Purpose of M&E of MH Projects Improve management (better planning, reporting). Track and measure progress/benefits. Ensure accountability to GoN, donors (results for investment). Generate support from stakeholders. Allow learning from experience. 4

5 MH M&E Development: Steps & Status Completed Nov08-Dec08 Completed Jan09-May09 Next Jun09-Jul09 5

6 Information Collection/Aggregation Actors GoN, donors track indicators received from AEPC AEPC AEPC tracks indicators received from District District Level DDC tracks indicators received from Community/Field Office Community/Field Level CO collects field level information 6

7 Review of Existing M&E System: Areas of Improvement KPIs are not captured comprehensively and formats are not well defined. Limited human resource for collecting and recording information. Also limited skills of M&E personnel. Roles and responsibilities are not well defined at field and head office level. No sophisticated tools and software within AEPC to analyze collected information. Calculations are often done manually. 7

8 Building up of KPIs: Project Stage Outline Inputs Outputs Short-term & Intermediate Outcomes Impact (Long term outcomes) Resources committed Budget spent Products & services provided No. of MH systems installed Behavior changes resulting from outputs No. of HHs with electricity Study hours in evening for children Long term improvement for beneficiaries Increase in school performance 8

9 Building up of KPIs: Project Stage Details Inputs Outputs Short-term & intermediate Outcomes Impact Technical - Land, Labor Community Mobilization - # awareness programs Financial -Funds utilized TECHNICAL -No. of MH systems Installed COMMUNITY PARTICIPATION -No. of COs CAPACITY BUILDING -No. of training programs ACCESS -% hhold connected AFFORDABILITY -Tariff OPERATING EFFICIENCY -No. of complaints FINANCIAL PERFORMANCE -Contribution by CO RELIABILITY -Hrs of electricity consumed Reduction in coping costs Increased productivity, new activities No. of study hours with improved lighting, time saving Increased awareness with access to information through TV, radio. Increased participation in CO No. of visits to health clinics Improved health facilities Reduced indoor pollution Less firewood consumption Income Education empowerment Health Environment 9

10 Building up of KPIs: What to Know Project stage input, output. Definition high level, in measurable way. Unit of measurement. Value baseline, current, target. Frequency of collection monthly, bi-yearly. Level of collection community, district. Person/entity responsible. Source interviews, bills. Analysis detail frequency, method, level. 10

11 Building up of KPIs: Sample Worksheet St age Type of Indicat or KPI Defn Unit Unit of aggregat ion & monit oring Source of Informat ion CO/ MHFG level Freq. Dist rict level Freq. AEPC Freq. Input Input Implement at ion Progress No. of MH project s commissioned Fund Ut ilizat ion Ut ilizat ion of Funds Out put Technical Inst alled capacit y of MH Out put Access Number of HHs wit h elect ricit y access t hrough MH Out come Reliabilit y Planned supply int errupt ions in t he mont h Out come Reliabilit y Unplanned power shut downs in t he mont h Total funds utilized No. of hholds connected to MH No. DEES Trimesterly RESD Annual DEES Rupees MHFG Monthly DEES Monthly MEC Monthly Community Energy Fund kw MHFG Project Completion DEES Annually RESD Annual MHFG Number MHFG Timesterly DEES Trimesterly MEC Trimesterly - No. of hholds connected to MH: MHFG - Total no. of hholds: DPR No. MHFG Monthly DEES Monthly MEC Monthly MHFG Hrs. MHFG Monthly DEES Monthly MEC Monthly MHFG 11

12 M&E Operationalization: Closer Look at Information Flow 12

13 M&E Operationalization: Data Collection and Processing Plan specifically by level (field, district, central) and mode of collection (interviews, records (bills, reports)). Develop efficient data entry/processing and report generation systems, and connect different levels through network (MS Visual Basic, XML, SQL Server). Establish proper sequence and timeline for activities. Train staff and define staff roles and responsibilities Provide hands-on demonstrations of the whole process. 13

14 Impact Assessment of MH Projects: Context Covers the last stage of M&E framework. Provides hard evidences of intervention effects which qualitative or anecdotal studies cannot. Makes interventions more efficient and properly targeted. Gives accountability and justification for interventions to donors, stakeholders and policymakers. Done in longer intervals (usually 2+ years), because of the time it takes for causality to materialize. Also more expensive than implementations of earlier stages of M&E. 14

15 Impact Assessment of MH: Potential Outcomes Income. Expenditure. Schooling performance. Health outcomes. empowerment. Enterprise revenue and profit. 15

16 Completed Mar09-May0 Impact Assessment of MH: Steps & Status Completed Nov08- Feb09 Next May09- July09 16

17 Impact Assessment of MH: Sample at a Glance Total Sample 2,500 Treatment HHs 1,500 Control HHs from treatment area Control HHs from project areas Control HHs from non-project area

18 Impact Assessment of MH: Data Analysis Will quantify benefits of MH projects on the outcomes mentioned before. Appropriate impact assessment methodology will be adopted (for example, PSM) to isolate MH impacts to ensure benefits are due only to MH projects. Findings can serve as a test case which can be replicated to similar projects. 18

19 Thank you. 19