Corporate KPI Framework: Initial proposal for Board Constituency review. 22 th Jan 2016 Doc No.: KPI/bc02

Transcription

1 Corporate KPI Framework: Initial proposal for Board Constituency review 22 th Jan 2016 Doc No.: KPI/bc02

2 Developing the Corporate KPI Framework Status of framework development Context & guiding principles for framework revision Framework overview Indicator proposals Annex 1

3 1 Status update of framework development Overview Next Steps The KPI Framework for the new strategy has been in development with technical and operational teams since April 2015 with brief updates provided to SIIC over this period The Strategic Framework was approved by the Board at its 34 th meeting in November 2015 providing the firm basis on which this first draft KPI framework has been designed Five MEC & Steering Committee meetings since the autumn provided strong guidance on Gender, RSSH, Key populations, Human Rights, Domestic investments indicator development Technical partners have been engaged in reviewing the technical soundness of indicator proposals Preliminary feedback from the OIG audit of the current KPI framework has showed good level of acceptance of framework design This document and the accompanying narrative sets out the preliminary first draft of the proposed framework for Board constituency input Written feedback is requested from constituencies which will then be reviewed and summarized by the Secretariat (see accompanying form) Consultation meetings with constituencies will be held to resolve differing viewpoints on major issues This input will be integrated into the second draft of the proposals for review by the Board Committees 8/9 th March 2016 After Board approval of Corporate KPI framework, Secretariat will fully develop the framework of cascaded Operational KPIs 2

4 1 Developing the Corporate KPI Framework Proposals are being developed with significant input from internal and external stakeholders Committee Technical Constituency update development review Consultation meetings 5 Review & approval June Oct 2015 High level framework design shared with June SIIC Example KPI development thinking shared with October SIIC No presentations yet made to FOPC (or AEC) April ongoing All proposals developed in partnership with technical teams (60+ staff involved) Ongoing consultation with technical partners Five MEC & Steering Committee meetings over autumn provided strong guidance on: Gender RSSH Key populations Human Rights Domestic investments Jan 22 nd 2016 Full framework will be sent to Board Constituency groups Written feedback will be requested, and every point raised will receive a response Areas of contention will be discussed during the planned consultations with the implementer and donor groups Jan Feb 2016 One-on-one meetings with constituency groups Implementer group meeting (date TBC) Civil society meetings Jan 19 th & Feb 11 th Donor group meeting (date TBC) March 8 th 9 th Review of second draft of framework incorporating constituency feedback by Board Committees April Final revisions will be incorporated into submission to Board for approval May - November Final methodological development, baseline & target setting for Board approval Full development of cascaded Operational KPIs 3

5 2 New framework design draws from best practice structures and guiding principles Leveling indicators Thematic reporting Best practice from comparable organizations Performance management framework includes multiple levels of KPIs targeted to different audiences: global trends, contributive and attributive indicators, as well as operational monitoring and management data Tracking and reporting on strategic themes are consistently provided to fill information gaps at goal-level tracking Lesson learned Strategic and activity-level KPIs will track Global Fund s contribution to mission, while additional monitoring from operational KPIs and management information will provide deeper analysis, in particular on levers for action Corporate indicators will be directly underpinned by operational KPIs, consistent management information, and thematic reporting for each strategic objective Variance analysis Clear concise analysis is required to link and contextualize results and enable target audiences to better understand and use performance information for strategic decision making Increased clarity will be provided on the levers for management action available, accountability for action and how information across and between the different levels links together

6 2 Hierarchy of performance management framework A few, measurable, results-oriented KPIs covering the life of the Strategy Current focus KPIs focused on global targets & impact Corporate level Strategic KPIs Activity specific KPIs KPIs to enable Board/ Committee oversight for governance on key activities for strategy goals Secretariat level Operational KPIs More granular measures measuring management performance at entity levels of the Global Fund 5

7 2 Principles guiding revision of the framework have been updated Principles used to guide development of the KPI Framework Align the framework with the Strategy Set the Framework for the lifetime of the Strategy Define a clear hierarchy with logical links between the levels Reduce the number of KPIs and increase focus Complement the KPI framework with a regular Quarterly Dashboard of other specific indicators Additional principles Activity specific KPIs should measure the overall effect of the Strategic Objectives and not the specific effect of each subobjective Indicators should be measured on a meaningful number of countries Accountability should be clear and interpretation defined accordingly Ensure indicators are visible and measurable 6

8 2 Secretariat-level information will complement Corporate level KPIs Corporate level current focus Secretariat level to be fully developed once the corporate level KPI framework is approved Strategic KPIs Activity specific KPIs Operational KPIs Thematic Reporting & Management Information KPIs focused on global targets & impact KPIs to enable Board/ Committee oversight for governance on key activities for strategy goals More granular measures measuring management performance at entity levels of the Global Fund Example Operational KPIs provided in this document for illustration Routine tracking at management level should provide underpinning analysis. Important gaps currently exist here, and the proposed performance management framework will fill these gaps, notably on programmatic investments, activities, results and impact 7

9 3 Proposed change to the current framework Retain Revise Delegate to operational Drop Strategic goals Strategic targets Strategic KPIs 1 Performance against strategic goals 2 Quality and coverage of services 3 Performance against strategic service delivery targets Strategic Objectives Invest more strategically Evolve the funding model Actively support grant implementation success Promote and protect human rights Sustain the gains, mobilize resources Strategic enablers Implementing operational excellence Enhance partnerships Activity specific KPIs 4 Efficiency of Global Fund investment decisions 7 Access to funding 9 Effective operational risk management 5 Health system 8 New Funding Value for money strengthening Model transition 6 Alignment with national reporting systems 11 Grant expenses forecast 12 Human rights 13 Resource 15 Efficiency of grant protection mobilization management operations Domestic 16 Quality of financing for AIDS, TB & Malaria management and leadership Transition out more operational measures and replace with fewer, but more strategic measures 8

10 3 Initial view on the framework Retained Revised Addition Strategic targets Strategic KPIs 1 Performance against impact targets 2 Performance against service delivery targets Strategic Objectives Build resilient & sustainable systems for health Promote and protect human rights & gender equality Mobilize increased resources Strategic vision Invest funds to maximize portfolio impact Improve the performance of strategically important components of national systems for health Reduce human rights barriers to service access; & Reduce gender and age disparities in health Increase available resources for HIV, TB & Malaria; & Ensure availability of affordable quality-assured health technologies 3 Alignment of investment & need 6 Strengthen systems for health 8 Gender & age equality 10 Resource mobilization Activity specific KPIs 4 Investment efficiency 7 Fund utilization 9 Human rights 11 Domestic investments 5 Service coverage for key populations 12 Availability of affordable health technologies 9

11 3 The guiding principles were used to examine the proposed indicators Dimensions Accountability Data availability Actionable & reliable results Coverage Projections available Frequency of reporting Definition Contribution: Global Fund is one actor among many contributing to results achieved Attribution: Global Fund can largely attribute results to its actions Data may currently be available in an existing system; collected in a structured manner (but not available in an existing system); or not yet available at Secretariat level and/or country level Results are replicable, reliable and guide Global Fund decision-making What segment of countries, donors, etc., is represented in result Is it possible to forecast this KPI s results, thus enabling the Secretariat to take action to mitigate potential future underperformance Given the frequency of data collection, or the size cohort sampled, how often will meaningful KPI results be produced? (eg. Annually, quarterly) A summary by indicator is provided on the next page 10

12 3 KPI Proposals Accountability Data availability Actionable & reliable results Coverage Projections available Frequency of reporting 1 Performance against impact targets Contribution Structured Full portfolio Annually 2 Performance against service delivery targets Contribution System Full portfolio Semi annually 3 Alignment of investment with need Attribution System Full portfolio Quarterly 4 Investment efficiency Contribution Structured Portfolio segment 5 Key population service coverage Contribution 6 Resilient & sustainable systems for health Mix Not yet available Not yet available Semi annually Portfolio segment? Annually Portfolio segment Annually 7 Fund utilization Mix System Mix Mix Quarterly 8 Gender & age equality Contribution Structured Portfolio segment? Annually 9 Human rights Mix Not yet available Portfolio segment Mix Annually 10 Resource mobilization Contribution System Full portfolio Quarterly 11 Domestic investment Contribution Structured Full portfolio Annually 12 Availability of affordable health technologies Mix Structured Mix Mix Annually 11

13 Key issues and Secretariat s proposed approach Accountability Challenge Most corporate indicators report on a contribution basis. Because many are outcome/impact measures, Global Fund is one of many actors and cannot be held solely accountable for results Proposed approach Framing of accountability and available levers for action should be made clear. Operational indicators and management information will be more attributable to the Global Fund. Initial thinking on potential more attributable operational KPIs for each Strategic Objective are outlined in this document for information Data Challenges Many indicators will require significant development of data collection systems/processes, only some of which will be covered by Project AIM. Retain the ambition of what we aim to achieve, and invest in data systems where feasible; operational progress towards longer term achievements should be reported until data is available to measure proposed KPI Time lag to ascertain measurable effect For some indicators, there will be important time lags before expected scale up effects will be visible on the ground e.g. 5 th Replenishment starts 2017; first new grants signed end-2017; first data from new grants available mid-2018; 6 th replenishment 2019 Acknowledge that for outcome/impact indicators, a time lag to measurable effect will always exist. Accurate projections of expected future performance and KPI target setting should reflect expected lag 12

14 Indicator proposals by Strategic Objective Each Strategic Objective section includes: Outline of Strategic Objective Proposed Corporate KPIs for discussion and input Analysis of each proposal (data availability, accountability, etc.) Examples of potential Operational KPIs as illustration of tracking information that is more attributable to the Global Fund s management and operations 13

15 Service delivery targets 4 Investing to End Epidemics: Impact targets Global Plans Strategic targets Under development with partners Global Fund leading targets Indicators for other selected services HIV / AIDS Rapidly reduce HIV mortality and incidence through scaling up universal access to HIV prevention and treatment in line with the UNAIDS Fast Track and WHO Global Strategy TB Malaria Rapidly reduce TB, TB-HIV and MDR-TB incidence and related mortality through universal access to high quality care and prevention in line with the End TB Strategy and Global Plan to End TB Scale up and maintain interventions to reduce Malaria transmission and deaths and support countries to eliminate Malaria, in line with the Global Technical Strategy and AIM Under development with partners 14

16 KPI 1 Strategic Vision Performance against impact targets Invest funds to maximize portfolio impact Aim of indicator Measures the extent to which Strategic Objectives are achieving high level goals of lives saved (or reduction of deaths) and reduction of new infection/cases. Given limitations inherent in modeling methodology, data timeliness, and projection reliability, this indicator provides a high level view on overall progress. However, coupled with KPI 4, cascading these targets down to regional/country level will increase accountability and help to close the loop. The measure should be interpreted with these limitations in mind or reported separately as a high level tracking measure rather than a KPI. Measure a) Estimated number of lives saved or reduction in deaths b) Reduction in new infections/cases Limitations & mitigation actions Estimates produced by WHO/UNAIDS use standardized models and country-specific data with variable quality and availability. Country-level impact modelling is being undertaken for select countries Decision required on whether impact measures should be expressed in terms of lives saved or reduction in deaths. Numerical targets will be aligned with the investment case modelling which has been developed with partners Data is available with a one-year lag, and is sensitive to changes to the modelling methodology and changes to historical data 15

17 KPI 1 Indicator dimensions Performance against impact targets Dimension Comment Accountability Data availability Actionable & reliable results Coverage Projection available Contribution Structured Full portfolio Data is sourced externally. Global Fund contribution to overall impact is estimated in-house Indicator works as more of a high level tracker. Target setting will be informed by the replenishment result and allocation model assumptions. Disaggregation of targets to region/country levels will inform analysis of performance along with results for KPIs 2, 3, 4 & 7. Sophistication of the projection methodology is being improved. Targets will require periodic recalibration to take into account changes to modelling methodology and historical data Frequency of reporting Annually Data is reported with a one-year lag due to partner data collection schedules 16

18 KPI 2 Strategic Vision Performance against service delivery targets Invest funds to maximize portfolio impact Measure a) # of people alive on ARV therapy b) # of TB cases treated according WHO end TB strategy c) # of LLINs distributed d) # of bacteriologically confirmed drug resistant TB treated with a 2 nd line regimen e) # males circumcised f) # of IRS services delivered g) # of people who received HIV testing & counseling and know their results Aim of indicator Measures extent to which the Strategic Objectives are achieving the high level service delivery targets. Services tracked as part of the measure will be reviewed with technical partners to ensure alignment. As projection methodology is strengthened and results forecast is institutionalized, the indicator will drive portfolio performance management in conjunction with ITP project. Limitations & mitigation actions Validity: Global Fund share of national results will be based on a contribution model Availability: Data reported with a 6 month lag Reliability: Country- and secretariat-level data/system limitations Work with partners to ensure consistent application of reporting guidance (eg. Reporting of full national results) Accurate projections will be key: An operational KPI tracking forecast accuracy will embed results forecast into Secretariat decisions Data Management for Impact initiative & Project AIM will improve data and systems for reporting internally and at country level 17

19 KPI 2 Performance against service delivery targets Indicator dimensions Dimension Accountability Data availability Actionable & reliable results Coverage Projection available Frequency of reporting Contribution System Full portfolio Semi annual Comment It is proposed that all grants will report national results, however, the Global Fund share of national results will be determined based on a contributive model Target setting will be informed by the replenishment result and allocation model assumptions. KPI performance is affected by expected levels of service delivery being signed into grants and intervention level grant performance. Countries representing 90% of investment will report semi annually with annual reports from whole portfolio. Projections could fill gaps for semi-annual reporting Data is reported with a six-month lag 18

20 Op-KPI Strategic targets Examples of more attributable Operational KPIs that aim to monitor whether: Strategic level service delivery targets are built into grant agreements Programs are able to deliver the expected levels of service Programs are able to expend the funds budgeted for service delivery Examples of potential Operational KPIs Percentage of expected service delivery targets: i. Built into concept note submissions; ii. Signed into grant agreements Aim Portfolio level targets of KPI 2 will be aggregated from estimates of country level service delivery. The first step in ensuring these targets will be achieved will be to ensure that expected service delivery is built into concept notes and grant agreements Percentage of program level service delivery targets achieved This will enable service delivery performance to be disaggregated and more effectively tracked at the regional & country level Intervention level absorptive capacity (grant expenditure / grant agreement budget) An early warning of potential underperformance on service delivery will be underspending in grants at the intervention budget level 19

21 Investing to End Epidemics strategic objectives Build Resilient & Sustainable Systems for Health Maximize Impact Against HIV, TB and malaria Mobilize Increased Promote and Protect Human Rights & Gender 1. Innovative approaches to meet diverse country needs are essential to accelerate the end of the epidemics a) Scale-up evidence-based interventions with a focus on the highest burden countries with the lowest economic capacity and on key and vulnerable populations disproportionately affected by the three diseases b) Evolve the allocation model and processes for greater impact, including innovative approaches differentiated to country needs c) Support grant implementation success based on impact, effectiveness, risk analysis and value-for-money d) Improve effectiveness in challenging operating environments through innovation, increased flexibility and partnerships e) Support sustainable responses for epidemic control and successful transitions 20

22 KPI 3 Strategic Vision Alignment of investment with need Further improve alignment of investments with country need Measure Alignment between investment decisions and country "need"; with need defined in terms of disease burden and ability to pay Aim of indicator The measure tracks the extent to which the Global Fund is able to rebalance the grant portfolio to effectively invest funds in the countries where need is greatest. The current replenishment period has seen a major improvement in rebalancing, providing a solid basis for further improvements going forward. Performance is driven by the design of the allocation model and the ability of countries, particularly those with high burden and low economic capacity, to use allocated funds. Limitations & mitigation actions Accuracy of target setting will be determined by the Mid-Tem Plan three year financial forecast Country need is determined by the allocation model. The feasibility of revising the disease burden metric of the model is being assessed GNI has limitations as a measure of ability to pay in the health and development context relevant outcomes of the Equitable Access Initiative may be considered by the SIIC as it reviews the allocation methodology, specifically with respect to economic capacity 21

23 KPI 3 Alignment of investment with need Indicator dimensions Dimension Accountability Data availability Actionable & reliable results Coverage Attribution System Full portfolio Comment Illustrates extent to which grant expenses are committed to countries with most need, and not necessarily those with best absorption. Provides a control for KPI7 Fund utilization Projection available Based on Mid-Term Plan commitments forecast Frequency of reporting Quarterly 22

24 KPI 4 Strategic Vision Aim of indicator Investment efficiency Increase the efficiency of program design to maximize impact of fund investments Increased use of disease impact models to improve the design of country-level programming will increase value for money of grant investments this indicator will track the these gains The indicator will close of the loop by linking grant level programmatic targets and investments with strategic targets which will be set using partner supported disease impact models The indicator provides a strong link to the objective on strengthening national strategic plans, and provides an opportunity to link this modelling effort to cost-effective service modalities (e.g. community based care models) and the work on program level quality Measure Improvement in cost per life saved or infection averted from supported programs Limitations & mitigation actions Limited partner capacity currently available to support country level modelling A new regional approach to providing country support is being implemented through the Value for Money Special Initiative Costing data needs to be improved for this exercise to become effective A Cost consortium partnership with the Gates foundation has recently started with aims to address these gaps 23

25 KPI 4 Investment efficiency Indicator dimensions Dimension Comment Accountability Contribution Focus is on the national strategic plan, not Global Fund specific Data availability Structured Building disease modeling into grant processes will increase data availability Actionable & reliable results Coverage Portfolio segment Close country engagement during modelling process will increase scope for taking action Initial focus on high impact portfolios, but there is also a demand for modelling in UMICs Projection available The KPI result is already a projection Frequency of reporting Semi annually Countries will mostly use disease models during Concept Note and/or NSP development there may be one data point every 3 years per country assessed 24

26 KPI 5 Strategic Vision Aim of indicator Service coverage for key populations Reduce the number of new infections in key and vulnerable populations disproportionately affected by the three diseases Indicator will track provision of comprehensive HIV prevention services to specified key population groups These groups face the double burden of low coverage of prevention services and high rates of infection. Increased coverage of preventions services will be essential to accelerate the end of the epidemic Indicator builds on work undertaken during the current strategy to measure the size of key populations in 55 countries Proposal has close links to the strategic objectives in SO3 on removing barriers to accessing services, SO2 on data systems Measure Coverage of comprehensive prevention services in key populations Limitations & mitigation actions Considerable data limitations: Advice from technical partners indicates that data on new infections is not readily available or may lack in sufficient quality to use to track HIV incidence in key populations, and treatment provision to key populations is not tracked Technical partners suggest to track service coverage for these population groups A number of issues are currently being addressed: methodology of coverage measurement; how to bridge data gaps between surveys; potential risk of harm to these populations and minimization of that risk; country coverage Indicator focuses on HIV only New Global Plan for TB has a focus on key populations, but work remains at an early stage of development 25

27 KPI 5 Service coverage for key populations Indicator dimensions Dimension Accountability Data availability Actionable & reliable results Contribution Not yet available Comment No routine data collection exists at country level either through surveys or programmatic monitoring Coverage Portfolio segment Projection available? Frequency of reporting Annually Countries will be selected from the 55 expected to have key population size estimates by the end of 2016 (HIV only) If baseline coverage data available, projections may be possible based on availability of service delivery targets in Performance Frameworks focusing on key population interventions Each reporting period will include few new data points as countries may only conduct surveys every 2-3 years 26

28 Op-KPI Maximize impact against HIV, TB & malaria Examples of more attributable Operational KPIs that aim to monitor whether: Disease impact modelling is being used to inform national program design Programs are adequately investing in and delivering services targeting key populations to maximize impact against the three diseases Examples of potential Operational KPIs Percentage of prioritized countries where country level modelling has been employed to maximize the impact of program design Percentage of prioritized countries with a comprehensive package of interventions for key populations built into program design Percentage of program level key population service delivery targets achieved. Absorptive capacity for key populations interventions and population groups (grant expenditure / grant agreement budget) Aim Country disease impact modelling, with full engagement of country level actors, will ensure that program design is informed by the best available evidence Tracks roll out of key population programming across the portfolio This will enable service delivery performance to be disaggregated and more effectively tracked at the regional & country level An early warning of potential underperformance on intervention coverage for key populations will be underspending in grants on these services 27

29 Investing to End Epidemics strategic objectives Build Resilient & Sustainable Systems for Health Maximize Impact Against HIV, TB and malaria Mobilize Increased Promote and Protect Human Rights & Gender 2. Build Resilient and Sustainable Strengthening systems for health is critical to attain universal health coverage and to accelerate the end of the epidemics a) Strengthen community responses and systems b) Support reproductive, women s, children s, and adolescent health, and platforms for integrated service delivery c) Strengthen global and in-country procurement and supply chain systems d) Leverage critical investments in human resources for health e) Strengthen data systems for health and countries capacities for analysis and use f) Strengthen and align to robust national health strategies and national disease-specific strategic plans g) Strengthen financial management and oversight 28

30 KPI 6 Strategic Vision Aim of indicator Strengthen systems for health Increase the share of countries with resilient and sustainable national systems for health that meet standards for use by Global Fund programs Indicator should provide portfolio-level view of national health system strength based on explicit risk, functional and quality standards Measure should improve alignment between risk assessment and systems strengthening investments Indicator provides a common metric for comparing quality of systems differentiated standards for systems would align with aid effectiveness and IHP+ principles and could link to strategic objectives on sustainability and transition This measure will aggregate data from a number of linked operational KPIs (see slides 33-45) providing a more granular assessment for each of the three sub-systems Measure Share of the portfolio that meet expected standards for: a) Procurement and supply chain systems b) Financial management systems c) Data systems and analytical capacity Limitations & mitigation measures Agreement will be needed internally and with relevant partners on expected functional and quality standards Careful consideration will be needed to ensure that definitions and standards are relevant to country context, in particular for procurement systems where potential incentives to exit pooled procurement will need to be countered Data collection mechanisms do not yet exist or will require considerable revision for the linked operational KPIs A clearly defined control structure for signing off systems as compliant would need to be developed and implemented to limit potential gaming 29

31 KPI 6 Strengthen systems for health Indicator dimensions Dimension Accountability Data availability Actionable & reliable results Coverage Contribution Not yet available Portfolio segment Comment Systems do not yet exist for some of this data. Health facility assessments & PQR may provide some data Measure represents aggregate measure for strategic level reporting. Operational indicators should drive performance improvement in each focus area Cohort of countries sampled may differ across the three focus areas Projection available Likely no projections will be available for forecasting system improvements Frequency of reporting Annually Each reporting period may include few new data points for some elements, as countries may only conduct health facility assessments every 2-3 years 30

32 KPI 7 Strategic Vision Aim of indicator Fund utilization Increase the rate at which countries effectively absorb allocated funds A resilient and sustainable system for health should be able to effectively use the full allocation of funds to deliver services to increase program impact. This will be measured in two ways: a) Allocation utilization provides high level view on the extent to which: countries can use their allocation, and the Secretariat can optimize portfolio level investments b) Absorptive capacity measures whether programs can spend the budgeted funds measure will focus on the 20 ITP countries with strong links to Supply Chain Investment work and other absorption related initiatives Measure a) Allocation utilization Grant Expense (actual + MTP) / Allocation b) Absorptive capacity Cum. expenditure / Cum. grant agreement budget Limitations & mitigation measures The Allocation utilization indicator risks two negative incentives: 1. Over-commitment to meet allocation Risk controlled by indicators tracking absorption and cash balance (absorptive capacity, fund absorption rate, cash balance) 2. Re-direction of funds through portfolio optimization from portfolios with the greatest need to portfolios better able to absorb funds without dealing with underlying health system constraints Risk controlled by KPI-3 on alignment between investments and need 31

33 KPI 7 Fund utilization Indicator dimensions Dimension Comment Accountability Mix Allocation utilization: attribution; Absorptive capacity: contribution Data availability Actionable & reliable results Coverage Projection available Frequency of reporting System Mix Mix Quarterly Sourced from existing systems for grant expense actuals and forecasts, as well as Enhanced Financial Reporting systems to provide expenditure data Directly highlights absorption problems as they arise and guides portfolio optimization efforts Allocation Utilization will cover whole portfolio, while Absorptive Capacity will cover only selected countries Allocation Utilization will be forecast as part of the Mid-Term Plan, but Absorptive Capacity is based on country expenditures and cannot be forecast 32

34 Example Operational KPIs for RSSH Note: The proposed Corporate KPI 5 Strengthen systems for health has been designed to aggregate the results of a series of operational KPIs. For that reason these operational KPI proposals have been further developed as set out in this next section. This information would be complemented by expenditure tracking for each of the RSSH three focus areas. 33

35 Op-KPI Strengthen systems for health: a) procurement Strategic Vision Countries have sufficient procurement capacity to achieve improved procurement outcomes Aim of indicator Ensure that procurement capacity is actually delivering improved outcomes in terms of prices, on-time delivery and lead time Focus procurement capacity-building efforts on delivery of results, rather than delivery of service Should lead to OIG country audits identifying procurement activities as major area of concern is decreasing Measure Improved outcomes for procurements conducted through countries national systems: a) Price; b) OTIF delivery; c) Administrative lead time Limitations & mitigation measures PQR could be primary data source, but this would mean delays in data reporting (sometimes up to one year) May be challenging to track administrative lead time without additional data request to country Outcomes can be impacted by factors outside the procurer s control (e.g. changes in market conditions for API can impact price or OTIF) Could consider amending trigger for data entry to PQR Compare country outcomes to international benchmarks 34

36 Op-KPI Strengthen systems for health: a) procurement Indicator dimensions Dimension Accountability Data availability Actionable & reliable results Coverage Projection available Frequency of reporting Contribution Not yet available Portfolio segment Annual Comment Currently data systems only exist for countries participating in PPM. Price data, OTIF and Lead Time could be sourced through PQR, though with up to one-year lag Countries that participate in procurement and capacity building programs. Issue also of considerable relevance to UMICs Projected prices may be available, OTIF and Lead Time forecasts may not be possible Data points available at each reporting period may be limited given PQR and OTIF data coverage 35

37 Op-KPI Strengthen systems for health: b) supply chains Strategic Vision Health facilities have the tracer medicines and diagnostics required to deliver high quality services for AIDS, TB & Malaria Aim of indicator To measure the extent to which investments in strengthening the different components of health product management systems contribute to the uninterrupted availability of essential health products at service delivery points This is based on the mean availability score for tracer items, and aligned with the recommendations of the interagency task force. Diagnostic services readiness (i.e. the capacity of the health facility to provide laboratory diagnostic services) based on a list of tracer lab items Measure a) Percentage of health facilities with tracer medicines available on the day of the visit b) Percentage of health facilities providing diagnostic services with tracer items on the day of the visit Limitations & mitigation measures Health facility assessments would provide data every two years, other systems may exist in country but data quality is uncertain Range of tracer items is country specific, may include items for programs other than Stock out on the day is a Y/N; not a measure of stock-out days Harmonize indicator measurement (guidance on tracer items & calculation of availability) Exploring potential option to collect additional, more frequent measures (stock outs, expired medicines) through sentinel surveillance as part of Assurance Model to complement health facility assessments every 3 years 36

38 Op-KPI Strengthen systems for health: b) supply chains Indicator dimensions Dimension Accountability Data availability Actionable & reliable results Contribution Not yet available Comment Some data may be available through health facility assessments, but likely not at the frequency required for this indicator to be useful Depending on system developed for data collection, results should reveal where supply chains are in need of strengthening Coverage Projection available Portfolio segment Frequency of reporting? tbc Likely data will only be available from countries conducting health facility assessments or similar surveys 37

39 Op-KPI Strategic Vision Aim of indicator Strengthen systems for health: c) financial management Implementer financial management systems in key countries are sustainable and meet standards for use by Global Fund programs Financial management is a critical process and enabler in delivering program impact from investments. The indicator goals are as follows: Indicator (a): will focus on use of country systems (i.e. elements of public financial management system) and will measure #/% of PFM system components used by GF in concert with development partners under the IHP+ framework in 15 high priority countries Indicator (b): will address routine financial management capacity gaps outside PFM scope in indicator (a) and will measure #/% qualified audits; #/% of timely & accurate financial reports (AFR, quarterly expenditure reports); #/% of qualified finance personnel Measure a) Number of public financial management transition efforts in high priority countries completed b) Number of countries with financial management systems meeting defined standards for optimal absorption & portfolio management Limitations & mitigation measures Partner buy-in for use of country systems may not be forthcoming Effective engagement within IHP+ to influence other partners and foster harmonization In-country capacity, ownership and co-ordination is a challenge for public financial management initiatives Mobilize IHP+ partners to conduct Joint Financial Management Capacity Assessments (JFMAs) with consolidated action plans; and engagement of CTs for follow up of action plans Establishing baseline data for PFM performance may be a challenge Co-ordination with IHP+ including in-country studies to collect required data Weak PR capacity & coordination in capacity building Co-ordinate approach to provision of financial management technical assistance & provide clarity on requirements (e.g. financial management handbook for implementers) 38

40 Op-KPI Strengthen systems for health: c) financial management Indicator dimensions Dimension Accountability Mix Indicator a: Contribution; Indicator b: Attribution Data availability Actionable & reliable results Coverage Projection available Frequency of reporting Not yet available Portfolio segment Semi Annually? System to track this information is not yet available Assessment of national system strength should guide investment and system strengthening focus in the selected countries priority countries will be targeted Each reporting period will include few new data points as countries may only conduct assessments periodically (tbc) 39

41 Op-KPI Strategic Vision Strengthen systems for health: d) HMIS coverage Well functioning HMIS enables better decision making and ultimately better programs Measure Percent of high impact countries with fully deployed (80% of facilities reporting for combined set of indicators), functional (good data quality per last assessment) HMIS Aim of indicator While many Global Fund supported countries can now report on key indicators for, many are still reliant on survey data or global estimates. To really be able to use data for program improvement, data need to be available routinely. Significant investments are being made to building these routine HMIS and this indicator measures the success of these investments The indicator is designed to measure two key aspects of an HMIS system that have proven problematic in the past coverage (fully deployed) and functional (defined in terms of data quality) This also aligns with international priorities set out at the Measurement Summit in June by Global Health Leaders Limitations & mitigation measures This indicator does not measure data use directly, rather the completeness and functionality of the information systems needed for facilities/districts to have access to the data and tools to use the data This aligns well with the significant GF investments into deploying and improving these systems, as well as with international priorities This KPI option s alignment with investments and priorities, and its feasibility to measure and reflect change, make it a preferred option of for this KPI. However, the data are still more likely to be available for high impact and core countries than focus countries Strengthening HMIS is a priority workstream of the Global Collaborative from the Measurement Summit, and specific tasks are planned around this 40

42 Op-KPI Strengthen systems for health: d) HMIS coverage Indicator dimensions Dimension Accountability Data availability Actionable & reliable results Coverage Projection available Frequency of reporting Contribution Not yet available Portfolio segment Annually HFAs and HMIS desk review are proposed as primary data sources Assessment of HMIS coverage and data quality should guide investment and data system strengthening focus in the countries monitored Data will be more prevalent in High Impact and core countries 41

43 Op-KPI Strategic Vision Aim of indicator Strengthen systems for health: e) ability to report on key indicators Countries are able to report on the minimum set of outcome and impact indicators to enable country monitoring and meet international commitments A set of high level indicators have been selected for each diseases/program area that measure key aspects of the program in terms of measuring performance. It is critical that GF-supported countries have this minimum set of data for their own purposes to understand the epidemic and their programs, as well as for Global Fund (and other donors) to understand basic performance and any changes to the epidemiology The indicator will include the ability of countries to report data disaggregated by age and sex, and to report on selected key populations indicators Gaps remain even within the High Impact countries ability to report on these indicators. This indicator aims to bring attention to this issue for PRs and key stakeholders Measure The percent of high impact and priority countries that were able to report on core set of HIV, TB, Malaria and RMNCH indicators Limitations & mitigation measures Establishing data systems or a survey to produce data is a longer-term process. This indicator may not show rapid change, but should show progress over time In the short-term, having the basic data is important, but this indicator doesn t demonstrate the source of the data. A country may be reliant on a survey or modelling for some data values, when quality routine data is preferable for program utility A comprehensive mapping has been done by country and by indicator in the High Impact and other priority countries to specifically identify gaps and resource needs GF is investing significantly in routine HMIS systems through grants 42

44 Op-KPI Strengthen systems for health: e) ability to report on key indicators Indicator dimensions Dimension Accountability Data availability Actionable & reliable results Coverage Projection available Frequency of reporting Contribution Structured Portfolio segment Annually Indicator will use existing grant performance frameworks complemented by additional data collection on key indicators. Current systems do not track country s ability to report data disaggregated by age and sex Indicator will likely assess High Impact and core countries Cohort of countries assessed at each reporting period will change depending on expected update schedule 43

45 Op-KPI Strategic Vision Strengthen systems for health: f) use of data Countries not only have data available, but also use those data to improve planning, budgeting, allocation process and quality improvement Measure Number of countries that are using WHO DQR or PRISM tools to analyze their data, and/or are putting into place quality improvement monitoring Aim of indicator To track the extent to which countries have the data systems and the capacity for analysis and decision making to improve the efficiency and effectiveness of service provision Ethiopia, Rwanda and Senegal cited as examples to inform development of this strategic objective Limitations & mitigation measures Indicator does not measure true use of data, but rather the country s use of data quality and data use tools Periodic surveys would provide data on this KPI every two years, and for only a portion of the portfolio As a complement to this KPI, evaluations every three years on data use for in-country decision making may provide additional information beyond the use of tools There are current discussions of harmonizing and incorporating data use measurement into the WHO s Data Quality Review tools which the Global Fund is supporting - this would provide an option for measuring data use across high impact countries in the long term 44

46 Op-KPI Strengthen systems for health: f) use of data Indicator dimensions Dimension Accountability Data availability Contribution Not yet available Comment Annual desk reviews and PR interviews will provide indicator data Actionable & reliable results Results should guide future investments in health systems Coverage Projection available Frequency of reporting Portfolio segment Annual Evaluations would be rolled out only in High Impact portfolio, for example Additional outcome evaluations on data use for decision-making could provide information on the efficiency and effectiveness of program management 45

47 Investing to End Epidemics strategic objectives Build Resilient & Sustainable Systems for Health Maximize Impact Against HIV, TB and malaria Mobilize Increased Promote and Protect Human Rights & Gender 3. Human Rights and Gender Promoting and protecting human rights and gender equality is required to accelerate the end of the epidemics a) Scale-up programs to support women and girls, including programs to advance sexual and reproductive health and rights b) Invest to reduce health inequities including gender- and age-related disparities c) Introduce and scale-up programs that remove human rights barriers to accessing services. d) Integrate human rights considerations throughout the grant cycle and in policies and policy-making processes e) Support meaningful engagement of key and vulnerable populations and networks in Global Fund-related processes 46

48 KPI 8 Gender & age equality Strategic Vision Reduce gender and age disparities in health Measure HIV incidence in women aged Aim of indicator HIV infection rates among young women are twice as high as among young men in some regions. The indicator will track the extent to which an enhanced programmatic focus on women and girls results in a reduction in new infections in selected countries with large disparities in incident infections This objective is closely linked to other strategic objectives focused on scale-up of programs supporting women and girls; advancing sexual and reproductive health and rights; support to women s, children s, and adolescent health; and removing barriers to access Limitations & mitigation measures Considerable data limitations: Advice from technical partners indicate that current modelled data on numbers of new infections can indicate a trend but not a statistically significant change attributable to an enhanced focus on women and girls Technical partners recommend use of improved sex and age disaggregated models that are currently under development and should provide data from Any targets approved in 2016 may need to be recalibrated in 2017 Data are reported with a one year lag Projection models would need to be developed to enable effective tracking 47

49 KPI 8 Indicator dimensions Gender & age equality Dimension Accountability Data availability Actionable & reliable results Contribution Structured Comment Sourced from UNAIDS, modelled data of new incident cases, reported with a one year lag Coverage Portfolio segment Projection available? Frequency of reporting Annually Cohort of countries to be defined based on where disparities are greatest Projections will be required if performance on this objective is to be effectively managed With a one year lag emphasizing the need for good projections and complementary data 48

50 KPI 9 Strategic Vision Human rights: a) Reduce human rights barriers to services Human rights barriers to services are reduced, resulting in improved uptake of and adherence to treatment and preventions programs Measure # of priority countries with comprehensive programs aimed at reducing human rights barriers to services in operation Aim of indicator With a focus on priority countries this indicator will measure the extent to which comprehensive programs to tackle human rights barriers to access are established The programs will be designed around the 7 key interventions to reduce stigma and discrimination and increase access to justice of UNAIDS Where available, established WHO indicators for assessing enabling environments will be used to track progress in operationalizing the interventions The aim is that these programs will contribute to a meaningful reduction in human rights barriers to services and that increased access will lead to increased impact. This will be measured through in-depth evaluations as baseline in 2016, at mid-term in 2019 and at the end of the strategy period in 2022 Limitations & mitigation measures Human rights interventions to reduce barriers to service are well defined for HIV, but more work is needed for TB & Malaria (Malaria will be conducted as a second phase in 2017) Specific indicators to track progress beyond those proposed by WHO need to be defined and tracking systems to collect the relevant data will have to be established in countries The in-depth evaluations will use mixed-method assessment of human rights barriers and interventions that reduce barriers. This is a relatively new idea with respect to health programs, but a solid and program-relevant assessment method could set a useful precedent for such assessments for policy-makers and program practitioners The Global Fund has a key niche as the major funder of interventions aimed at removing legal barriers to access. There is strong backing from partners for this work WHO, UNAIDS, OSF, Ford 49

51 KPI 9 Human rights: a) Reduce human rights barriers to services Indicator dimensions Dimension Accountability Data availability Actionable & reliable results Coverage Projection available Contribution Not yet available Portfolio segment Comment The global fund has a key niche role as the major funder of interventions aimed at removing legal barriers to access, but partner support remains key Considerable work will be required to build the necessary tracking systems The country dialogue and Concept note development stages will be key, as will be using the time before then to start building capacity and partnerships. After these stages of the funding process, building human rights programs into grant design will become increasingly difficult Countries will be selected across different epidemic, burden, conducive environment and barrier contexts Frequency of reporting? tbc 50

52 KPI 9 Strategic Vision Human rights: b) key populations and human rights in middle income countries Increase programing for key populations and human rights in middle income countries Measure Percentage of country allocation invested in programs targeting key populations and human rights barriers to access in middle income countries, for: i) Generalised epidemics ii) Concentrated epidemics Aim of indicator To ensure a successful transition, comprehensive programs for key populations and human rights will have to be established in middle income countries before they reach this stage This objective could be operationalized by enhancing the focus of stipulations in funding policy that a set percentage of the allocation for countries be invested in programs targeting key populations and human rights barriers to access Limitations & mitigation measures These plans would have to be integrated in to funding, eligibility and domestic investment policies 51

53 KPI 9 Human rights: b) key populations and human rights in middle income countries Indicator dimensions Dimension Comment Accountability Data availability Actionable & reliable results Attribution Structured A KPI will not be required if funding policy is sufficiently specific and rigorously enforced Coverage Portfolio segment Projection available? Frequency of reporting Quarterly Middle income countries A projection should be possible when new grants, subject to these stipulations, come into force 52

54 KPI 9 Human rights: c) key populations and human rights in transition countries Strategic Vision Upper middle income countries in transition take over programing for key populations and human rights Measure Percentage of funding for programs targeting key populations and human rights barriers to access from domestic (public & private) sources Aim of indicator To measure the extent to which, in upper middle income countries transitioning out of Global Fund support, governments recognize that support to services for key populations is essential and increasingly take over responsibility for and funding of these services This would allowing remaining external funding to be used to support initiatives that support effective transition Limitations & mitigation measures These plans would have to be integrated into funding, eligibility and domestic investment policies Initiatives that support effective transition will be defined in early 2016 and tracked at the operational level Criteria would be required to define countries in transition e.g. transition expected within 10 years In some countries, even sustained efforts are unlikely going to lead to governments to take over funding of services for key populations and human rights programs 53

55 KPI 9 Human rights: c) key populations and human rights in transition countries Indicator dimensions Dimension Accountability Contribution Data availability? Comment Tracking may be possible through UNAIDS sources and through counterpart financing and willingness to pay mechanisms Actionable & reliable results If suitable policy levers to enforce compliance can be introduced Coverage Projection available Portfolio segment Cohort of countries to be defined. HIV and TB programs only Formal projections are unlikely to prove accurate, but risk of non-compliance could be tracked on a more qualitative basis Frequency of reporting Annually There will be a lag of months after the start of the replenishment period 54

56 Op-KPI Examples of more attributable Operational KPIs that aim to monitor whether: Interventions to tackle gender and age disparities and human rights barriers to access are being rolled out in prioritized countries Examples of potential Operational KPIs Percentage of females at higher risk, in target districts of priority countries, who completed a standardized HIV prevention intervention during the reporting period Aim A measure of service coverage for districts where the Global Fund is a significant funder of interventions targeting women & girls Absorptive capacity for interventions targeting: i. removing human rights barriers to access ii. women & girls (grant expenditure / grant agreement budget) An early warning of potential underperformance on human rights and women & girls will be underspending in grants on these services Note the planned in-depth country-level evaluations of human right barriers to access proposed for 2016, 2019 & 2022 will provide an additional source of information to understand the performance of the Global Fund in this area 55

57 Investing to End Epidemics strategic objectives Build Resilient & Sustainable Systems for Health Maximize Impact Against HIV, TB and malaria Mobilize Increased Promote and Protect Human Rights & Gender 4. Increased programmatic and financial resources from diverse sources are needed to accelerate the end of the epidemics a) Attract additional financial and programmatic resources for health from current and new public and private sources b) Support countries to use existing resources more efficiently and to increase domestic resource mobilization c) Implement and partner on market shaping efforts that increase access to affordable, quality-assured key medicines and technologies d) Support efforts to stimulate innovation and facilitate the rapid introduction and scale-up of cost-effective health technologies and implementation models 56

58 KPI 10 Resource mobilization Strategic Vision Increase the financial resources available to the Global Fund for investment in programs to tackle the three diseases Measure a) Actual pledges as a percentage of the replenishment target b) Pledge conversion rate. Actual 5 th replenishment contributions as a percentage of forecast contributions Aim of Indicator A key objective of the Global Fund is to mobilize resources for health from current and new public and private sources The indicator directly measures the volume of new pledges made, and the extent to which these pledges are fulfilled as contributions Limitations & mitigation measures The 5 th replenishment launch is scheduled before the KPI framework for the new strategy comes in to force. Therefore part a) will only be measured in 2019 The current measure tracks pledge conversion on an annual basis, which makes it sensitive to time shifts in contribution schedules Improved forecasting methodology, developed during the current replenishment period, will enable the measure to by tracked on a three year basis maintaining accuracy but reducing the potential for over-interpretation of small time shifts 57

59 KPI 10 Resource mobilization Indicator dimensions Dimension Comment Accountability Contribution Joint responsibility between Global Fund and donors to fulfil contribution agreements Data availability System Directly sourced from Hyperion Pledge cube Actionable & reliable results Coverage Full portfolio All donors are included in tracking Projection available Frequency of reporting Quarterly 58

60 KPI 11 Strategic Vision Domestic investments in programs for HIV, TB & malaria continue to increase over the replenishment period Aim of indicator Domestic investments An increase in domestic investments in programs for are required to accelerate the end of the epidemics and to foster sustainable programs The Global Fund directly supports these aims through advocacy and access to funding policy This indicator directly measures the extent to which domestic health commitments are fulfilled to meet this need Measure Percentage of domestic commitments to programs supported by the Global Fund realized as expenditures Limitations & mitigation measures Current policies on eligibility, counterpart financing and willingness to pay, including levers to drive performance need to be strengthened Internal roles and responsibilities on advocacy, monitoring and accountability for performance need to be clarified and aligned with expectations The indicator focuses on conversion of commitments into expenditures, not the scale of commitments Policy in this area is under review with the aim to align and streamline Data on domestic commitments will be tracked as management information 59

61 KPI 11 Domestic investments Indicator dimensions Dimension Accountability Data availability Contribution Structured Comment No system exists currently to track this data, but this will be addressed by Project AIM Actionable & reliable results Policy levers may need to be enhanced Coverage Projection available Frequency of reporting Full portfolio Annually 60

62 KPI 12 Strategic Vision Availability of affordable health technologies: a) Availability A stable supply of key quality-assured health products sufficient to meet country demand Measure Percentage of a defined set of products with more than two suppliers that meet Quality Assurance requirements Aim of indicator Ensure that supply is available from multiple qualityassured manufacturers, reducing risk of supply disruption Ensure a balance between decreasing prices and maintaining a secure, stable supply base Promote competition between suppliers for key products Align with Expert Review Panel requirements and processes Limitations & mitigation measures Even with more than two suppliers, manufacturing capacity may still be insufficient to meet demand, especially during peak times Manufacturing capacity is estimated / self-reported by suppliers and difficult to validate (as is global demand) Estimated manufacturing capacity vs. forecast annual demand (GF and global) will be monitored as management information 61

63 KPI 12 Availability of affordable health technologies: a) Availability Indicator dimensions Dimension Comment Accountability Contribution Data availability Structured Underlying data is currently tracked, but requires manual synthesis Actionable & reliable results Coverage Projection available Full portfolio Where risks are identified, a number of actions can be taken, such as: proactive outreach to potential new suppliers, targeting ERP, reserving volumes for new entrants in PPM tenders, employing mechanisms to incorporate new entrants in existing PPM tenders As a global indicator, all countries will be included; however, only select products will be tracked Frequency of reporting Annually 62

64 KPI 12 Strategic Vision Availability of affordable health technologies: b) Affordability Market shaping efforts reduce prices for PRs accessing PPM framework agreements, yielding savings which can be used to support unfunded programmatic needs Measure Annual savings achieved through PPM Framework Agreements on a defined set of key products (mature and new), compared to the prior year Aim of indicator Captures effectiveness in increasing the affordability of key medicines and technologies Reflects achievement of target savings based on tenders conducted in the prior year and forecast demand Takes into account market conditions for different products in the PPM portfolio (e.g. anticipate greater savings on new product strategies or recently introduced products) Limitations & mitigation measures The measure does not capture affordability of products in countries that do not access PPM framework agreements Operational KPIs measuring RSSH achievements will provide information for these countries Cost avoidance (slowing the rate of cost increases) is not covered due to difficulties in validating estimates If considered alone, the indicator could lead to negative incentives for product availability - driving reduced supplier base and reduced investment Benchmark reference prices for key products will be tracked as management information KPI 12a will be used to control for potential negative effects on availability; management information will also track additional risk indicators 63

65 KPI 12 Indicator dimensions Availability of affordable health technologies: b) Affordability Dimension Accountability Data availability Actionable & reliable results Coverage Attribution Structured Portfolio segment Comment Underlying price data is captured via current systems, but requires manual analysis. Forecast volumes are not captured via current systems Target setting will be closely linked to anticipated volumes by product from PPM, e-marketplace and other channels Countries that access PPM framework agreement prices as part of PPM, e- marketplace or through other channels Projection available This should be feasible with development of data capture system for LoHPs Frequency of reporting Annually Framework agreements are typically negotiated for a multi-year period and therefore, little value added by more frequent reporting. In addition, large orders in particular quarters can skew results if reported partway through the year. 64

66 Op-KPI Examples of more attributable Operational KPIs that aim to monitor whether: Funding policy is supporting countries to increase domestic resources for HIV, TB & malaria Supported programs are able to access affordable, quality-assured key medicines and technologies Examples of potential Operational KPIs Percentage of priority countries with national health financing strategies Percentage of budgeted unit prices within expected benchmarks for targeted products Percentage of demand forecasts for key products with acceptable variance to actual demand Aim National health financing strategies will be developed with prioritized countries to better understand their capacity to increase funding to HIV, TB & malaria programs This indicator measures extent to which reference prices are disseminated and grants are taking advantage of price reductions achieved by the Global Fund, including grants purchasing outside of PPM This indicator assesses the Global Fund s ability to predict demand for health products, which enables lower prices and more consistent product availability 65

67 Annex 66

68 Timeline to Board Approval for the Corporate KPI Framework End November: Agreement internally on expectations for all SOs Nov-March: Consultations with technical partners 21 December: 1 st MEC review and revision 11 January: 2 nd MEC review and revision 22 January: Circulate draft framework for constituency input Jan-Feb: Consultation meetings with donor and implementer groups 29 February: 3 rd MEC review Feb-March: Revision and review of pre-final proposal by Committees End-March: 4 th MEC review Early April: Final revisions to indicator selection & submission to Board for approval May-August: Finalization of indicator methodology, baselines and targets Autumn: Submission of full final framework to Board for approval 67

69 KPI Framework follows a logic model from inputs to impact Attract funding Invest efficiently Bridge investments to services Address key challenges Deliver impact 10: Resource mobilization 11: Domestic investment 3: Alignment of investment with need 4: Investment efficiency 12: Availability of affordable health technologies 6: Resilient & sustainable systems for health 7: Fund utilization 5: Key population service coverage 8: Human rights 9: Gender & age equality 1: Performance against impact targets 2: Performance against service delivery targets This alignment, complemented by Operational indicators, should provide early warning system for tracking performance and taking management action 68

70 Focus the KPIs on needs for strategic level oversight Strategic KPIs Strategic goals Strategic targets 2-3 KPIs measuring programmatic results and impact Strategic Objectives Build resilient & sustainable systems for health Promote and protect human rights & gender equality Mobilize increased resources Activity specific KPIs 8-9 KPIs tracking progress on strategic objectives Operational KPIs 10 more granular measures guiding management action Rationalize, standardize and align routine reporting to MEC, the Committees and the Board 69

71 Corporate KPI development is one component of a larger exercise to measure performance at multiple levels Element Aim of measure Main audience Reporting Current Focus Corporate KPIs Operational KPIs 12 indicators, including Strategic KPIs to measure progress towards the impact targets, and activity specific KPIs to measure progress against the Strategic Objectives 10 indicators to measure progress towards meeting Corporate KPIs Board; Committees; MEC MEC; Secretariat Compiled 1-4 times per year Compiled 4 times per year Thematic reporting Metrics and qualitative reporting for elements out of KPI scope of measurement; ideally drawn from management information package Board; Committees; MEC Compiled biannually Management information Package of data reviewed at division and team level to better understand performance on day-to-day basis Secretariat; MEC as necessary Regular monitoring at divisional level Corporate KPIs are only the leading edge of structured routine reporting 70

72 Zoom in on Management Information & Thematic Reporting Strategic corporate KPIs 1 Performance against impact targets 2 Performance against service delivery targets Existing standard reporting Example gaps in standard reporting Strategic Objectives Build resilient & sustainable systems for health Promote and protect human rights & gender equality Mobilize increased resources Corporate KPIs 3 Alignment of investments & need 4 Investment efficiency (as recently submitted to SIIC/FOPC) 5 Service coverage for key populations 6 Strengthen systems for health 7 Fund utilization a) Allocation utilization b) Absorptive capacity 8 Human rights 9 Gender & age equality 10 Resource mobilization Domestic investments (current OKPI) Availability of affordable health technologies Operational KPIs Management Information & Thematic Reporting Portfolio Report (pipeline, investments, results, impact)? Key population report? Indicators tracking progress towards Corporate indicators Available to Board through corporate dashboard To be developed to align with approved corporate framework in 2016 Financial oversight RSSH report? Human rights report? Gender report? Regular updates: Risk management; Human updates Resource mobilization update Market Shaping report? Standardized thematic reporting under each Strategic Objective should provide consistent, quantitative & qualitative information complementary to KPI Framework Major reporting gaps currently exist 71

73 Pyramidal hierarchy is in line with other development organizations AfDB/AsDB uses four levels of indicators to track performance Levels 1 & 2 correspond to the Strategic and Activity specific levels of Global Fund s pyramid, and levels 3 & 4 correspond to operational KPIs and management information Corporate level Secretariat level Strategic KPIs Activity specific KPIs Operational KPIs Thematic Reporting & Management Information GAVI uses 3 mission indicators and 11 Goal-Level KPIs to report performance against Strategy Pyramidal Monitoring & Evaluation strategy fills operational-level information gaps by monitoring progress at country level 72

74 Global Fund s Pyramidal KPI approach is aligned with other development organizations Global Fund Development Banks GAVI Strategic Strategic KPIs measure Global Fund s progress against the highest level impact targets: Lives Saved; Infections Averted; Services delivered Level 1 monitors development progress against the two overarching goals: inclusive growth and the transition towards green growth Mission indicators measure progress towards overall mission: save children s lives and protect people s health by increasing access to immunization in poor countries Activity- Specific Activity-specific KPIs capture the Global Fund s progress towards achieving the four Strategic Objectives: Maximize Impact against the diseases; Build resilient & sustainable systems for health; Promote and protect human rights & gender equality; Mobilize increased resources Level 2 captures the Bank s five main channels to deliver its work and improve the quality of growth in Africa: infrastructure development, regional integration, private sector development, skills and technology, and governance and accountability Goal-level indicators capture progress towards achieving the organization s four Strategic Goals: Accelerate uptake; Strengthen capacity of systems; Increase financing sustainability; Shape vaccine markets Operational Management Information Operational KPIs tracking mostly input-level and MEC-level indicators to ensure higher level indicators are met Regular management reporting should fill information gaps by providing consistent, rationalized reporting to MEC, potentially Board-level. Examples: Risk report; HR updates; Financial reporting; Priority Project updates Level 3 tracks the quality and effectiveness of the Bank s operations Level 4 measures progress in important areas of internal reforms: decentralisation, engaging and mobilising staff, and improving cost-efficiency Management information, like the Global Fund s Operational KPIs and internal management information, is not publicly available. Pyramidal Monitoring & Evaluation framework provides countryand portfolio-level progress measurement 73