Innovation Centres Monitoring and Evaluation Framework (MEF) Introduction

Transcription

1 Innovation Centres Monitoring and Evaluation Framework (MEF) Introduction Innovation Centres (ICs) are long term investments and it is likely that the full economic impacts of the Centres on the Scottish economy will not be known for several years. It is important, therefore, for each Innovation Centre to put in place performance monitoring mechanisms that regularly capture and report progress against a range of indicators. The approach is designed to meet the following two objectives: to provide confidence that the Centre is on track (or not) to deliver value for money for the public purse and significant economic and wider impacts; to provide accurate information, captured on an ongoing basis, which can be used as important input into future evaluations of the Centre. This guidance sets out SFC s (and its partners) requirements of a monitoring and evaluation framework (MEF) for the IC programme. There are five components to the MEF: a standard approach to quarterly reporting to IC Boards and SFC; baselining; reporting on Strategic Investments (ROSI) online operating system; annual formal reviews; interim and full independent evaluations. Logic The logic behind delivering economic impacts is that impacts are the result of inputs (funding and time) leading to activities which in turn deliver outputs and outcomes (the benefits outputs deliver) and impacts. This logic chain can be expressed as shown below.

2 Inputs Activities Outputs Outcomes Impacts Standard approach to quarterly reporting The initial guidance to bid teams contained a list of core key performance indicators (KPIs). These have been updated and arranged into KPIs for inputs; activities; outputs and outcomes. Tracking and reporting these KPIs regularly over time should provide indicators that ICs are on track (or not), identify any remedial actions required, and will inform future evaluations to assess likely economic impacts. ICs may have additional KPIs and/or non-quantifiable indicators they wish to track. Appendix 1 is a generic monitoring template to be used by all ICs for monitoring and reporting progress. Page 2 of Appendix 1 comprises definitions for each KPI listed. The template may be added to or amended (within reason) to suit the individual needs of an IC. The monitoring template should be populated and submitted, as part of a Performance Monitoring Report, within 2 months of each quarter (following the academic year) see table one. Table one quarterly reporting schedule Quarter Period Report to SFC (following Board approval) 1 Quarter one August to October By December Quarter two November to January By March Quarter three February to April By June Quarter four May to July By September The quarterly Performance Monitoring Report should include the following sections: Executive Summary (1 page); Key Achievements in Quarter (1 page); Progress Report Narrative (2-3 pages); Achievements Forecast for Next Quarter (1 page); Financial Reporting 2 (1 page narrative plus spreadsheet); 1 ICs will have Board meetings scheduled up to a certain point which would be difficult to change. However, ICs may wish to align future Board meetings with this schedule. 2 To be clarified in line with recommendations arising from the current BDO commission for DHI, CENSIS and SMS-IC.

3 Completed Performance Monitoring Template (1 page spreadsheet - Appendix 1). The number of pages in brackets after each section is provided for guidance only. Quarterly reports should be concise and useful, not over-onerous. Whilst this is a requirement of SFC funding, it is important that the IC Board, through guidance from the IC Director, takes ownership of this level of monitoring. The Performance Monitoring Report should be incorporated into normal board processes and reported and signed off by the main Board prior to submission to SFC by the IC Director, or equivalent, within two calendar months of the end of the quarter. It is expected, as per SFC s standard conditions of grant, that the IC will secure and retain evidence to support performance claims and this will be available for inspection, if required. Appropriate evidence might include, for example, confirmation s from companies in support of claims; press releases; etc. The interim review and full evaluation (see later) will each follow up on claims. Baselining Establishing a baseline of activity for the sector or sub-sector addressed by an IC is important in order to assess the additionality which an IC brings. Some ICs have undertaken baselining as part of their business plan preparation but a requirement for all ICs will be that a baselining exercise on current activity levels/performance against KPIs is delivered. SE, on behalf of the partners, has commissioned a baseline exercise which is currently being run with the first cohort of ICs (DHI, CENSIS and SMS-IC) this is scheduled to report by October This exercise will be rolled out across the other ICs in due course. SFC Reporting On Strategic Investments (ROSI) In addition to the collection of key performance indicators and as specified in letters of grant, ICs, as is the case with all SFC funded strategic investments, are required by SFC to provide annual and final reports. From autumn 2014 these reports will be informed by an online monitoring and evaluation system called Reporting on Strategic Investments (ROSI). ROSI therefore forms part of the MEF.

4 ROSI uses a hosted data base service provided by ResearchFish ( Further information and guidance on ROSI will be provided to ICs by the SFC Evaluation Team in due course. The data collected by ROSI should inform the quarterly reports described above and vice versa. We will work closely with ICs to ensure both mechanisms are complementary. Annual formal reviews The annual formal reviews are important because they provide a useful feedback loop for both the ICs and the partners regarding progress of individual ICs and the programme more generally. We expect that the information supplied as part of the MEF, and any specific issues or themes the Panel wishes to cover, will provide the basis for discussion at these reviews and that a separate report template will not continue as is being used currently. This revised approach would, therefore, apply from August 2015 onwards. Evaluation It is planned that an interim review will be undertaken 2 years after start-up and a full evaluation after 4 years. The interim review, after 2 years, is undertaken primarily to provide an independent view on whether an IC is on track and to identify any specific areas requiring attention. Interim Reviews will also be used to help estimate likely future economic impact. The full evaluation is a more comprehensive evaluation on progress and impact, and will inform any decision making on possible future funding. Both the interim review and full evaluation will be undertaken by external consultants commissioned by SFC, SE and HIE, though taking into account the monitoring performance data and other intelligence secured by SFC, SE and HIE. Further information

5 Please contact Keith McDonald ( ; or Sophie Lowry ( ;