Annual Review - Summary Sheet

Transcription

1 Annual Review - Summary Sheet Title: International 2050 Pathways Partnerships and Global Calculator Programme Value: 2,409,585 ( 2,107,000 original + 302,585 extension) Start Date: 04/2015 End Date: 03/2016 Review Date: March 2016 Summary of Programme Performance Year Programme Score B A A Risk Rating Medium Medium Medium Summary of progress and lessons learnt since last review Summary of progress and lessons learnt since last review This year the 2050 Calculator programme has progressed well, as shown in the logical framework (logframe). Two more country Calculator models were launched (Mexico and Nigeria), improvements were made to previously launched Calculators, and the models began to be used in policy making and to provoke debate with stakeholders and within government. Four countries (India, Vietnam, Colombia and Nigeria) used their Calculator models in the formulation of their mitigation targets (known as Intended Nationally Determined Contributions or INDCs) for the UN climate change conference in Paris. In Vietnam, the model s results influenced the decision to stop giving licences for new coal power plants. There has been increased outreach work with finalised Calculators over the past twelve months, including stakeholder workshops in three countries. The South African government is using the tool in their climate change education strategy, which the ICF supported by funding the development of resources and training for teachers. The Global Calculator has gained new users and has been presented to audiences around the world. In addition, calculators have been used in research by universities, institutes and NGOs. The Calculator Community has continued to grow this year - countries are advising each other, and new projects have begun in both developing and developed countries and regions. An ICF-funded information sharing workshop was held in Mexico City in March 2016, attracting representatives of 26 countries (including all ten of the original ICF-funded countries). A number of important lessons have been learned: The first version of a country Calculator usually requires further development and refining to make it useful for policy makers and to carry out outreach work, showing that the initial timeline for the programme was too short. The impact of political events can result in milestones being missed the continued delay to the launch of the Brazil Calculator was due both to a severe drought, which led to the temporary redeployment of the Calculator team, and due to delays in announcing some energy-related policies, which need to be reflected in the Calculator. Missed milestones around improving the transparency of the models remain a concern. We have learnt from businesses that use the Global Calculator that it has less value as a planning tool, than as an educational tool, and this will help shape the future strategy for the Global Calculator. A significant challenge for the project this year has been staff resource, with two people in DECC managing the project and delivering some of the work, neither of whom are analysts. This resulted in an inability to provide dedicated technical support to Calculator teams, and contributed to delays in the completion of the project extension business case. Summary of recommendations for the next year The key recommendation is to develop a fully resourced work plan for the next two years, which should include: Securing dedicated analytical support for the project. A priority for these analysts will be quality assurance of the existing models to ensure consistency. Continued promotion of openness and transparency. A work strand on building the Calculator Community with the ultimate aim of making the project selfsustaining within two years. Global Calculator outreach work focused on using the tool to improve audiences overall understanding of effective climate action. 1

2 For any new countries, the initial project timeline should be extended to two years to allow for further improvements to the model and outreach work. Investigating whether it would be feasible and cost-effective to commission some basic My2050 software that could more easily convert any country s Excel model into a My2050 game. For the monitoring and evaluation of the project, it is recommended that the logframe be revised to represent the focus of the project in the next phase. A. Introduction and Context (1 page) Link to original business case: pathways-partnerships-and-global-calculator-business-case.pdf Link to extension business case: Link to logframe: Outline of the programme Following a project model developed with the Chinese government, DECC has provided direct technical and financial support to ten developing and emerging country governments to develop their own in-house version of the UK s 2050 Calculator - an interactive, open-source tool for developing energy and emissions scenarios. The aim is for these to be used to explore the low-carbon development pathways open to the countries, and to open up the debate to stakeholders and the public. The countries involved are: Bangladesh India Vietnam Thailand Indonesia South Africa Nigeria Colombia Mexico Brazil In addition, a small amount of money has been given to Algeria, Ecuador and Peru to provide them with initial training. When sought, advice has been given to developed countries, but no funding. The programme also involved building the Global Calculator, a 2050 Calculator covering the whole world that allows users to explore the options for reducing global emissions up to the year DECC led this work, which was co-funded by Climate-KIC and was carried out by a number of partner organisations. The Global Calculator is particularly aimed at businesses, NGOs and policy makers. It was launched in January 2015 with a report (titled Prosperous living for the world in 2050: lessons from the Global Calculator) outlining results from the tool that show a positive vision of a low-carbon future. In an initial budget of 200,000 was agreed. This was used to complete work on the final three country calculators that were awaiting publication, perform outreach for the Global Calculator, provide initial training for new countries, and support the further development of Calculators that had already launched. In October 2015 a project extension business case received ministerial approval. This was designed to support four country projects and a workshop in totalling 302,585, plus approval of up to 1 million over the next two years (subject to a new project plan being drawn up after business planning has been finalised). The five projects for were: India supporting the roll out of their Calculator around the country by holding workshops in five different states. Also supporting the development of their new National Energy Policy with knowledge exchange between DECC and the Indian Government, and producing resources for schools. Indonesia developing a cost sector of the Calculator so that the practical viability of each pathway can be explored by the Indonesian government. 2

3 South Africa improving the main Calculator tool by updating the data and assumptions and adding in example pathways. Supporting the roll-out of the My2050 version aimed at school children by developing lesson materials, and holding workshops for curriculum specialists and teachers around the country. Bangladesh holding workshops to train government officials and academics in how to use and maintain the Calculator so that it can be transferred from Cardiff University (which developed the Calculator) to Bangladesh. Also developing an app version of the Calculator for Android phones to improve access to the tool. The app has the potential to be adapted by other countries in the future. Workshop holding another international meeting of all the teams in early 2016 to build on the momentum of the Taipei conference. This would bring together project teams around the world to present their work, learn from each other, and to build relationships between teams. B: PERFORMANCE AND CONCLUSIONS (1-2 pages) Annual outcome assessment This has been a largely successful year for the 2050 Calculator programme, as progress has been made against all the active indicators in the logframe, and some significant results have been achieved. The project has met or exceeded around half of its outcome milestones. For the other half, in the majority of cases the milestones were only narrowly missed, achieving 80-90% of what was expected. In the impacts section, eight milestones were met or surpassed, and two were missed. There are no milestones in the logframe beyond March 2015 currently, but evidence this year suggests that the theory of change is sound Calculator models are being used by countries to articulate and explore their energy and emissions futures, and all ten country teams want to continue to use and improve their models in the future. There is now evidence that in five ICF-supported countries the Calculators are becoming part of the governments strategic decision making progress. For example, in India, Colombia, Vietnam and Nigeria the models were used in the formulation of the INDCs for the UN climate negotiations in Paris. In the other ICF-supported countries, the Calculator is not fully integrated into the policy making process, but the project teams are either working towards this, or they are using the tool for stakeholder outreach this year (one of the other main outcomes of the programme s theory of change). Other developing countries have expressed an interest in receiving support from DECC to start Calculator projects, showing demand for this service. The main risks to this progress continuing are: A lack of dedicated analytical support in the DECC programme team to advise the country-based teams and quality assure the work. The ability of countries to invest further in their Calculator projects. This year, most countries have required financial support from the ICF or the FCO s Prosperity Fund for outreach work or further model development, which were necessary for progress along the theory of change. The continuing risk that the governments involved could withdraw support for the Calculator projects, which is beyond our control. Work on the Global Calculator has focused on outreach this year following the launch in January 2015, and has had mixed results. There is a risk that this project s high-level impacts will not be achieved if the outreach strategy is not changed to reflect the new information about the tool s strengths and weaknesses. The indicators in the logframe will need to be amended accordingly. Overall output score and description A - Outputs met with expectations Key actions See recommendations in the cover sheet for a list of actions for next year. Outstanding actions from last year s annual review include: The Brazilian Calculator has still not been launched due to changing domestic political priorities in response to the county s severe drought, and due to a delay in the announcement of energy policies that need to be reflected in the Calculator model. This was out of DECC s control. At the end of Brazil re-started work to finalise the model, and hopes to launch in summer The DECC Calculator team were not able to formally secure technical assistance for the project and instead arranged ad hoc cover using ex-team members and members of the country Calculator teams we have trained. A few of the specific proposed Global Calculator activities were not completed due to time constraints, or because the DECC programme team became aware of better alternative options. For example, DECC did 3

4 not organise a series of workshops in the autumn for UK businesses, but did support one organised by the LSE that also gathered information for their research project into what information businesses want from climate scientists. Has the logframe been updated since the last review? The structure of the country Calculator and Global Calculator logframes were updated shortly after the last review during the process of developing a new business case for work. This involved setting milestones for March 2016 (previously most of the milestones ended in early 2015) and improving the quality of the indicators. The logframe was updated again in January 2016, merging the separate country and Global Calculator logframes into one. This followed discussion with the International Climate Fund Programme Management Office (ICF PMO) and Evaluation team, as it was felt that it was not proportionate to have a separate review process for the Global Calculator given both the size of the project, and the resources currently devoted to it. In line with this update the theories of change were also combined. Less relevant indicators from the original Global Calculator logframe were removed in this process. Results have been fed into this newly combined logframe in February and March This required sending out a questionnaire to FCO posts involved in the project and going through the project s records. Nine out of the ten FCO Posts returned the questionnaire. The Nigeria form was not returned due to personnel changes in the Abuja embassy, but was completed instead by the project leader from the Nigerian government. Following this annual review the logframe will continue to be refined to reflect the next stage of the project, to make it more manageable, and to set milestones beyond March

5 C: DETAILED OUTPUT SCORING (1 page per output) Output Title Modelling capability: Improved institutional capability for modelling long-term energy and emission scenarios out to 2050 through the development of the Calculator models. Output number per LF 1 Output Score A++ Risk rating (H, M or L): Low Impact weighting (%): 20% Risk revised since last AR? No Impact weighting % revised since last AR? Yes was 25% Indicator(s) Milestones Progress 1.1 Number of countries where DECC held week-long workshops to train the Calculator teams and other stakeholders. 10 countries having held workshops by end of 2014 Surpassed 11 ICF-supported countries had workshops, plus at least 9 others (including Ecuador) had technical support during period to the end of Since then, introductory training has been given to Peru, Venezuela and Chile as well. 1.2 Number of national teams that have developed a set of "one pagers" for their 2050 Calculator (between 30 and 50 one pagers). 1.3 Number of national teams that have developed an Excel model suiting the requirements of the 2050 Calculator approach. 10 countries having developed one pagers by end countries having developed Excel models by end 2014 Surpassed 11 countries had developed these by the end of 2014, though the majority had not published them by then. Since then, Ecuador has also developed onepagers. Surpassed 11 countries had developed working models by the end of Since then, Ecuador has also developed an Excel model. Key Points No new milestones were set this year for Output 1 because this stage of the project is complete. The score, therefore, is the same as in the previous annual review. In addition to the original 10 countries, teams from Ecuador, Chile, Venezuela and Peru have received introductory training over the last year from the Colombian and Mexican teams, with travel funded by the ICF project. The Ecuadorian team has finished its model and plans to launch in March Knowledge exchange about wider energy modelling and UK policy has taken place between DECC and the Calculator team in India, which is based in NITI Aayog (the government s planning commission). This was part of the Statement of Principles agreement between the two organisations for the development of their Calculator. The head of DECC s Central Modelling Team visited them in Delhi in January 2016 and shared his team s approach to quality assurance and professional development. This was followed up by a visit of the Indian team to London to meet modellers in DECC and other institutes, and to learn more about DECC s policies in sectors highlighted by the Calculator as important for India. NITI Aayog has asked to extend this co-operation. Summary of responses to issues raised in previous annual reviews (where relevant) Recommendation from last annual review: DECC should continue to support the development of the Calculator community so that it can become self-sustaining without DECC funding in the future. This should be done by developing the 2050 Calculator community website and online book (which details the process of building a Calculator and serves as our main guidance document), and by organising another conference in early 2016 to bring everyone together. Response: Due to staffing constraints, the work on the website and book this year was de-prioritised this year, with only minimal updates completed. This should be reprioritised in

6 A conference (now called a workshop to reflect its more focused, technical nature) was held in Mexico City on the th March 2016, with representatives of 26 countries attending. The workshop included the launch of the Mexico My2050 game and updated webtool, and presentations from all the ICF-supported countries as well as others. Initial feedback received from attendees has been positive. Recommendations Priority should be given to the further development of the online resources and community. This should involve getting case studies from the countries DECC has supported, so that learning can be shared. Deadline for revising the book: September 2016 (lead: DECC 2050 Team). Set up a LinkendIn group for people involved in the Calculators as a low-cost way to contact each other and share knowledge (suggested by attendees at the Mexico workshop). Deadline: end of April 2016 (lead: DECC 2050 Team). Evaluate the usefulness of the Mexico workshop, including getting feedback from attendees. Consider whether this is the model the programme should use going forward. Deadline: end of April 2016 (lead: DECC 2050 Team). 6

7 Output Title Transparency: Energy and emission data and future pathways made publicly available in an easy-to-use, open-source format. Output number per LF 2 Output Score B Risk rating (H, M or L): Medium Impact weighting (%): 15% Risk revised since last AR? No Impact weighting % revised since last AR? Yes was 25% Indicator(s) Milestones Progress 2.1 Number of national teams that publish an Excel version of their 2050 Calculator online. 10 countries by end of March 2016 Not met 8 of the 10 original ICF-supported countries have made their Excel available online. Vietnam is the only launched country that has not put it online, though they have sent it to NGOs that have asked for it. Brazil has not launched yet. In addition, Ecuador plans to do this by the end of March. 2.2 Number of national teams that publish a web interface for their 2050 Calculator, enabling users who are not comfortable with Excel to run the model. 2.3 Number of countries where teams publish additional documentation to explain assumptions (these could be included within the Excel and web page versions). 10 countries by end of March countries by end of March Not met 9 of the 10 original ICF-supported countries have webtools that are live. Brazil is the only one that does not. In addition, Ecuador plans to do this by the end of March. Not met 4 of the original 10 do have this information online (these are India, Bangladesh, Mexico, and Thailand). In addition, Ecuador plans to do this by the end of March. Key Points This output indicator scored the same as last year. Milestone targets remained unchanged, and while some progress has been made, all were still missed. This year, two of the remaining three countries have published their models (Mexico and Nigeria), leaving only Brazil left. While the Brazilian Excel model is largely complete, the team working on the Calculator was pulled from the project to work on a national emergency: a severe drought. The announcement of some new energy policies was also delayed, and the government wants the tool to launch after these so that the tool can reflect them. Over the last few months, work has started again, and they plan to launch in summer 2016 and further develop the model after that. Many of the countries that launched their initial version a year or so ago have continued to work on their models and improve them. For example, India launched version two of their Calculator in August 2015 with an added cost sector and more detail in the industrial and building sectors. Only one of the countries that has launched has not published its Excel model Vietnam. They have shared it with NGOs that have asked for it, but have not put it online. Additional documentation is only available for four of the nine released Calculators. This is most likely due to the fact that most teams have not had enough time to create detailed websites and documents. This year s score is an improvement compared to last year, and India s website and Mexico s consultation paper are good examples of what can be done, but there is still a long way from the goal. Two other countries (Colombia and Vietnam) have websites with introductory material, but do not contain extra details. Summary of responses to issues raised in previous annual reviews (where relevant) Recommendation in last annual review: DECC should support the final three teams to publish their Calculators. 7

8 Response: Two of the three have published, and were given support by the DECC team. The Brazil team aims to publish in summer Recommendation in last annual review: DECC should focus on existing Calculators and promoting accessibility and transparency, rather than new countries. Response: This became part of the strategy for the year. Funding for new countries was limited to the initial training of four Latin American country teams, as described in Output 1 above. Recommendation in last annual review: DECC should continue to encourage the publication of additional material. Response: This work was de-prioritised this year due to staffing constraints, but should be considered again in the next year. Recommendations Continue to work with the FCO in Brazil to support the completion and launch of their model, by end of August 2016 (lead: DECC 2050 Team). Work with FCO posts to evaluate country Calculator projects, with the view to better allocate future funding to maximise value for money. Deadline: May 2016 (lead: DECC 2050 Team). Speak to all the Calculator teams about the importance of transparency and support them to publish their spreadsheets, assumptions and other data online (for example, providing them with examples of best practice and funding web hosting if needed). Deadline: March 2017 (lead: DECC 2050 Team). 8

9 Output Title Peer review: Stakeholders are involved in reviewing data to improve assumptions and to generate buy-in. Output number per LF 3 Output Score A Risk rating (H, M or L): Low Impact weighting (%): 10% Risk revised since last AR? No Impact weighting % revised since last AR? Yes was 15% Indicator(s) Milestones Progress 3.1 Number of countries where teams consult a broad range of relevant stakeholders from government, business and NGOs 10 countries by end of December 2014 Met 10 countries have achieved this. 3.2 Number of countries where teams use evidence provided by stakeholders in their final model. 10 countries by end of December 2014 Met 10 countries developing Calculators had received input from stakeholders by end In all cases, national teams had utilised input from stakeholders. For example, Colombia has used information from other studies and revised down many of their assumptions for Level 4 on the basis of their workshops. Key Points No new milestones were set this year for Output 3 because this stage of the project is complete. The score, therefore, has not altered since the last annual review. Higher level Outcomes and Impacts from this early engagement of stakeholders are now being seen (see Output 4 below). Summary of responses to issues raised in previous annual reviews (where relevant) Recommendation in last annual review: DECC should continue to promote stakeholder engagement in 2050 Calculator projects. Response: This has continued. For example, the positive impact of stakeholder engagement is stressed in introductory training and online resources. Recommendations Once the Brazil model is launched, encourage Post to invite a wide range of stakeholders to the launch event, and to consider holding stakeholder workshops with the finished model in other regions and cities. Deadline: August 2016 (lead: DECC 2050 Team). Continue to promote stakeholder engagement in new countries, for example through including more case studies in training resources. Deadline for revising the book: September 2016 (lead: DECC 2050 Team). 9

10 Output Title Society engagement: Outreach activities are held to allow government departments, NGOs, academia, business, and wider civil society to use the Calculator and debate future pathways Output number per LF 4 Output Score B Risk rating (H, M or L): Low Impact weighting (%): 15% Risk revised since last AR? Was Medium Impact weighting % revised since last AR? Yes was 25% Indicator(s) Milestones Progress 4.1 Number of countries which hold launch events with government representatives, NGOs and business people present. 10 countries by end of March 2016 groups: 4.2 Number of countries in which the Calculator project gets at least three positive press mentions (online, print or TV). 4.3 FCO Posts in supported countries agree with the statement: "Activities around the launch met or exceeded our expectations and the expectations of the team in terms of engaging an influential audience and promoting the tool more widely." 4.4 Number of countries that have developed a My2050 or app version, specially designed to engage the general public or young people. Key Points 10 countries by end of March countries by end of March countries by end of March 2016 Not met - 6 of the 9 launched had attendees from all three groups. The remaining countries had attendees from some of the Thailand - government and businesses, no NGOs Colombia - NGOs and government, perhaps one or two consultancies, but no real business involvement. Bangladesh - NGOs and businesses, but no government. Not met 8 of the 9 that have launched got 3 or more press mentions. South Africa got 2 press mentions, just missing the milestone. Met 9 of the 9 that have launched. Posts in 8 of the 9 that have launched agreed with the statement. Due to personnel changes in the British High Commission in Nigeria, there is no FCO Post who was able to answer this question but DECC received good feedback on the event before the individual involved left, and the write-up of the event shows that high-level people attended from a range of organisations. The programme team therefore feels comfortable stating that Nigeria achieved this indicator. Met 4 of the 9 that have launched have developed My2050 games or apps. South Africa has launched theirs, and Mexico s launched on the 15 th March. Bangladesh and Colombia s are close to completion and launch. In last year s annual review, Output 4 received a B and this has remained unchanged. Although positive progress has been made this year, three of the four milestones were narrowly missed. Most of the Calculator launches have had high profile speakers (for example, the Director General of the Energy Commission of Nigeria and the Colombian Minister of Environment and Sustainable Development), had a varied audience and received press coverage some significant (for example 25 web news stories after the launch of India s second version). All the My2050s and apps that were planned at the beginning of the year have been developed and have either been launched (South Africa and Mexico) or are close to being launched (Colombia and Bangladesh). 10

11 o The Colombian team have held a launch event for their My2050 game, but the online release of the game has been delayed. It was presented in the workshop in Mexico and appeared to be a very well thought out computer game, with mini-games and challenges that would help children understand more about climate change. There are plans for it to be included on all computers that the government donates to schools. o The Bangladesh app was also presented at the Mexico workshop, but is still being finalised. It has been designed so that other countries can easily adapt it, so there are clear signs that its impact will be replicable. The India team are interested in adapting the app s code, as they have been having problems finding a suitable development company in India. o The South African tool is live and there are clear plans for how this is going to be used in schools. The ministry is working with partners to develop training materials and workshops for teachers so that it can be rolled out across the country. o The Mexico My2050 website has been launched, but plans for it are a little less clear at this stage. There is interest in developing My2050s in a number of other countries. However, because the UK My2050 was created by an external company and other countries have since developed improved versions, DECC is unable to provide direct technical assistance. Higher level outcomes and impacts from the engagement of stakeholders and the public are now being seen, in the ten countries. o o In the logframe, Outcome Indicator 3.2 is the number of countries in which stakeholders have used the Calculators in their work. This has now reached seven for example, a research institute in India used their Calculator to examine the impact of different energy futures on the quality of life of the Indian population. For Impact Indicator 2.2, we asked Posts in the ten Calculator countries if they agreed with the statement The FCO and/or Calculator team believes that the tool and its development process have led to better co-ordination and co-operation between government and stakeholders. Posts in seven countries agreed. This may be because engagement with stakeholders has continued after the launch in many countries. For example, workshops for stakeholders using the finished Calculators were held in South Africa, Indonesia, India, Vietnam, Bangladesh and Colombia. Summary of responses to issues raised in previous annual reviews (where relevant) Recommendation from previous annual review: Supporting countries to do more outreach work as part of the next stage of their projects by providing advice for all, and in some countries providing funding support. Response: This was carried out more funding was secured for work on South Africa s My2050 and Bangladesh s app, and India s regional workshops. In addition, the FCO secured Prosperity Funding to hold regional workshops in Indonesia Recommendation from previous annual review: There was a recommendation to encourage countries to share more web statistics on how much the Calculators are being used, or to develop different indicators. This was because the logframe used last year included an indicator about how many people were accessing the webtools and also required qualitative feedback on their experiences. As most teams had not set up web analytics on their sites so could not provide statistics, and as no clear milestones had been set, it was not possible to report on this indicator in the last annual review. Response: This indicator has been removed from the logframe this year because the DECC Calculator team was not able to provide technical support to set up web analytics on each country s site. Recommendation from previous annual review: If DECC supports new Calculator countries in the future, we should consider whether the organisation involved shares our objectives and is able to influence government policy and raise awareness. Response: This has not been an issue for the programme this year, as DECC has not given significant support to any new country projects. Recommendations Engagement should be encouraged and supported once the model is completed in all Calculator countries. To facilitate this, good practice on outreach should be shared between the country teams through case studies included in training resources. Consideration should be given to funding further international workshops. Deadline for revising the resources: September 2016 (lead: DECC 2050 Team). For future My2050 models and apps, prior to funding being committed, clear objectives and full outreach plans for these tools should be set to make sure that the finished game meets the needs of the audience and the learning outcomes that are desired. Investigate whether it would be possible to commission some basic My2050 software that could more easily convert a country s Excel model into a My2050 game which could then be made open source and offered to all teams. This will be subject to feasibility and cost considerations. 11

12 Add information to the online training resources about how to add web analytics to a Calculator site so that information on the number of users will be available in the future. Encourage teams to add this to their completed sites. Consider adding suitable indicators to the logframe to measure the number of users of the tools and also the quality of user experience. 12

13 Output Title LCD co-benefits: The calculator models show that low-carbon development delivers extra benefits, e.g. air quality, fossil fuel imports Output number per LF 5 Output Score A Risk rating (H, M or L): Medium Impact weighting (%): 15% Risk revised since last AR? No Impact weighting % revised since last AR? Yes was 10% Indicator(s) Milestones Progress 5.1 Number of countries whose 2050 Calculator tools include relevant developmental cobenefits of reducing emissions - e.g. land use implications, air quality, energy security, water quality/availability. 8 countries by end of March 2016 Met - 9 countries have achieved this, with land use and energy security the most common co-benefits included. Key Points All of the finished Calculators include at least one of these co-benefits. This has increased from five in last year s annual review, showing that progress has been made. This increase not only represents the addition of two new Calculators launched this year but also includes the older Calculators adding co-benefits to their models. For example, India, Bangladesh and Vietnam have added costs, Indonesia added land use in its second version and is now adding costs. There is a lot of interest in developing more of these advanced sectors across the ten Calculator countries. Most are interested in cost, as it is such an important consideration for governments when making planning decisions. Air quality has only been implemented in one of the nine so far, but more countries have expressed an interest in adding it. Summary of responses to issues raised in previous annual reviews (where relevant) Recommendation in last annual review: Continue to work with countries to get a first working version of their model out, and then add these extra sectors afterwards. Response: This is how DECC has been working with these ten countries over the last few years, and is the approach we recommend to new countries. Recommendation in last annual review: Cover how these sectors are being modelled at the next conference so that all the teams can learn from each other. Particularly look at China s work with water in Inner Mongolia (which will be the first Calculator to include water pollution and availability). Response: The Mexico workshop included information about the innovative way that cost is being added to the Indonesian Calculator, and the benefits of adding this sector were shown in the presentations of other countries. Unfortunately the Chinese team could not attend to share their experience with modelling water. A Calculator for Odisha state in India has been developed this year that also contains water, though the methodology was not presented in Mexico. Recommendations Continue to encourage development of the models into these more advanced areas. Consider supporting this financially if it would offer value for money. When regular analytical support for DECC is secured, the analyst should prioritise quality assurance of the models that have been launched or edited since the Calculator team downsized in March 2015 to ensure consistency and accuracy. Find alternative ways of sharing the water sectors of China and Odisha (for example, share the spreadsheet and information, and/or include a case study in the online book). 13

14 Output Title Development of the Global Calculator: Publication of a peer-reviewed global tool with full documentation and launch event. Output number per LF 6 Output Score A++ Risk rating (H, M or L): Low Impact weighting (%): 25% Risk revised since last AR? No Impact weighting % revised since last AR? Yes was 45% of the separate GC annual review Indicator(s) Milestones Progress 6.1 Develop and publish a Global Calculator model, with wide stakeholder consultation and peer review, adhering to the project plan. Launch the final version in January 2015, following stakeholder engagement and a call for Met Workshops were held in spring. The call for evidence, with a spreadsheet and webtool online, was launched on 17 th July 2014 with the launch of the final version on 28 th January Target countries: number of stakeholders (businesses, NGOs and govt officials) from Brazil, India, China and South Africa attending workshops or meetings to help develop the Global Calculator. 6.3 Globally: total number of stakeholders (businesses, NGOs and government officials) attending workshops or meetings to help develop the Global Calculator. evidence. 20 stakeholders from these countries by December stakeholders in total by December Met At least 22 experts from these countries attended our workshops (5 from businesses, 7 from NGOs and 10 from governments), and 1 responded to our call for evidence. Met Over 100 unique stakeholders took part in our workshops. The team received 44 comments during the call for evidence. Various presentations were held, including a cross-whitehall event attended by around 60 people Key Points No new milestones were set this year for Output 6 because this stage of the project is complete. The score, therefore, is the same as in the previous annual review. Since the launch of the Global Calculator in January, outreach work using the finished model has been carried out, and has met some of the higher level outcome and impact milestones. For example, the webtool has now had over 22,000 users, DECC is aware of over 50 presentations that have been made to our target audience, and at least 14 universities and colleges are using the Global Calculator in teaching. However, some outcome and impact milestones were missed. The number of stakeholders engaging with the tool in our target countries of China, India, South Africa and Brazil has been very low, despite events being held in three of those four countries. It has also been difficult to get our target audience of businesses, NGOs and policy makers to engage more deeply with the tool only one organisation submitted a pathway that could be published in the tool this year, and only four organisations outside the Global Calculator team have presented the tool themselves to audiences. Feedback from businesses has been that the tool is interesting but not useful to them directly (for example, for helping them plan their investment strategy) as it is too high level. Many people also do not feel confident using the tool independently without first having someone demonstrate how it works. Outreach work this year has given the Calculator team a better sense of how the target audience reacts to the model. The main learning points are: o Users need to be introduced to the Global Calculator by someone, as it is not immediately clear how it works. However, most people find it quite easy to use after the introduction. This suggests that in future outreach should focus on facilitated events like workshops, presentations and webinars, or on further developing the introductory screens for the tool to make them more engaging. 14

15 o The Global Calculator is not a suitable planning tool for businesses. Its primary benefit is as an educational and discussion tool as it considers climate change on a global scale. However, it can be used by the original target audience of business people, policy makers and NGOs, as well as students. Summary of responses to issues raised in previous annual reviews (where relevant) Recommendation in last annual review: Continue to gather feedback from stakeholders so that the model can be improved during its next update. Response: The Global Calculator team has recorded specific feedback on data in the model for future updates. Outreach this year has allowed the project team to assess how the target audience reacts to the model. Recommendation in last annual review: Disseminate the tool more widely following the launch. Response: This has been the main focuses of the DECC Calculator team s work this year. Most of the outreach carried out this year has been organised in conjunction with the FCO to build on their work or events they are running around the world. DECC also commissioned the World Resources Institute to hold a workshop in Washington DC for businesses, and worked with the LSE to hold a workshop in London looking at how climate scientists and businesses can work together. Recommendations Work with the project s partners to develop a strategy that further improves the Global Calculator usability and impact on the audience. Timescale: by December 2016 (lead: DECC 2050 Team). Also work with our partners on an outreach strategy that takes into account what has been learnt about the tool in practice, and looks to scale up the impact of the tool by reaching out to organisations who already work with our target audience. Timescale: by March 2018 (lead: DECC 2050 Team). In addition, find ways to contact universities and those who work in climate education to try to better engage the student audience. Timescale: by March 2018 (lead: DECC 2050 Team). 15

16 D: VALUE FOR MONEY & FINANCIAL PERFORMANCE (1 page) Key cost drivers and performance Cost drivers: The key cost drivers for the projects in each country are the staff costs for teams and the costs of outreach activities like holding workshops in each country. These have not changed significantly this year. Three Es framework: The business case that was approved in October 2015 contained an assessment of how the proposed work for the year met the three Es of value for money (VfM), which provides a framework for evaluating the programme. 1. Economy (Are we or our agents buying inputs of the appropriate quality at the right price?): In the business case we stated that economy would be promoted by working with organisations that have been approved by FCO Posts following their procurement guidelines for VfM and competition, by working with local organisations to help build capacity, and by working with existing partners where possible. We would also only pay for work after it is completed, not in advance. These principles have been followed this year where appropriate. If projects have required that we deviate from these, DECC s procurement and financial rules have been followed to ensure VfM. 2. Efficiency (How well do we or our agents convert inputs into outputs?): In the business case we stated that efficiency would be promoted by working with existing partners where possible to save time and to reduce the risk of low quality work. We also said that we would monitor progress carefully with FCO Posts and draw up clear milestones for the project. This year we have worked with existing partners almost exclusively. We have only given small amount of money to new Calculator teams. Most of the output measures have been met or exceeded, and the programme has largely moved from the outputs to the outcomes stage. For the five strands of work that received further funding this year, three have been completed to schedule and early feedback from Posts suggests they were completed to a good standard. The delay to signing off the business case has meant the project in Indonesia is likely to be completed a few months late, and in India the educational element of the project plan was not carried out and therefore not funded. 3. Effectiveness (How well are the outputs from an intervention achieving the desired outcomes?) Effectiveness is best tracked through the logframe, particularly the outcome and impact sections. The logframe shows that a lot of progress has been achieved this year, as the outputs have now translated into the intended outcomes and impacts in many of the countries we have worked with. For example, four countries have reported they used their Calculator models in formulating their INDCs. However, there were areas where performance was weaker, particularly those relating to outcomes for the Global Calculator (the creation of which has an impact weighting of 25% in the logframe). VfM performance compared to the original VfM proposition in the business case Five qualitative VfM indicators were identified in the original business case: 1. Builds developing country government capacity This year, we have continued to build government capacity in many of the ten countries, as shown in the log frame. Some countries proactively identified training gaps, and DECC arranged targeted training and support to meet these. Models have continued to be updated and improved this year, with new sectors added. 2. Facilitates low-carbon dialogue The logframe shows that the Calculators have been used to generate a dialogue around climate change this year. For example, being used as a platform for discussion between government departments in the INDC formulation in India and Colombia, and the educational programme in South African schools. In addition, we know of over 50 presentations at which the Global Calculator and its results have been presented to our target audience of NGOs, businesses and policy makers. 3. Uses existing resources 16

17 This year we continued to use DECC technical expertise where possible, but also made better use of the expertise built up in the Calculator countries to facilitate knowledge sharing. For example, the Indian web programmers provided advice to Nigeria and Brazil. 4. Does not use consultants In the business case that was signed off in October, it was decided that this was not an appropriate measure of VfM because some governments have preferred to use consultants or other partner organisations to develop the Calculator. 5. Enhances UK co-operation as a leader on low-carbon development At least 25 extra countries and regions have started projects, 17 of which have completed their models, since the ICF programme started. The UK is at the centre of this movement. Engagement around the Calculator has created opportunities to introduce UK climate policy to developing countries, such as the Climate Change Act and Carbon Budgets. Assessment of whether the programme continues to represent value for money The programme continues to represent value for money based on the indicators set during the original business case. When the last annual review was written, most of the Calculator models had just been completed. This year the programme has moved from the outputs sections of the logframe to outcomes and impacts the value of the original investment of time and ICF funding is now being realised as the tools are being embedded in the policy making process of countries. There is demand from existing countries for further investment. Going forward, these proposals must be considered on a case-by-case basis, taking into account the total spend so far against the potential benefit. Quality of financial management Normal DECC budget and reporting procedures were followed throughout the project to the correct standard. Date of last narrative financial report Date of last audited annual statement E: RISK (½ page) Overall risk rating: Medium Overview of programme risk An update on the risks covered in the project extension business case approved in October 2015: Risk description Initial Current Comment at 3 year review stage risk risk Risk 1: Projects will overrun or fail to deliver 2 2 This remains a medium risk as there have been delays to some of the projects this year. Work in Indonesia may extend into The educational work in India has not been delivered and the team is considering how best to proceed. There is no financial risk, as Risk 2: Lack of staff resource in DECC to manage the project. Risk 3: Lack of technical knowledge in DECC to support the work. Risk 4: Work does not live up to the ideals of the 2050 Calculator. DECC will not pay for work that has not been delivered. 2 2 This remains a risk. The end-of-year period is expected to be particularly busy with project, finance and reporting deadlines. 3 3 This has remained an issue for the programme team this year. We have received some ad hoc support from previous Calculator team members, and the Calculator teams (particularly those in Colombia, India and Indonesia) have also offered support. This solution is not sustainable in the long run and risks damaging DECC s reputation. Dedicated support is needed in the future. 2 2 Overall our country Calculator teams have worked transparently and have engaged with stakeholders and the public. However, the logframe shows that not all countries have published documentation about their 17