Interim Evaluation of the Eurostars-2 Joint Programme

Transcription

1 Interim Evaluation of the Eurostars-2 Joint Programme Written by Expert Group Chair: Marcel Shaton Expert Group Members: Axel Lehmann, Eva Pando, Isella Vicini Expert Group Rapporteur: Michele Cincera June 2017

2 Interim Evaluation of the Eurostars-2 Joint Programme European Commission Directorate-General for Research and Innovation Directorate B Open Innovation and Open Science Unit B.3 SMEs, Financial Instruments and State Aid Contact Maria KAYAMANIDOU, Laura PIANI Maria.Kayamanidou@ec.europa.eu Laura.Piani@ec.europa.eu RTD-PUBLICATIONS@ec.europa.eu European Commission B-1049 Brussels Manuscript completed in June Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of the following information. The views expressed in this publication are the sole responsibility of the author and do not necessarily reflect the views of the European Commission. More information on the European Union is available on the internet ( Luxembourg: Publications Office of the European Union, 2017 PDF ISBN doi: / KI EN-N European Union, Reuse is authorised provided the source is acknowledged. The reuse policy of European Commission documents is regulated by Decision 2011/833/EU (OJ L 330, , p. 39). For any use or reproduction of photos or other material that is not under the EU copyright, permission must be sought directly from the copyright holders.

3 EUROPEAN COMMISSION Interim Evaluation of the Eurostars-2 Joint Programme Written by Expert Group Chair: Marcel Shaton Expert Group Members: Axel Lehmann, Eva Pando, Isella Vicini Expert Group Rapporteur: Michele Cincera Directorate-General for Research and Innovation 2017 Eurostars-2 Joint Programme

4 Table of Contents EXECUTIVE SUMMARY INTRODUCTION Purpose of the evaluation Scope of the evaluation CONTEXT Description of the initiative and its objectives Objectives of the Eurostars-2 Joint Programme Baseline EVALUATION QUESTIONS Relevance and appropriateness Efficiency and use of resources Effectiveness Coherence European Added Value METHODOLOGY IMPLEMENTATION STATE OF PLAY Governance and Implementation Project Portfolio RESPONSES TO THE EVALUATION QUESTIONS Relevance and appropriateness Efficiency and use of resources Effectiveness Coherence European Added Value ANALYSIS OF IMPACT OF COMPLETED EUROSTARS-1 PROJECTS POST 2020: FUTURE OPTIONS AND MODELS CONCLUSIONS AND RECOMMENDATIONS Key Issues Programme implementation Governance ANNEX 1: EUROSTARS-2 JOINT PROGRAMME LEGAL BASIS ANNEX 2: MANDATE OF THE INTERIM EVALUATION EXPERT GROUP ANNEX 3: ANALYSIS OF IMPACT OF COMPLETED EUROSTARS-1 PROJECTS ANNEX 4: THE INTERIM EVALUATION EXPERT GROUP ANNEX 5: MID-TERM EVALUATION EUROSTARS-2 TASKFORCE REPORT

5 EXECUTIVE SUMMARY Scope of the Evaluation The Decision of the European Parliament and the Council on the Union s participation in the Article 185 initiative Eurostars-2 Joint Programme ("Eurostars-2") under Horizon 2020 foresaw that the European Commission shall carry out an interim evaluation by 30 June This report accordingly presents the findings of the Interim Evaluation of "Eurostars-2" prepared by an Expert Panel appointed by the European Commission, DG Research & Innovation and chaired by Mr Marcel Shaton. The task of the Expert Group is to prepare a report and make recommendations based on available relevant evidence, covering all aspects of the interim evaluation as set out in the legal basis and respecting the better regulation Guidelines of the Commission. Main Findings of the Interim Evaluation The major benefits of Eurostars rest in several niche features of the programme: Bottom up approach; Strengthening transnational cooperation among R&D performing SMEs; Division of work between central structure (ESE) and decentralised structures (NFBs); Introduction of new products, processes and services within two years of projects' completion; Targeting of SMEs without any previous international experience in transnational industrial R&D collaborations. On the other side, some weaknesses could also be identified: High heterogeneity of times to contract; Insufficient accuracy and up-to-date information in the ESE database; Uncertainty to get funding for selected projects when the contribution of Participating States has been exhausted by other projects; Lack of synchronization of procedures for implementing the programme; Not enough Participating States at the EU level are active in the Eurostars-2 jointprogramme. Recommendations for the Remainder of the Eurostars Programme Based on overall analysis, the expert group came up with the following recommendations: Key Issues In order to strengthen the EU added value, at least two R&D performing SMEs from two different Participating States should be mandatory in the proposed project consortium with a fair distribution of activities and a good balanced budget (20% minimum for each R&D performing SME, i.e. 70% minimum for both). This is important to respect the main feature of Eurostars-2 Joint Programme to go to the transnational market with innovative products within two years after projects completion. (short term) The core objective of Eurostars-2 is to introduce into the market two years after the project completion the new products, services or processes: this should be organised 1 Decision No 553/2014/EU of the European Parliament and of the Council of 15 May 2014 on the participation of the Union in a Research and Development jointly undertaken by several Member States aimed at supporting Research and Development Performing Small and Medium-sized enterprises. 3

6 and effectively implemented at programme level as well as at project level. The two years requirement should be a clear criterion to take into consideration in the evaluation process. Follow-up and market readiness measures should be monitored by the Participating States which have a much better knowledge of their beneficiaries and their market potential. (short term) R&D performing SMEs as eligibility criteria should be assessed at national level as it is done for the SME self-declaration. Today, this action is pursed by ESE. (short term) The uniqueness of the Eurostars programme in the European panoply of market creating innovation tools to support SMEs should be preserved and re-strengthened. (short term) Programme implementation The EC should consider in the short term granting the EU top-up to those Eurostars projects for which grant agreements have not been signed within one year of the cutoff period. (short term) Shorter time to contract will permit to avoid jeopardizing the innovation potential of projects. Time to contract should be calculated at the project level when the consortium agreement is signed and not at the participant level when they sign their grant agreement with their respective NFB. (short term) The gathering of information from the centralized ESE database which should be constantly updated would constitute an advantage to monitor the overall implementation of the joint programme. (short term) The data collection, selection and monitoring mechanisms in the ESE Database should be improved. (short term) Participating States should first receive MIRs and assess them on efficiency, effectiveness and pragmatism to-go-to-the-market. Their assessment per project should be addressed to the ESE as an element for the payment of the EC top-up. It will allow creating an instrument for assessment of the bottlenecks for commercialisation to build new go-to-the-market and financial tool boxes. (long term) Structure and content of MIR reports should be improved in order to assess projects results on efficiency, effectiveness and pragmatism with regards to-go-to-the-market. (short term) Put in place a follow-up of the results of funded projects (i.e. FiRs) and their introduction into the market within two years after projects completion (MIRs) to better inform Participating States and the European Commission. (short term) The management of the programme should foresee the total annual EU funding as requested in the Annual Work Plan (AWP). Losses of EU annual funding should be avoided. The increase of the EU top-up rate to the NFBs in order to use the left-over EU contribution is a counter-productive option because this will definitively decrease the total budget (and impact) of the joint programme. (long term) R&D performing SMEs as eligibility criteria should be assessed at national level as it is done for the SME self-declaration. Today, this action is pursued by ESE. (short term) 4

7 Governance The Eurostars budget is around 50% of the total administrative budget of ESE including EUREKA activities. This is not perceived in the institutional governance of EUREKA. A distinctive (from EUREKA) Eurostars governance agenda should be set up. (short term) An in depth and representative survey of the barriers encountered by underrepresented Participating States should be carried out following two vectors: exogenous barriers and endogenous barriers. A particular emphasis will be put on the role of NPCs of under-represented Participating States. (short term) The call for convergence of Art.185 creates difficulties and tensions with Participating States that are reluctant to further harmonize the national rules and procedures and funding rates for Eurostars-2. Article 185 TFEU may be not the most appropriate instrument in absence of further convergence potentialities. (long term). The current evaluation process is too fragmented and the evaluation criteria are too numerous and narrow to have a realistic overview of the project's quality. More feedbacks should be given to the evaluators about the outcomes of the submitted projects they examined. (short term) Future Perspectives: Options and Models post scenarios of the future Eurostars can be envisioned along the spectrum from sole operation of the participating states to the sole operation of the European Commission through the Framework Programme. Eureka states participating operate the program on their own trough ESE without any involvement of the EU Commission. Partnership associating the Eureka Participating States and the EU Commission through an adjusted article 185 or another tool such as coordination with EIC maintaining the top-up and coverage of the administrative cost incurred by ESE. This option requires strengthening the uniqueness of Eurostar as specified in our recommendations. Eureka Participating States are the only providers of R&D founding. The Commission covers the administrative costs. This option requires as well strengthening the uniqueness of Eurostar as specified in our recommendations but can be built invoking article 185. Eurostars will be an integral part of the Framework Program (could be EIC) under its procedures and rules, without any involvement of the Eureka participating states. 5

8 1. INTRODUCTION 1.1 Purpose of the evaluation The Decision of the European Parliament and the Council on the Union s participation in the Article 185 initiative Eurostars-2 Joint Programme ("Eurostars-2") under Horizon 2020 foresaw that the European Commission shall carry out an interim evaluation by 30 June This report accordingly presents the findings of the Interim Evaluation of "Eurostars-2" prepared by an Expert Panel appointed by the European Commission, DG Research & Innovation and chaired by Mr Marcel Shaton. The task of the Expert Group is to prepare a report and make recommendations based on available relevant evidence, covering all aspects of the interim evaluation as set out in the legal basis and respecting the better regulation Guidelines of the Commission. The main purpose of the Expert Group is to: Assess the progress towards the objectives of the Art. 185 "Eurostars-2" Joint Programme and to what extent Eurostars-2 is relevant with respect to the demands of the involved Participating States and of the beneficiaries; Assess the efficiency with respect to the specific and operational objectives of the initiative as laid down in its basic act [Decision No 553/2014/EC of the European Parliament and of the Council of 14 May 2014] and the corresponding Impact Assessment (IA); Assess the qualitative and quantitative (expected) impacts of Eurostars-2 at programme and project levels and in particular the efficiency of the entry-into-the market of results/achievements of ended projects. Assess the coherence with other EU-level interventions which have similar objectives to Art. 185 "Eurostars-2" as well as synergies including with other funding programmes such as the ESIF Regional Funds, collaborative projects were SMEs are participating in the frame of bilateral S&T&I agreements between the Participating States; Assess the coherence of the interaction between the different bodies intervening in Eurostars-2 governance (EUREKA High Level Group (HLG), Eurostars HLG, Eurostars-2 Advisory Group, EUREKA Secretariat, NPCs, National Funding Bodies, European Commission) and their role during the execution of the programme; Assess the effectiveness of the mobilisation of national and European funding for R&D performing SMEs, the leverage effect from the Union's contribution and the way Eurostars-2 compare to the 'mainstream' EUREKA with regard to the participation of SMEs; Assess the additional value resulting from the EU intervention, compared to what could be achieved by Member States at national, regional and/or local levels and also, the added-value of the Eurostars-2 Programme compared to other forms of support to R&D&I in Horizon 2020 Framework programme (in particular SME-Instrument, collaborative projects in the Thematic priorities of LEITs and Societal Challenges, Joint Technology Initiatives). The Eurostars-2 Programme evaluation is one of a series of interim evaluations being undertaken of existing Art. 185 initiatives under Horizon 2020 (H2020). It will feed into a wider meta-analysis of Art. 185 initiatives in order to take stock of the experiences in their preparation and implementation, identify critical issues that need to be addressed and propose if necessary adjustments, and assess how the Art. 185 instrument can efficiently contribute to policy developments. 2 Decision No 553/2014/EU of the European Parliament and of the Council of 15 May 2014 on the participation of the Union in a Research and Development jointly undertaken by several Member States aimed at supporting Research and Development Performing Small and Medium-sized enterprises. 6

9 1.2 Scope of the evaluation The Eurostars-2 Programme runs for seven years, from This Interim Evaluation covers its operation over the first three years, Whilst the focus is primarily on the Eurostars-2 Programme, it is also necessary to take into account data and results from the predecessor initiative the Eurostars Joint Programme (referred to for convenience as Eurostars-1 ) which operated from Particular attention is paid to issues relating to the transition between the two programmes and the impact of the Eurostars(-1) in terms of new products, processes, or services introduced into the market two years after the project completion. Article 185 initiatives are joint programmes established by Member States or/and Associated Countries ( Participating States ) with the financial participation of the European Union. Their establishment refers to Art. 185 of the Treaty on the Functioning of the European Union (TFEU). They are designed to meet particular challenges in the research area and to complement Horizon 2020, to leverage national with EU funding, and to create economies of scales and synergies between national and EU research programmes and investments. This Interim Evaluation addresses primarily the EU s participation and benefits from the Eurostars-2 rather than the overall performance of the programme. Aspects such as European added value are critical. The evaluation is timely as it comes when the European Commission is proposing the preparatory phase of the European Innovation Council (EIC) which will support innovators developing market breakthrough innovation. as well as starting to discuss the future of the Eurostars-2 Programme beyond Horizon As such, the report will feed into the wider debate on how to accelerate the emergence and maximise the scale-up of European innovative SMEs. 2. CONTEXT 2.1 Description of the initiative and its objectives R&D SMEs transnational research which contributes to competitiveness, growth and job creation in Europe. Eurostars-2 is jointly undertaken by 34 Eurostars-2 Participating States and Partner countries 4 and the European Union on the basis of Article 185 of the Treaty on the Functioning of the European Union (TFEU) (ex-art. 169 TEC). The participation of the EU was formally acted through the Decision No 553/2014/EU of the European Parliament and of the Council of 15 May 2014 ("Eurostars-2 Decision") 5. The decision entered into force on 27 June The European Union supports financially the Eurostars-2 Programme, with maximum 287 million for the period , coming from the Horizon 2020 budget allocated to "Innovation in SMEs" (Industrial Leadership pillar). The European Union contribution is equivalent to one third of the effective contribution of the Participating States (for both operational and administrative expenditure) and may go up to a maximum of half of the contributions of the Participating States. The EU contribution applies to the sole 3 Decision No. 743/2008/European Commission of the European Parliament and the Council of 9 July 2008 (OJ L 201, , p.58) The Decision can be found on: 4 The following 34 Participating states and Partner Countries are currently participating to Eurostars-2: Participating States: Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Italy, Ireland, Israel, Latvia, Lithuania, Luxembourg, Malta, The Netherlands, Norway, Poland, Portugal, Romania, Slovak Republic, Slovenia, Spain, Sweden, Turkey, United Kingdom. Partner Countries: South Korea, Switzerland. 5 Decision No 553/2014/EC of the European Parliament and of the Council of 14 May 2014: 7

10 Participating States which are either Member States or Associated Countries to H2020; Partner countries are not eligible for EU contribution. Eurostars-2 is organised on the basis of continuously open call for proposals, with at least 2 cut-off dates per year for the award of financial support. Financial support should mainly take the form of grants to selected projects. The programme is managed by the EUREKA Secretariat AISBL 6 ("ESE") in Brussels. Eurostars-2 is aiming to provide financial support to transnational market-oriented research projects initiated and driven by R&D performing SMEs 7. A project should last no longer than three years and, within two years of completion, the product of research should be ready for market introduction. Research and development performing SMEs shall take the lead and should be able to exploit commercially the project results, thus improving their competitive position. Research organisations, universities, other SMEs, large companies and other actors of the innovation chain can also participate in Eurostars-2 projects. Eurostars-2 activities should be in line with the objectives and the bottom-up principle of Horizon 2020: projects shall be innovative with no thematic restriction but limited to civilian purposes. For the purpose of being eligible as a Eurostars-2 project, the consortium is a partnership composed of at least two entities that are independent of one another and established in two different Eurostars-2 Participating States or Eurostars-2 Partner Countries. In order to be considered as eligible for receiving EU financial contribution, the project is required to meet the eligibility criteria described above and to be a partnership composed of at least two entities that are independent of one another and established in two different Eurostars-2 Participating States or Eurostars-2 Partner countries, out of which at least one Member State or Horizon 2020 Associated country. Eurostars-2 aims at combining integration (managerial, scientific and financial) with a decentralised access and national funding rules. The partners in the selected Eurostars-2 projects are administratively handled by their respective national administrations. It is assumed that the bottom-up nature of the scheme and national funding procedures fit the specific needs of R&D performing SMEs. The initiative is the continuation of the Eurostars Joint Programme ( ) where the European Union agreed to participate in the programme by making a financial contribution to the equivalent of a maximum of one third of the effective contributions of the participating Member States and the other participating countries, within a ceiling of EUR 100 million for the whole period 8. 6 EUREKA is an intergovernmental network launched in 1985, to support market-oriented R&D and innovation projects by industry, research centres and universities across all technological sectors. It is composed of 41 members, including the European Union represented by the Commission. The EUREKA Secretariat, based in Brussels, acts as the central support unit for the network. 7 Within the frame of the Eurostars-2 Decision (Art.2), R&D performing SME means an SME which meets at least one of the following conditions: (i) SMEs with 100 Full Time Equivalent (FTE) employees or fewer must have at least 5.0 FTEs dedicated to R&D activities, or dedicate at least 10.00% of their FTE to R&D activities, or dedicate at least 10.00% of their turnover to R&D activities and (ii) SMEs with a headcount of more than 100 FTE employees must have at least 10.0 FTEs dedicated to R&D activities or dedicate at least 10.00% of their FTE to R&D activities or dedicate at least 10.00% of their turnover to R&D activities. The current definition of SMEs is set out in the Recommendation 2003/361/EC (OJ L124, , p.36). 8 Decision No 743/2008/EC of the European Parliament and of the Council of 9 July 2008 on the Community s participation in a research and development programme undertaken by several Member States aimed at supporting research and development performing small and medium-sized enterprises. (OJ L 201, , p.58-67): 8

11 2.2 Objectives of the Eurostars-2 Joint Programme. Eurostars-2 aims to support such R&D performing SMEs by: encouraging them to create new economic activities based on R&D results and bring new products, processes and services to the market faster than would otherwise be possible; creating an easily accessible and sustainable European R&D support mechanism for them; promoting their technological and business development and internationalisation. Eurostars projects are defined by the following specificities: SME-oriented as at least one partner should be an R&D-performing SME; market-oriented as they must have a maximum duration of three years and within two years of project completion the product of the research should be ready for launch into the market. collaborative and international as in any project there should be at least two partners (autonomous legal entities) from two different participating states; bottom-up as the technological areas to be addressed within the projects are selected by the applicants, consequently the national and European funds being directed at certain innovation areas by the beneficiaries research potential and interest; Eurostars shall align and synchronise relevant national research and innovation programmes, and complement existing national and European Union programmes aimed at supporting R&D performing SMEs in their innovation process. 2.3 Baseline The Eurostars-2 largely continues and builds on structures and activities begun under the previous Eurostars Joint Programme ( ). As such, the baseline for this evaluation is essentially the situation pertaining at the end of Eurostars-1 as set out in the Final Evaluation report for Eurostars-1 published in November and the Ex Ante Impact Assessment for Eurostars-2 undertaken by the European Commission. 10 In the mid-term evaluation of Eurostars-1 as well as the final evaluation of Eurostars-1 impacts could not have been assessed due to any availability of ended projects. Indeed, the ultimate objective of the joint programme is to allow R&D performing SMEs to bring their results into the market in the period of 2 years after the end of the project. The ongoing mid-term evaluation of Eurostars-2 will be in the position for the first time to take stock of the results arising from the "Analysis of impact of completed Eurostars-1 projects 11 " Overall, the Final Evaluation concluded that the Eurostars-1 fully reached its target group of R&D performing SMEs and an important feature of Eurostars-1 was its bottom-up approach which permitted to select all type of technologies in which those SMEs innovate. The Programme was justified and had made continuous progress towards its objectives. It was operating as a coherent framework that delivered clear added value for Europe and the associated research, development and innovation activity was reaching critical mass. Activities aimed at improving conditions for industrial exploitation had expanded significantly. The Programme was well managed, the governance system 9 Final Evaluation of the Eurostars Joint Programme, Report of Expert Group chaired by Marja Makarow, European Commission, November Ex-ante Impact assessment accompanying the document Proposal for a Decision of the European Parliament and of the Council on the participation of the Union in a Research and Development Programme jointly undertaken by several Member States aimed at supporting research performing small and medium-sized enterprises SWD(2013) 242 final. 11 "Analysis of impact of completed Eurostars-1 projects", study of the European Commission under publication. 9

12 functioned properly and the interaction between the national and central administrative bodies was effective. The Final Evaluation Expert Group made a series of recommendations in relation to the Programme Strategy in order to better address the strategic challenges. In summary, these were that: Concerning the target group and scope of the programme, all R&D-performing SMEs need to be targeted, whether or not they have been involved in international collaborations before. The ESE should support successful Eurostars projects by providing links to the EUREKA network and to public and private financing instruments supporting follow-up development and commercialisation. Regarding governance, the High-Level Group s decisions should be implemented, and this implementation should be monitored. National parallel application and evaluation needs to be abolished, and the transparency and feedback mechanisms of the evaluation improved. Concerning management and operations, time-spans from submission deadline to evaluation outcomes, signed grant and consortium agreements, and to activation of funding, must be synchronized and shortened to binding deadlines. The Eurostars database should be improved, based on re-designed application and reporting forms, in order to serve the purpose of assessing the impact of the programme on employment as well as R&D and innovation activities of the SMEs, and for information exchange with national project coordinators. Regarding funding, a commonly agreed baseline of harmonized funding rules needs to be agreed, providing the same type of partners with the same maximum rate of funding in each country In addition, a series of specific recommendations were made in relation to the Programme s operational excellence. These recommendations, and the extent to which progress has been made, are discussed in Annex 1. The situation regarding national participation has changed significantly since Eurostars-1 (for reasons that are discussed in more detail below). For this Interim Evaluation, therefore, the Expert Group has given special consideration to: 1) The situation regarding the alignment, harmonization and synchronization of national funding mechanisms; 2) The uniqueness of the programme and its position in the European Innovation landscape. 3) The continuing added value of the Eurostars-2 Joint Programme, not only from a national, but also from a European perspective; and 4) The boundary conditions for improving national participation (commitments) in the future. Eurostars-2 is no longer a green field. Both markets and policy initiatives have developed considerably over the three years since the current Programme was initiated, and even more so over the eight years since the predecessor Eurostars-1 was first launched. 10

13 3. EVALUATION QUESTIONS In accordance with the Expert Group's Terms of Reference, the Evaluation focused on five key aspects: relevance and appropriateness, efficiency and use of resources, effectiveness, coherence and EU added value. The evaluation brief is summarised below and reproduced in Annex Relevance and appropriateness To what extent is the Eurostars-2 Joint Programme relevant with respect to the demands of the involved Participating States and of the beneficiaries? Do the objectives still correspond to the needs of the Horizon 2020 Framework Programme? To what extent is Eurostars-2 appropriate to support the realization of the EU policy objectives especially ERA, Innovation Union, the three "O", thematic sector policies including 'Innovation in SMEs'? 3.2 Efficiency and use of resources Efficiency with respect to the specific and operational objectives of the initiative as laid down in its basic act [Decision No 553/2014/EC of the European Parliament and of the Council of 14 May 2014] and the corresponding Impact Assessment (IA), including: Progress made towards achievement of the following specific objectives: Promotion of transnational market-oriented research activities for R&D performing SMEs in any field, leading to the introduction of new or improved products, processes or services in the market by the participating SMEs; Contribution to the completion of the ERA and increasing the accessibility, efficiency and efficacy of public funding for R&D performing SMEs in Europe by aligning, harmonising and synchronising the national funding mechanisms Progress made towards achievement of the following operational objectives taking into consideration the indicators and targets as mentioned in Chapter 8 of the IA: Three years after the end of each project, for each 1 M of public funding (from EU and participating Eurostars countries), on average, the turnover of the participants should increase by at least 10M, at least 25 new jobs should be created and three new or improved products, processes or services should be on the market; Scientific integration of national programmes: Ensure excellence and impact of the projects selected through international (EUREKA initiative) competition and the application of a single evaluation and selection process; Management integration of national programmes: Further improve operational excellence and accountability for the programme by reducing the time to contract while maintaining an optimal frequency of calls per year; Financial integration of national programmes: Harmonisation of national funding rules and application of a binding ranking list; Facilitate the participation of R&D performing SMEs without previous experience in transnational R&D activities. 11

14 3.2.4 Efficiency with respect to the implementation structures of Eurostars-2, including How efficient is the programme implementation by ESE in collaboration with the National Funding Bodies. What were its main limitations in relation to National Funding Bodies (NFBs) during the execution of the Programme? What could be the lessons learnt for the future? What resources are needed at the different levels (Commission services, Participating States and their NFBs) for the preparation and implementation of the initiative? Are these justified by the scale and scope of the initiative? Efficiency with respect to Eurostars-2 as an instrument to foster transnational R&D cooperation within Europe, including: To what extent has Eurostars-2 been cost-effective? Were the costs involved justified, given the changes/effects which have been achieved? Is the operational performance of EUREKA/ESE and its role proportionate to its allocation of budget? Is the initiative implemented in an efficient way? Have the management aspects been properly addressed? Are effective monitoring and supervision arrangements in place to ensure adequate monitoring of the initiative? Has the initiative been implemented in accordance with the provisions of the adopted work plans, budgets and the delegation agreement? Are the reporting requirements, including audit provisions efficient? What kind of approaches could be considered to generate further efficiency gains? 3.3 Effectiveness What have been the (qualitative and quantitative) (expected) impacts of Eurostars-2 - At programme and project levels, in particular: At Programme level: To what extent is Eurostars-2 able to mobilize national and European funding for R&D performing SMEs? How strong is the leverage effect from the Union contribution? Is the programme sufficiently accessible, in particular for the R&D performing SMEs? Are there identified "barriers to entry"? How does Eurostars-2 compare to the 'mainstream' EUREKA with regard to the participation of SMEs, synchronisation of funding, type of projects etc. What's the efficiency of the mechanisms and tools ensuring the entry-into-the-market of results/achievements of Eurostars-2 ended projects? To what extent are the (expected) impacts of Eurostars-2 in line with its objectives? At Project Level: How effectively are the projects managed by the participants? What are the main economic and social impacts for R&D performing SMEs participating in Eurostars-2? What is the impact in terms of new products/processes/services or significant improvement of existing ones? What is the impact in terms of competitive position, company profile, employment, qualification of staff, R&D investments and attitude towards transnational collaboration? 12

15 What is the added value for an R&D-performing SME to participate in Eurostars-2 (i.e. what is the return on investment)? What is the motivation of other participant types to participate? Do the benefits from participating outweigh the costs? What improvements/modifications are proposed to enhance the participation of SMEs to the programme and maximise the benefits they can get from their participation? How are the SMEs using the IPR resulting from the project? Is the current project format (at least one R&D performing SME and another partner from a different country) appropriate with regard to Eurostars-2 objectives? Does Eurostars-2 play an adequate role in promoting excellence in scientific and technological research, development and demonstration in the considered field and impact of the projects selected? Does Eurostars-2 play an adequate role in supporting innovation in the considered field? Does Eurostars-2 play an adequate role in positioning Europe on the global map of science and technology in research in the concerned field? 3.4 Coherence External coherence: To what extent is Eurostars-2 coherent with other EU-level interventions which have similar objectives? What are the relations (i) complementarity, (ii) synergies including with other funding programmes such as the ESIF Regional Funds, collaborative projects were SMEs are participating in the frame of bilateral S&T&I agreements between the Participating States (iii) and potential overlaps (iv)? Is the performance of Eurostars-2 in line with the spirit of Article 185 TFEU? Are the procedures implemented at national level, coherent and aligned to ensure the best performance of the Programme? Internal coherence: How was the interaction between the different bodies intervening in Eurostars-2 governance (EUREKA High Level Group (HLG), Eurostars HLG, Eurostars-2 Advisory Group, EUREKA Secretariat, NPCs, National Funding Bodies, European Commission)? Did their role evolve during the execution of the programme? Did the role of the Commission change? 3.5 European Added Value What is the additional value resulting from the EU intervention(s), compared to what could be achieved by Member States at national, regional and/or local levels? What is the added-value of the Eurostars-2 Programme compared to other forms of support to R&D&I in Horizon 2020 Framework programme (in particular SME- Instrument, collaborative projects in the Thematic priorities of LEITs and Societal Challenges, joint technology initiative, ERA-net)? 4. METHODOLOGY The Interim Evaluation Expert Group comprised four independent experts chosen for their experience in industrial research and technological development and a rapporteur. The CVs of Panel Members are presented in Annex 4. The evaluation process foresees that the methodologies used to address the questions mentioned above must be both qualitative and quantitative: 13

16 For the qualitative part, the methodologies included desk review of documents/reports, extensive in-depth interviews as well as case-studies. The interviews included staff of the EUREKA Secretariat, members of the Eurostars-2 High Level Group, members of the Eurostars-2 Advisory group, staff members of National Funding Bodies involved in Eurostars-2, Commission services, evaluators, participants in the Eurostars-2 Programme (in particular R&D performing SMEs) (interviews have been carried out via the phone, video conferencing or face to face). The bottom line of the interviews and consultations are part of the inputs used for the elaborations of the assessment. Participation to the HLG meeting and to the EC-EUREKA Working group meeting in Bilbao (October 2016), to the Eurostars Advisory Group meeting (November 2016) to the HLG meeting in Brussels (January 2017) and to the HLG meeting in Seville (March 2017) Five plenary meetings in Brussels to discuss findings, results from interviews and the draft report Two meetings with the ESE staff to discuss procedures and data One meeting with the Eurostars-2 mid-term evaluation Taskforce. Results arising from the internet-based public consultation that lasted 12 weeks (January- April 2017). For the quantitative part, the Group carried out its activities through an independent, robust process built partly on existing evidence base as well as collecting new data including both quantitative and qualitative evidence. The Expert Group has been given access to the EUREKA/ESE database of both Eurostars-1 and Eurostars-2 projects. The results of this Interim Evaluation are supplemented by: The report on "Analysis of impact of completed Eurostars-1 projects" performed from September 2016 until April 2017 provides important inputs relevant for this assessment assignment. The main results and recommendations of this report are available in Annex 3. The results of the analysis based on the internet based public consultation from January 2017 to April 2017 and launched by the European Commission in 2017 is available in Annex 8. The evaluation framework comprised four task assignments. The Experts analysed the content of the interviews during the plenary meetings together with the background documents and other information in order to formulate this report. Task 1 Quantitative analysis of the ESE Eurostars-2 database Task 2 Assessment through 5 evaluation main criteria Task 3 Analysis of commercial results of Eurostars-1 projects Task 4 Recommendations 14

17 5. IMPLEMENTATION STATE OF PLAY 5.1 Governance and Implementation Eurostars-2 Joint Programme is implemented through a Dedicated Implementation Structure (DIS), the Eureka Secretariat (ESE), which is responsible for the execution of the programme, in particular for the organisation of the calls for proposals, the coordination of the eligibility, peer-review evaluation and selection of projects, the monitoring of projects and the allocation of the EU contribution. The governance system of the programme involves three main bodies: The Eurostars-2 High Level Group (EUREKA/Eurostars-2 HLG) is composed of the European Commission and the EUREKA High Level Representatives of the Eurostars-2 participating countries, which have signed a Eurostars-2 Bilateral Agreement with the EUREKA Secretariat. Other EUREKA member countries not participating in the Eurostars-2 Programme retain the option to send representatives to Eurostars-2 HLG meetings as observers. The Eurostars-2 HLG is competent to supervise the implementation of Eurostars-2. The Eurostars-2 Advisory Group (EAG) is composed of EUREKA National Project Coordinators (EUREKA/Eurostars-2 NPCs) from the Eurostars-2 participating countries. The European Commission may participate in the EAG meetings as observer. The EAG meetings will be chaired by the EUREKA Secretariat. The EAG shall advise the EUREKA Secretariat as well as the Eurostars HLG on the execution of the Eurostars-2 Programme and shall provide advice on the arrangements for its implementation. National Funding Bodies (NFBs) are the most important stakeholders for the implementation of Eurostars-2 at national level and to support Eurostars-2 in its challenging aim of further synchronisation and financial flexibility. NFBs will actively collaborate to ensure the implementation of national milestones. They should improve and adapt their national underlying programmes and procedures in such a way that they are compliant with the Eurostars-2 rules and the specific implementing guidelines for Eurostars-2. They should also be willing to implement further improvements of the programme during the execution of Eurostars-2 from 2014 to The Eurostars-2 HLG, EAG and the National Funding Bodies can delegate thematic issues to be discussed by Working Groups. Such a "multilateral approach" (see Box 1), combines a centralized management and decentralised entry points: central international and independent peer review evaluation of the projects (with a common set of evaluation and eligibility rules) run by the ESE in Brussels, combined with local support to project participants in the application and in the funding phase provided by the NPCs and the NFBs based on a binding ranking list and national earmarked budgets. The model is designed to optimize the coordination among the national programmes for R&D performing SMEs while remaining close to final beneficiaries. 15

18 Box 1: Governance Structure of the Eurostars-2 Programme What is EUREKA? EUREKA was established as a platform for market-driven industrial R&D. It is a decentralized network facilitating the coordination of national funding for innovation, and aims to boost the productivity and competitiveness of European industries. EUREKA comprises three pillars for funding R&D projects driven by the private sector: Individual projects, Clusters, and Eurostars. EUREKA Umbrellas are thematic networks to facilitate the generation of future EUREKA projects. Founded in 1985, EUREKA now unites over 40 countries and also includes the European Union (represented by the European Commission). As stated in the Hannover Declaration, envisaging already a cutting-edge trajectory, it was assumed that EUREKA will enable Europe to master and exploit the technologies that are important for its future and to build up its capability in crucial areas 12 and foresaw possible supportive measures from the European Union 13. The EUREKA s structure is composed of the following main bodies 14 : The EUREKA Secretariat (ESE) is an international association based in Brussels acting as the central support unit for the network. An Executive Board (EB) is responsible for the management of ESE. ESE acts also as the implementing body of the Eurostars programme. The EUREKA Chair rotates yearly between EUREKA s member countries, with each Chairs mandate running from July to June the following year. The Chair implements a three-year rolling programme in cooperation with the previous and future chairs (the 'Troika'). The Troika countries form an Executive Group (EG) to which the EU representative is invited. It is responsible for reporting, and implements decisions taken by the HLG as well as for debating key policy issues. The MINISTERIAL CONFERENCE (MC) is the political body of EUREKA, gathering every two years the ministers from EUREKA countries and an EU Commissioner. The INTER-PARLIAMENTARY CONFERENCE (IPC) raises public awareness of EUREKA s role. It is organized in alternate years with the MC The HIGH-LEVEL GROUP (HLG) is the key decision-making body of EUREKA. The ministry responsible for EUREKA in each member country names its High-Level Representative (HLR), who in turn endorses new EUREKA projects, takes decisions on the management of EUREKA, and prepares EUREKA policy discussions for the MC. The EXECUTIVE GROUP (EG) reports and implements the decisions taken by the HLG. The EG is also responsible for debating key policy issues, deciding on topics delegated by the HLG, and advising successive Chairs. The NATIONAL PROJECT COORDINATORs (NPCs) run the National EUREKA Offices at operational level and are responsible for project generation, national and international support and follow-up. They are the direct contact with project participants, facilitating the setting-up and running of a project. The NATIONAL INFORMATION POINTs (NIPs) provide industry and research institutes with an interface with EUREKA and facilitate participation in projects. Source: (information adapted from EUREKA site, sections Structure and History) th Anniversary Report, Two decades of support for European Innovation, EUREKA Secretariat, September 2005 General Conditions, number 4, from Hannover Declaration, in: EUREKA Governing Bodies which are not relevant to this evaluation are not included as it is the case of National Information Points (NIPs). 16

19 5.2 Project Portfolio To fulfil its objectives and reflecting its market orientation, over the period the Programme issued 6 calls for proposals. These have resulted in 579 projects (as of December 2016) being selected for funding participating states. As can be seen in Table 1, in total the number of participants of approved projects amounts to 1512 participants. The success rate under Eurostars-2 was more than 30% in total (number of approvals/number of applications). Table 1. Number of participants in Eurostars-2 projects by status (2014 November 2016) Approved Below Incom Ineli Not Unqua Veto With total success Threshold plete gible Funded lified drawn rate in % i ii Iii iv v vi vii viii ix i*100/ix Participating states (EU 28) Austria Belgium Bulgaria Croatia Cyprus Czech Republic Denmark Estonia Finland France Germany Greece Hungary Ireland Italy Latvia Lithuania Luxembourg Malta Netherlands Poland Portugal Romania Slovakia Slovenia Spain Sweden United Kingdom Participating states (non EU) Iceland Israel Norway Turkey Partner countries South Korea Switzerland Total Note1: Switzerland is for the period a Eurostars-2 Partner Country within the meaning of the Eurostars-2 legal basis. Source: authors own calculations based on ESE Eurostars-2 database - version as of December Note: approved participants also include those that withdrew from projects. It follows from Table 1 that the number of participants that applied varies a lot across participating countries. Among the top 5 countries we find Germany (664 participants), 17

20 The Netherlands (432); Spain (421); France (353) Switzerland (353). Last 5: Malta (3); Estonia (4); Croatia (11); Ireland (13); Luxembourg (14). Why do some countries have relatively low participation rate (e.g. Malta, Estonia, Croatia, Ireland, Italy, and Luxembourg), while the others have a rather relatively high participation rate (e.g. Germany, The Netherlands, Spain, France, and Switzerland)? One explanation to this disparity in participation rates rest in the larger size of the first group of countries compared to the second group. Economic and social variables can be considered to explain these differences such as GDP per capita, total population, average level of higher education, R&D expenditures, the number of patents, or other potentially relevant qualitative aspects (awareness/knowledge of European projects, motivation/willingness to apply for European projects, existing similar funding schemes at national level). In order to increase/ incentivise participations, more promotional activities and information campaigns/sessions could be organized at national level in countries showing relatively low participation rates. Switzerland is a partner country from January 2014 and participating country from January Another fact emerging form Table 1 is the success rate of applicants which varies a lot across participating countries. The success rate for all participating states is 31.65%. Among the top 5 successful applicant countries we have Norway (40%); Switzerland (38.5%); The Netherlands (38.9%); Sweden (37.8%); Denmark (37.7%). At the other end, the last 5 successful applicant countries are Greece (0%); Malta (0%); Estonia (0%); Croatia (9.1%); Cyprus (9.3%). Once again, this disparity can be explained by economic and social factors specific to each country and also by their previous experience in Eureka and more or less similar European projects. We can also mention the level of scientific collaborations in submitting projects that could play a determinant role depending on each country. The provision of accompanying support measures to help the application process like in Spain or improving scientific collaborations when submitting projects could have a positive influence on the success rate. Before to discuss other salient facts emerging from the Eurostars-2 database, it is worth mentioning that in several cases discrepancies could be noticed between the official figures reported in the ESE annual reports and the ones calculated here. 15 One reason to explain this rests in the different time periods considered to make the calculations. For instance, in this report we use the database as received in December So, the number of projects accepted or withdrawn or with another status could be different. This leads the expert group to propose the following recommendation: Recommendation: The gathering of information from the centralized ESE database which should be constantly updated would constitute an advantage to monitor the overall implementation of the joint programme. 15 For instance, we noticed some differences in the sum of total budgets committed by NFBs and the EC contribution as reported in ESE annual reports of 2015 (p.85) and 2016 (p.68). 18

21 Figure 1. Number of approved Eurostars-2 participants by country (2014-November 2016) Note1: Switzerland is for the period a Eurostars-2 Partner Country within the meaning of the Eurostars-2 legal basis. Source: Authors own calculations based on ESE Eurostars-2 database - version as of December, Figure 1 shows that five countries concentrate more than half of all the 1512 participants of projects that were approved, namely, Germany, The Netherlands, Switzerland, France, and Sweden. The category Other represents 125 participants from 16 countries (Italy, Turkey, Hungary, Poland, Romania, Bulgaria, Latvia, Slovenia, Iceland, Cyprus, Portugal, Slovakia, Israel, Luxembourg, Ireland and Croatia). Three countries have no participant (Estonia, Greece and Malta). 19

22 Table 2. Total National Funding Bodies (NFBs) commitment to participant project costs by top-up (in millions ) Total NFB commitment to participant project costs (M ) Cut-off Total growth rate (in %) between last and first top-up Participating states (EU 28) Austria Austria Belgium Belgium Bulgaria Bulgaria Croatia Croatia Cyprus Cyprus Czech Republic Czech Republic Denmark Denmark Estonia Estonia Finland Finland France France Germany Germany Greece Greece Hungary Hungary Ireland Ireland Italy Italy Latvia Latvia Lithuania Lithuania Luxembourg Luxembourg Malta Malta Netherlands Netherlands Poland Poland Portugal Portugal Romania Romania Slovakia Slovakia Slovenia Slovenia Spain Spain Sweden Sweden United Kingdom United Kingdom Participating states (non-eu) Iceland Iceland Israel Israel Norway Norway Turkey Turkey Partner countries South Korea South Korea Switzerland Switzerland Total Note1: Switzerland is for the period a Eurostars-2 Partner Country within the meaning of the Eurostars-2 legal basis. Source: authors own calculations based on ESE Eurostars-2 database - version as of December Table 2 indicates that in total, the budget commitment of NFB has increased very slightly (+0.4%) from the first cut off (2014) to the fifth one (first half of 2016). Romania (1.7%), The Czech Republic (1.5%) and Germany (1.5%) are the three countries with the highest increase rates. On the other hand, Finland (-2.7%), Hungary (-1.3%) and Turkey (-1.5) exhibit the highest decreases. 20

23 country Table 3. Eurostars-2 projects budget by source of funding (in millions ) Participant total costs (M ) EC commitment NFB commitment total Participant share of total costs (in %) EC funding with respect to total project costs (in %) NFB commitment with respect to total project costs (in %) I ii iii iv=i+ii+iii i*100/iv ii*100/iv iii*100/iv Participating states (EU 28) Austria ,3 64,5 Belgium ,7 30,1 Bulgaria ,0 24,0 Croatia ,0 0,0 Cyprus ,0 12,5 Czech Republic ,4 27,3 Denmark ,2 32,5 Estonia Finland ,2 23,5 France ,2 16,8 Germany ,5 26,0 Greece Hungary ,0 25,6 Ireland ,0 0,0 Italy ,0 0,0 Latvia ,9 23,5 Lithuania ,7 27,8 Luxembourg ,7 26,1 Malta Netherlands ,8 23,0 Poland ,8 26,0 Portugal ,3 25,0 Romania ,8 29,5 Slovakia ,3 26,3 Slovenia ,1 9,4 Spain ,9 23,6 Sweden ,1 24,4 United Kingdom ,5 26,0 Participating states (non EU) Iceland ,7 26,7 Israel ,0 0,0 Norway ,5 21,9 Turkey ,5 34,0 Partner countries South Korea ,8 34,7 Switzerland ,5 29,7 Total Note1: Switzerland is for the period a Eurostars-2 Partner Country within the meaning of the Eurostars-2 legal basis. Source: authors own calculations based on ESE Eurostars-2 database - version as of December As it can be seen from Table 3, the top 4 participating countries with the highest NFB commitment are Germany (35.7 million ); The Netherlands (29.8); Denmark (22.6) and Sweden (18.4). The share of EC funding with respect to the total project costs varies across participating countries. This can be explained by the fact that the funding rates vary from one country to the other (for instance in Romania, universities cannot be directly funded since the budget depends on the ministry of industry and not from the Ministry of education). Table 3 indicates that the top 5 countries with the relative highest EC funding are Austria (21.3%), Luxembourg (8.7%), Portugal (8.3%), Denmark (8.2%), and Belgium (7.7%). 21

24 Table 4 summarises the current funding rates for Eureka projects for the majority of EUREKA countries. Table 4. Current funding rates for Eureka projects Country current funding rate (, 2017) Max funding / notes Austria 60% Belgium 50 to 80 Bulgaria 75 to to Croatia Cyprus Up to * Czech 50% or Denmark None available Finland Typically loans 70, grants 50 France Up to 65 None 380,000 of costs eligible for Germany funding Greece Hungary up to 75 Max funding amount per project 50 Iceland 30 MISK for 3 yrs Ireland Up to Israel max. 60 no limit Italy 45% The total maximum amount of ** + 15 *** aid from the eligible costs for one participant is EUR / Latvia year. Lithuania max 50 / max (per project) Luxembourg Up to 70 Pre-competitive development: up to 45% Industrial research: up to 70% max. 25 for Research max. 25 for Netherlands Development Norway No limit Poland Up to 80 For medium entreprise the percentage of eligible costs funded is 75%. For development research it is of 60% for small companies and 50% for medium entreprises. Portugal Romania 7000% up to ~ 100,000/year/project per year per project 5000% Slovenia partner Spain Up to 85 Totally 1,000,000 (1 = 6700% South Korea 1500KRW) / project (Max 3 years) SME as defined by the European max. 50 Sweden Union Switzerland None available Turkey 75 UK N/A Source: (other countries can be selected on the right hand side of the web page). 22

25 Table 5. National contribution to the Eurostars-2 programme relatively to the national knowledge base (as measured by BERD) budget contribution/ indicative budget BERD country sum 5 first cut offs BERD 2014 UE15=100 Participating states (EU 28) , Austria , Belgium , Bulgaria Croatia Cyprus Czech Republic , Denmark , Estonia Finland , France , Germany , Greece 0.0 1, Hungary , Ireland , Italy , Latvia Lithuania Luxembourg Malta Netherlands , Poland , Portugal , Romania Slovak republic Slovenia Spain , Sweden , United Kingdom , Participating states Iceland Israel Norway , Turkey , Partner countries South Korea , Switzerland , Total Note1: BERD = Business Enterprise R&D. Note2: Switzerland is for the period a Eurostars-2 Partner Country within the meaning of the Eurostars-2 legal basis Source: authors own calculations based on ESE Eurostars-2 database - version as of December 2016, Eurostars-2 Annual Work Plan (2014,2015,2016) and Eurostars-2 Annual Report

26 Figures highlighted in red in the 6 th column of Table 5 are the ones that contribute most in absolute terms to the Eureka budget. On the other hand, figures highlighted in red in the last column of Table 4 are associated with countries (e.g. Belgium, France, Germany, Italy, Portugal or UK) that contribute relatively less to the Eureka budget comparatively to the importance of their knowledge bases (as measured by their total expenditures on R&D in the private sector). The benchmark is for EU28 = 100. On the contrary, countries like Bulgaria, Croatia, Cyprus, Czech Republic, Hungary, Ireland, Latvia, Lithuania, Luxembourg, Malta, Romania, Slovak republic, or Slovenia - despite their relative small contribution to the Eureka budget - are in relative terms contributing more if we relativize their contribution to the importance of their private R&D. Finally, Denmark, Finland, The Netherlands Spain and Sweden are in relative terms the most important contributors to the Eureka budget. Table 6 shows the geographic distribution of participants of approved Eurostars-2 projects. This table is not symmetric and must be read line by line. For instance, for Austria, considering all the approved projects with at least one Austrian organization, leads to a total of 170 participants among which 68 are from Austria, 36 from Germany and 12 from the Netherlands. One interesting fact that emerges from this table is that overall, participants of a given country tend to collaborate mainly with other participants from the same country or from neighbouring countries (border-related effect). 24

27 Table 6. Geographic distribution of participants of approved Eurostars-2 projects AT BE BG HR CY CZ DK FI FR DE HU IE IT LV LT LU NL PL PT RO SR SL ES SE UK IS IL NO TR KR CH Total Austria Belgium Bulgaria Croatia Cyprus Czech Republic Denmark Finland France Germany Greece Hungary Ireland Italy Latvia Lithuania Luxembourg Malta Netherlands Poland Portugal Romania Slovakia Slovenia Spain Sweden United Kingdom Iceland Israel Norway Turkey South Korea Switzerland Total Source: authors own calculations based on ESE Eurostars-2 database - version as of December NB: Switzerland is for the period a Eurostars-2 Partner Country within the meaning of the Eurostars-2 legal basis 25

28 Figure 2. Share (in %) of approved Eurostars-2 participants by type of organisation - (2014- November 2016) Research institute 10% Large company 5% Other 1% Non R&D performing SMEs 3% University 15% R&D performing SMEs 66% R&D performing SMEs University Research institute Large company Non R&D performing SMEs Other Source: Own calculations based on ESE Eurostars-2 database - version as of December Figure 2 indicates that about two-third of participants are R&D active SMEs, while 2.6% of participants are non-r&d SMEs. These statistics clearly show that Eurostars-2 projects are mainly granted to support SMEs active in R&D activities. The other two categories of organizations which received an important part of Eurostars-2 funding are universities and research institutes with 15% and 10% approval rates respectively. These two other statistics further confirm that the focus of Eurostars-2 projects is R&D activities. Figure 3 also show the distribution of participants of approved Eurostars-2 projects by type of organisation for the top 10 countries participating in this programme. Here also the R&D performing SMEs represent in all countries more than 50% of all participants with the highest share in the UK (96%) and the lowest (51%) in Austria. 26

29 Figure 3. Top 10 countries in Eurostars-2: share of participants by type of organisation Source: authors own calculations based on ESE Eurostars-2 database - version as of December NB: Switzerland is for the period a Eurostars-2 Partner Country within the meaning of the Eurostars-2 legal basis. 27

30 Figure 4. Share (in %) of approved Eurostars-2 participants by technological area (2014-November 2016) 13% 4% 30% 24% 29% Biotech ICT Industrial Environment Energy Source: authors own calculations based on ESE Eurostars-2 database - version as of December, Figure 4 shows that Eurostars-2 participants come mainly from Biotech, ICT, and Industrial sectors respectively followed by the Environment and Energy sectors. These statistics tend to suggest that, at first, Eurostars-2 projects beneficiaries came from only five technological sectors. Second, among the five sectors, one was not particularly targeted as focus of intervention as it was the case for categories of beneficiaries where we saw that R&D SMEs were predominant. Figure 5. Share (in %) of projects by economic (market) sector Agriculture, Forestry And Fishing Consumer Related Other Electronics Related Industrial Products / Manufacturing Services Energy Communications Transportation Construction And Building Products Medical / Health Related Computer Related Biotechnology / Molecular Biology Source: authors own calculations based on ESE Eurostars-2 database - version as of December Figure 5 also displays the distribution of approved projects by economic sector. The ranking of sectors according to their importance (number of project) is more or less in line with the ranking based on the (more aggregated) distribution in terms of technological sectors (see Figure 4). The energy sector is at the bottom of the scale in Figure 4, whereas in Figure 5 it is at the middle. This can be explained by the fact that in the economic distribution, the energy sector includes not only the producers of energy but also the distributors and the firms operating in the services sectors for energy. 28

31 Figure 6. Share (in %) of projects by number of participants per project (2014-November 2016) 7% 4% 33% 21% 35% or more Source: authors own calculations based on ESE Eurostars-2 database - version as of December, Figure 6 shows that the less participants are in a project, the more we observe approved projects. Actually, projects with two or three participants constitute 68% of all approved projects while those with four, five or six participants constitute only 32%. These statistics tend to suggest that beyond the number of participants in each Eurostars-2 projects might determine the likelihood of being approved. Figures 7 and 8 show the share of projects that started and ended in a given year. Figure 7. Number of participants (in %) in Eurostars-2 projects that started their project in a given year - (2014-November 2016) 1% 13% 47% 39% Source: authors own calculations based on ESE Eurostars-2 database - version as of December,

32 Figure 8. Number of participants (in %) in Eurostars-2 projects that completed their project in a given year - (2014-November 2016) - Projections 15% 19% 32% 34% Source: authors own calculations based on ESE Eurostars-2 database - version as of December, RESPONSES TO THE EVALUATION QUESTIONS This section aims at responding the five evaluation questions (relevance and appropriateness, efficiency, effectiveness, coherence, and European added value) using all the material gathered during the study (interviews and evidence based facts extracted from the ESE database of Eurostars-2 projects, previous reports, ) 6.1 Relevance and appropriateness Eurostars-2 supports international innovative projects led by research and developmentperforming small- and medium-sized enterprises (R&D-performing SMEs). Eurostars-2 targets R&D SMEs transnational research which contributes to innovation, competitiveness, growth and job creation. It contributes to speeding up the development of the technologies and innovations that will underpin tomorrow s businesses and help innovative European SMEs to grow into world-leading companies. Eurostars-2 is jointly undertaken by 34 Participating states and Partner countries and the European Union. In the period, it has a total public budget of 1.14 billion. The Eurostars-2 programme is well aligned with the overall European Research and Innovation policy objectives. The role of SMEs for the economy has never been so important. Eurostars aims to bring increased value to the economy, higher growth and more job opportunities. Eurostars targets therefore all the market sectors, and with its bottom-up approach, it supports the development of rapidly marketable innovative 30

33 products, processes and services that help improve the daily lives of people around the world. The Eurostars-2 programme is considered a very valuable tool to foster R&D performing SMEs. It is complementary to other programmes for SMEs but with the added value of encouraging transnational research cooperation between at least two SMEs. The full bottom-up approach, together with the small size of the consortia which eases project management, are also added value of the programme. In addition, other key strengths are represented by the geographic coverage, with extra-eu partners allowed, the long-term perspective, and the quite high success rate with an average of 30%. These characteristics could attract a relatively high share of newcomers to collaborative European funding, roughly 50%. As illustrated in Figures 2 and 3, the most represented partners are R&D performing SMEs, close to two thirds. A significant share is represented by research centres and universities (one fourth in total), demonstrating the capacity of the programme in fostering technology transfer activities from the research environment to industrial applications. The participation of large enterprises is beneficial, as it provides established marketing channels and lead clients for the innovative solutions developed within the projects. A more in-depth analysis of the ESE Eurostars-2 database also shows that: 79% of Eurostars-2 projects have at least two R&D performing SMEs. For 58% of these projects (i.e. with at least 2 R&D performing SMEs) 70% of the total project costs are concentrated in two R&D performing SMEs. These figures indicate that beneficiaries of Eurostars-2 projects are mainly R&D performing SMEs. 84 % of R&D performing SMEs in Eurostars-2 projects fund at least 20% of the project total costs. 11 projects involving at least one R&D performing SME (out of 470 of this type of projects) have R&D performing SMEs from the same participating state. In other words, most of R&D performing SMEs which participate in Eurostars-2 projects perform collaborations with other participating states. Finally, a recent report 16 shows that Eurostars-1 projects with only two R&D performing SMEs participating have higher performances compared to larger consortia (i.e. three or more participants in the same project). This result was also found in the econometric analysis carried out in the ongoing current study on the socio-economic impact of Eureka projects. The main issue highlighted during the interviews with the various stakeholders rests in the difficulty for in Art. 185 to align or harmonise the national rules of the Participating States (convergence issue), and to the difference of procedures between them, also in terms of timing. Other issues were mentioned as well among which the different procedures at stake across the Participating States or the possibility to have projects that are de facto mono beneficiary. 16 Analysis of impact of completed Eurostars-1 projects carried out by Ernst and Young (2017) and commissioned by the European Commission. 31

34 In light of these findings, the expert group proposes the following recommendations that could be implemented in the short term: Recommendation: In order to strengthen the EU added value, at least two R&D performing SMEs from two different Participating States should be mandatory in the proposed project consortium with a fair distribution of activities and a good balanced budget (20% minimum for each R&D performing SME, i.e. 70% minimum for both). This is important to respect the main feature of Eurostars-2 Joint Programme to go to the transnational market with innovative products within two years after projects completion. Recommendation: The core objective of Eurostars-2 is to introduce into the market two years after the project completion the new products, services or processes: this should be organised and effectively implemented at programme level as well as project level. The two years requirement should be a clear criterion to take into consideration in the evaluation process. Follow-up and market readiness measures should be monitored by the Participating States which have a much better knowledge of their beneficiaries and their market potential. In the long term: Recommendation: The call for convergence of Art.185 creates difficulties and tensions between Participating States that are reluctant to further harmonize the national rules and procedures and funding rates for Eurostars-2. Article 185 TFEU may be not the most appropriate instrument in absence of further convergence possibilities. 6.2 Efficiency and use of resources The Eurostars programme is considered successful in achieving one of its main objectives, the promotion of transnational market-oriented research activities for R&D performing SMEs in any market area. The evaluation criteria based on a time to market of two years for the achieved results is strongly in line with the idea of promoting new and improved products, processes, and services. The mix of national and central management, and of national and central funding, as well, may in some instances successfully increase the accessibility for R&D performing SMEs to public funding, and foster the harmonisation of different national funding mechanisms under a common policy umbrella. However, different funding rules and rates represent a major issue in the programme (see Table 4). These differences arise because of the lack of convergence in aligning, harmonising and synchronising national funding mechanisms due to policy orientations and implementation of Participating States. As a result, the share of EC funding with regards to the total project costs varies across participating countries. This can for instance be explained by the fact that the funding rates vary from one country to the other (for instance in Romania, universities cannot be directly funded since the budget depends from the ministry for industry and not for education). Table 3 lists the top 5 countries with the highest EC funding, namely Austria; Luxembourg; Portugal; Denmark; and Belgium. In addition, timing issues represent an additional limit to maximise the programme efficiency (e.g. in Italy, where a procedural problem linked to the approval time of the national budget act, too close to the first cut off to launch a call, and too far from the September cut-off, when Italian funds are all normally allocated). It is worth to notice the role Eurostars-2 plays in fostering the access to public R&D funding by R&D performing SMEs. The limited size of consortia and the bottom-up approach are attractive features, for instance when compared to Horizon 2020, where the complicated management structures associated to large projects or the extreme 32

35 competitiveness of instruments, like the SME Instrument, may represent limiting features for SMEs. The comitology system underlying the governance of Eurostars-2 could be simplified, for instance by reducing the multiple layers characterizing it. The time to grant has been improved in comparison to Eurostars-1, and it should be aligned with the general Horizon 2020 time to grant of 8 months. However, because of differences across Participating States, of administrative efficiency and the mix of national and European level management, for some projects it may take a significant amount of time before all participants can sign the contracts. Funding synchronisation processes create delays in the time to sign contracts. This in turn can jeopardize the fast introduction of new products and services into the final market or reduce the technological performance of Eurostars projects (see the Ernst & Young report). Box 2 discusses the concept of innovation time-based competition and stresses the importance for companies to be fast when introducing new or improved products, processes or services to achieve and maintain a sustainable competitive advantage. Box 2. Innovation time-based competition Time-based competition is the extension of Just-In-Time (JIT) production into every aspect of the product delivery cycle, from research and development through marketing and distribution of the final product. Even quality, while still key to success, is not the competitive advantage it once was in many industries. Time-based competition is a broad-based competitive strategy which stresses time as the major element for achieving and maintaining a sustainable competitive advantage. It seeks to reduce the time needed to propose, develop, manufacture, market and deliver its products. Time-based competition materialises in two different forms: fast to market and fast to produce. Firms that compete with to-market speed emphasize reductions in design lead-time. In other words, the firm has the capability to minimize the time it takes to develop new products or make fast design changes. Products 50% over budget but introduced on time have been found to generate higher profit levels than products brought to market within budget but six months late. Also, this form allows firms to gain a market advantage by being able to consistently introduce more new products or large numbers of improved products more rapidly than its competitors, thereby dominating the market. Sun Microsystems, for instance, attained leadership in engineering workstations by reducing (by 50% compared to competitors) the time required to design and to introduce new systems. Additionally, these firms are now moving further along the learning curve than the competition. Both factors in the long run increase barriers to entry by competitors. Fast-to-product firms emphasize speed in responding to customer demands for existing products. Wal-Mart, for instance, has been able to dominate its industry by replenishing its stores twice as fast as its competitors. Firms competing in this field concentrate on lead-time diminution throughout the system, from the time the customer places an order until the customer in the end receives the product. This includes the capability to reduce the time it takes to manufacture products as well as the capability to shorten the time between taking a customer's order and actually delivering the product (delivery speed). These reductions in lead-time come typically together with significant reductions in inventory levels. As with JIT, there is less rework, fewer supervisors, lower carrying costs, less overhead, and so forth, as well as enhanced quality and on-time delivery performance. Some customers, known as impatient customers, place a great deal of value on reduced leadtime. These customers are willing to pay a premium to get their goods and services quickly. This combination of lower costs and higher revenues contributes significantly to an improved corporate performance. Source: adapted from: The central evaluation process is widely recognised by the different stakeholders consulted as positive and well structured, being fast, transparent, and professional. 17 Evaluation reports are considered complete and well detailed. Potential issues, but also 17 A detailed description of the evaluation process of the Eurostars-2 programme is available at: 33

36 areas of improvement, are represented by the fact that the evaluation criteria are numerous and narrow, which precludes an overall view of whole projects quality, as well as by the fact that no feedback about the projects outcomes is provided to evaluators. In addition, the common ranking of projects from different technological areas may prove difficult and the outcomes of the discussions and decision-making process of the Independent Evaluation Panel (IEP) should be more transparent. In terms of fostering international collaborations, the participation of extra-eu countries is a strength of the programme, and numerous projects are cross borders, being from neighbouring countries (see Table 5). However, data show that many projects are implemented starting from national collaborations, and Southern and Eastern-Central European countries are (except for Spain) still under-represented (cfr Table 1). Based on a survey targeting unsuccessful Eurostars-1 applicants 18, Figure 9 indicates that for this programme, the challenges to find suitable partners, administrative burden and bureaucracy associated with national as well as international funding and also the lack of transparency on how EUREKA works are considered to be the primary barriers to participate in Eurostars-1 projects. Notably, just 24 % of participants emphasized the fact that EUREKA provides just a label but no funding as being a barrier. In order to shed some more light into these issues, a more detailed survey investigating the main barriers and obstacles for participation of under-represented Participating States should be carried-out. The task force established by Eurostars members did not really address this issue. The time frame and the resource limitations did not allow tackling this aspect. Figure 9. Barriers to participate in the Eurostars-1 programme (% of respondents) Perceived bureaucracy assoc. with nat. funding Perceived bureaucracy assoc. with internat. Funding Lack of transparency on how EUREKA works Administrative barriers Finding suitable partners Management of the consortium EUREKA provides just a label, but no funding Expected low success rates Expected legal problems with partners Less attractive compared to other funding programmes Note: Multiple answers were possible. 18 Impact Assessment of EUREKA Network Projects and Cluster Projects (2017) ongoing study commissioned by the EUREKA Secretariat. 34

37 In light of these findings, the expert group proposes the following recommendations: Recommendation: Shorter time to contract will permit to avoid jeopardizing the innovation potential of projects. Time to contract should be calculated at the project level when the consortium agreement is signed and not at the participant level when they sign their grant agreement with their respective NFB. Recommendation: The EC should consider in the short term granting the EU top-up to those Eurostars projects for which grant agreements have not been signed within one year of the cut-off period. Recommendation: An in depth and representative survey of the barriers encountered by under-represented Participating States should be carried out following two vectors: exogenous barriers and endogenous barriers. A particular emphasis will be put on the role of NPCs of under-represented Participating States. Recommendation: Put in place a follow-up of the results of funded projects (i.e. FiRs) and their introduction into the market within two years after projects completion (MIRs) to better inform Participating States and the European Commission. Recommendation: The current evaluation process is too fragmented and the evaluation criteria are too numerous and narrow to have a realistic overview of the project's quality. More feedbacks should be given to the evaluators about the outcomes of the submitted projects they examined. 6.3 Effectiveness The Eurostars-2 programme targets mostly R&D performing SMEs (see Figure 1 and Figure 3) through transnational R&D market oriented (high Technological Readiness Levels) industrial collaborations. With 66% of all participants, the presence of R&D performing SMEs in Eurostars-2 is somewhat lower than it was the case in Eurostars-1 (72%). 19 The EU Funding contribution for Eurostars-2 has a significant influence on the success of the Programme, and for R&D performing SMEs. The number of projects submitted and the total number of thirty-five participating countries are also indicators of this success. Eurostars-2 is particularly effective in mobilising national funding, even though major differences exist between the participating countries (Figure 10). 19 Sources: Figure 1 and Final Evaluation of the Eurostars Joint Programme (2014). 35

38 Germany Switzerland Netherlands Denmark Sweden Spain United Kingdom Norway France Austria Belgium South Korea Finland Turkey Czech Republic Lithuania Poland Romania Bulgaria Iceland Hungary Luxembourg Slovakia Latvia Slovenia Portugal Cyprus Ireland Croatia Israel Italy Figure 10. Total NFBs commitment (share of countries in % with respect of total) As can be seen in Figure 11, Eurostars-2 contribution to Research & Development and Innovation activities in Europe and partner countries varies significantly across participating countries. Among the Top 5 countries, we found Germany (664 participants of approved proposals), The Netherlands (432); Spain (421); France (353) Switzerland (353). The countries with the lowest numbers of submitted proposals are: Greece (1), Malta (3); Estonia (4); Croatia (11); Ireland (13); Luxembourg (14). The success rate varies as well across participating countries, and is averaged at 30.6% which is considered by the stakeholders as high success rate. As can it can be seen in figure 12, the Top 5 successful applicant countries include Norway (40.0%); The Netherlands (38.9%); Switzerland (38.5%); Sweden (37.8%); Denmark (37.7%). The 5 less successful applicant countries are Estonia (0%); Greece (0%); Malta (0%); Croatia (9.1%); Cyprus (9.3%). 36

39 Norway Netherlands Switzerland Sweden Denmark Germany France Belgium Lithuania Italy Austria United Kingdom Finland South Korea Iceland Luxembourg Czech Republic Spain Ireland Portugal Turkey Latvia Poland Israel Slovenia Hungary Bulgaria Slovakia Romania Cyprus Croatia Estonia Greece Malta Germany Netherlands Spain France Switzerland Denmark Sweden United Kingdom Austria Norway Romania Belgium Czech Republic South Korea Finland Hungary Turkey Lithuania Poland Cyprus Italy Bulgaria Slovakia Slovenia Latvia Israel Portugal Iceland Luxembourg Ireland Croatia Estonia Malta Greece Figure 11. Number of participants of approved Eurostars-2 projects by country Figure 12. Success rate (in %)

40 In terms of project implementation and monitoring, a stronger control of a project s impact in terms of product innovation, successful placement of a product on the market within two years after projects completion or sustainability of transnational cooperation would be beneficial, possibly by establishing common impact measurements methodologies. The Eurostars-2 programme is overall considered to play an adequate and complimentary role in promoting excellence in technological research, development and demonstration. This is due to many aspects. First, compared to the ERA-NETS, which are sectoral programmes, Eurostars is a horizontal, bottom up approach, hence it diminishes fragmentation. It is easier to manage, to follow and makes it understandable for the beneficiaries; in addition, still in comparison with the ERA-Nets, more countries participate, allowing more options of collaboration between the stakeholders. In comparison to large national clustering programmes, it targets specifically SMEs, whilst cluster programmes are normally driven by large companies. Finally, in comparison to the SME instrument, it really fosters collaborative actions, and the evaluation process is deemed more credible and transparent. From all the above reasons, the Eurostars-2 programme sits in a unique niche and strengthens the transnational cooperation and research between SMEs and other research institutions. From a market perspective, the Eurostars-2 programme addresses successfully different technological areas and markets. The most represented technological areas are Biotech and ICT domains, the latter horizontally impacting several market areas (see Figures 4 and 5). In light of these findings, the expert group proposes the following recommendations: In the short term: Recommendation: R&D performing SMEs as eligibility criteria should be assessed at national level as it is done for the SME self-declaration. Today, this action is pursued by ESE. Recommendation: Structure and content of MIR reports should be improved in order to assess projects results on efficiency, effectiveness and pragmatism with regards to-goto-the-market. In the long term: Recommendation: Participating States should first receive MIRs and assess them on efficiency, effectiveness and pragmatism to-go-to-the-market. Their assessment per project should be addressed to the ESE as an element for the payment of the EC top-up. It will allow creating an instrument for assessment of the bottlenecks for commercialisation to build new go-to-the-market and financial tool boxes. 6.4 Coherence The main objective of Eurostars-2 is to enhance European competitiveness through a bottom-up approach aiming to support R&D performing SMEs by: Creating an easily-accessible and sustainable European R&D support mechanism; Encouraging R&D performing SMEs to create new economic activities based on their R&D results and bring new products, processes and services to the market faster than would otherwise be possible; 38

41 Promoting technological and business development and internationalisation. These companies will develop some capabilities that will permit them to access to a program that distinguishes itself from other types of public programs where SMEs have not been traditionally represented. According to an analysis carried out by ESE, in the 5 first cut offs of Eurostars-2, 51% of the R&D performing SMEs who participate in approved projects are less than 10 years old, and 49% of them have less than 10 Full- Time Equivalents (FTEs). In comparison with Eurostars-1, the percentage of companies with less than 10 FTEs with approved projects has increased. National Funding Bodies (NFBs) play a key role in maintaining a sustainable mechanism - they are the entry to the program, and the channel of first and main communication that facilitates the access for young and small SMEs with a weak structure and a short experience in an international environment. The proximity to NFBs and the sharing of the same language allow R&D performing SMEs to have a contact on a continuous basis and to solve problems and doubts during the application process and the implementation of the project. The Commission and Participating States agreed on certain amendments concerning the smooth implementation of the programme and use of the EU top-up. Convergence elements have been achieved in Eurostars-1 with the implementation of a centralized evaluation procedure operated by ESE. Yet, in Eurostars-2, no further developments of new convergence processes have been witnessed. The legal basis enabling the cooperation between the Commission, Participating States and the ESE is covered by Art. 185 of the Treaty on the Functioning of the European Union (TFEU) which stipulates that Article 185 initiatives are joint programmes established by Member States or/and Associated Countries ( Participating States ) with the financial participation of the European Union. They are designed to meet particular challenges in the research area and to complement Horizon 2020, to leverage national with EU funding, and to create economies of scales and synergies between national and EU research programmes and investments. The Eurostars programme is part of the palette of instruments put in place for the benefit of SMEs in Europe. It covers a specific niche that other EU, national and regional interventions do not address for the benefit of SMEs in Europe. The Eurostars-2 Programme cohabits with another among the most popular instruments of H2020, namely the SME Instrument. To secure their uniqueness, both programs must have their own value and vision. What makes Eurostars-2 unique is the cooperation among two or more international partners, with at least one R&D performing SME as a leader, although one should note that, in its current architecture, the programme does not prevent de facto "mono-beneficiarisation". If we analyse the composition of consortia in the first 5 cut offs of Eurostars-2, projects with at least 2-R&D performing SMEs coming from 2 different countries represents about 80% of all project funded. Furthermore, having a research centre as a partner increases significantly the success rate. 20 A main feature of the programme is the benefits it offers to all types of participants. For large companies, the benefit can be the collaboration in an innovation projects together with a small company, which by definition is more flexible and faster in incorporating innovation in new products or process. For a R&D performing SME or a public research institution, the benefit to collaborate with a larger company can rest in the higher capabilities of the latter to transfer new knowledge to the market, especially at international level. The commercialization of the products is one the main goals of the programme, and analysing data from Market Impact Reports (MIR) and Final Report (FIR) received until 20 Source: ESE internal document. 39

42 Switzerland Denmark Belgium United Kingdom Spain Lithuania Czech republic Austria Bulgaria Iceland Slovak republic Latvia Portugal Hungary Cyprus Slovenia Luxembourg Poland Malta Romania Norway Croatia Germany Ireland Netherlands Italy Finland Sweden Israel Turkey South Korea France Total mid-january 2017, reveals that 77% of the products, processes and services have already been brought to the final market, and the median time to market for commercialisation of all participants is 1.2 years, with some variations depending on the type of consortium. According to the Eurostars-1 annual report 2016, the participation is expected to bring benefits to R&D performing SMEs in terms of turnover growth and employment improvement. The FiR and MIR data show that on average, participating SMEs benefitted from additional turnover of 0.55 million after completion of an Eurostars project, up to the time of reporting. This amount increases to 0.79 million for main participants. The expected result for turnover within 2-3 years is 2.96 million and 4.47 million for main SMEs. In terms of employment, 2.3 jobs are created on average by main participants up to the time of reporting, and 1.9 for other types of partners. The number of new employees expected is 5 for main partners and 3.1 for other partners. Based on this data, we can conclude that the Eurostars program encourages SMEs to create new economic activities based on R&D results and bring new products, processes and services to the market, with an impact in their size. The impact on the technological base of the SMEs is improved as well. The MIR and FIR reports indicate that around 37% of participants report IPR activity, and for main participants this figure is of 45%. In terms of weaknesses, a real issue as indicated in Figure 13 rests in the fact that some participating states are transferring a lower amount than what was committed at the beginning of the year. This situation raises a real concern as it creates a EU funding loss should which should be avoided. This money is also lost by the EC because of the annuality of the budget allocation. As a consequence, less projects and participants can be funded as originally planned. Figure 13. Difference between total NFB commitment to participant project costs and funding received (M ) - (2014- November 2016) Source: authors own calculations based on ESE Eurostars-2 database - version as of December, In terms of coherence, the call for convergence of Art.185 also creates difficulties and tensions with Participating States that are unwilling to harmonize the national rules and procedures for Eurostars-2. Participating States should otherwise apply those rules to their national programmes given the full bottom-up feature of the Eurostars programme. Finally, ESE overall operations are highly dependent on Eurostars-2 (almost 50% of the ESE budget is dedicated to Eurostars-2 activities). Differentiation between EUREKA and Eurostars-2 activities is not well distinguished in the various governance bodies (e.g. the 40

43 systematic parallelism of the bodies of EUREKA and Eurostars-2 takes the same composition and structure) as shown in the diagram presenting the current Eureka governance Section. In light of these findings, the expert group proposes the following recommendations: In the short term: Recommendation: In order to strengthen the EU added value, at least two R&D performing SMEs from two different Participating States should be mandatory in the proposed project consortium with a fair distribution of activities and a good balanced budget (20% minimum for each R&D performing SME, i.e. 70% minimum for both). This is important to respect the main feature of Eurostars-2 Joint Programme to go to the transnational market within two years after projects completion. Recommendation: The management of the programme should foresee the total annual EU funding as requested in the Annual Work Plan (AWP). Losses of EU annual funding should be avoided. The increase of the EU top-up rate to the NFBs in order to use the left-over EU contribution is a counter-productive option because this will definitively decrease the total budget (and impact) of the joint programme. Recommendation: The Eurostars budget is around 50% of the total administrative budget of ESE including EUREKA activities. This is not perceived in the institutional governance of EUREKA. A distinctive (from EUREKA) Eurostars governance agenda should be set up. 6.5 European Added Value As an Article 185 initiative, the Eurostars-2 program aims at leveraging national efforts for the European good in a way that delivers added value for Europe. The top up contribution of the European Commission encourages member states to allocate funds in this programme thanks to this leverage effect. The EU financial contribution to Eurostars- 2 shall be up to 287 million. The Eurostars programme is a well-established and long-lasting programme and perceived by all stakeholders as an important part of the ERA. It represents an important support for R&D performing SMEs in Europe to become relevant players in the world-wide innovation ecosystem. Without exceeding this amount, the Union s contribution shall be at least one third of the contributions of the Participating States. It shall cover operational costs, including the costs of the evaluation of proposals, and administrative costs. Until the moment of the publication of this report, the level of funding allocated by the EC was 62 million, indicating that it would be necessary to increase the efforts from all of stakeholders to increment the number of applicants, and the budget of Member states, to be able to use all the budget allocated by the EC. Since the start of the Eurostars-2 programme in 2014, the budget commitment of National Funding Bodies increased on an annual basis on average very slightly (0.4%), with three countries with the highest increases, Romania (1.7%), The Czech Republic (1.5%) and Germany (1.5%), while at the other end, Finland (-2.7%), Hungary (-1.3%) and Turkey (-1.5%) exhibit a decrease of their contribution. 41

44 The number of countries active in the programme is made up of the 28 EU Members states, 4 non-eu participating states, and 2 partner countries, totalling 34 countries, but some of them with a level of participation very low or even null as Croatia, Estonia, Greece, Ireland, or Malta. Other (larger) countries exhibit a much higher participation such as for instance Germany, The Netherlands, Denmark, France, Switzerland or Spain. 8 participating countries concentrate more than half of all participating R&D SMEs: Germany, The Netherlands, United Kingdom, France, Spain, Denmark, Switzerland, Sweden and Norway. Partner countries (EUREKA member countries or EUREKA associated countries that are not EU member states or associated countries to H2020) are allowed to participate to the programme but are not entitled to receive any EU top-up. As already said, the number of applicants from different countries varies a lot, and the success rate too. To achieve the goal of the programme and deliver further added value for Europe, it would be necessary to work with the countries where the participation is low and encourage them to support the application process of domestic SMEs and other research organizations. To emphasize the European meaning of the programme, and to differentiate it from other cross border programmes, it would be necessary to reduce the number of consortia involving participants from the same country or from neighbouring countries (language border effect). The uniqueness of the programme Eurostars comes from the fact that it is the only European programme in near-the-market R&D industrial cooperations focusing on R&D performing SMEs. It embeds the subsidiarity principle between the EC and the Participating States that agreed on designating ESE as the DIS (Dedicated Implementation Structure) of the programme. Its main goal is that the results of the innovation projects steered by a R&D performing SME reach the final market in less than 2 years, with an increment in turnover and employment creation during the years after the end of the project. This type of companies often shows difficulties to participate in European R&D Cooperation Projects, and this programme supports especially young and small R&D performing SMEs to achieve for the first time the European level for their innovation projects. It is often easier for R&D performing SMEs to participate in programmes at regional and national levels, but thanks to the centralized evaluation process (with independent experts and a peer to peer panel approving the research proposal), these firms have an opportunity to test the level of their innovation and the international interest of the developed technologies. In light of these findings, the expert group proposes the following recommendations: In the short term: Recommendation: The uniqueness of the Eurostars programme in the European panoply of market creating innovation tools to support SMEs should be preserved and restrengthened. Recommendation: In order to strengthen the EU added value, at least two R&D performing SMEs from two different Participating States should be mandatory in the proposed project consortium with a fair distribution of activities and a good balanced budget (20% minimum for each R&D performing SME, i.e. 70% minimum for both). This is important to respect the main feature of Eurostars-2 Joint Programme to go to the transnational market within two years after projects completion. 42

45 7. ANALYSIS OF IMPACT OF COMPLETED EUROSTARS-1 PROJECTS Ernst and Young was contracted by the European Commission to evaluate the performance and to assess the impact of the Eurostars-1 in terms of new products, processes or services introduced into the market two years after the project s completion. The analysis was also aimed at providing concrete recommendations and comments on how to better implement the Eurostars-2. Extensive results and conclusion of this analysis are reported in Appendix 7. In light of the findings of this report, the following conclusions and recommendations are proposed. The data collection mechanism in place for the first five cut-offs of the Eurostars-1 significantly limits the monitoring and the measurement of the performance and impacts attributable to the Eurostars projects. Moreover, reliability of data included in the Final Report and Market Impact Report is limited. In addition, the submission of Final Report and Market Impact Report and the completeness of data is not granted as only a part of the beneficiaries submitted completed reports. This might indicate the reporting activity was perceived as optional by some beneficiaries. Due to the reasons mentioned above, measuring the impacts of completed Eurostars project is currently not possible in a comprehensive manner. As a result, this study was based on a sample of 182 projects for which completed information was available from the main beneficiaries. 88% of the analysed projects were successful in developing products/ services/ processes within the project implementation period and 82% of these successful projects succeeded in the commercialisation of results. Regarding the latter, in terms of impacts of the participation in the Eurostars-1 programme on the SMEs that implemented them, the main conclusion is that apart from the profitability of the investment, identified as limited, the other economic indicators for which data is available cannot be used to assess the direct impacts of the participation in the programme as they refer to the overall economic performance of the beneficiaries. The study also revealed that the participation in Eurostars-1 programme is perceived by the interviewed beneficiaries as a positive experience, having various types of benefits (i.a knowledge and network-related). However, the beneficiaries also identified barriers in the application phase (i.a. political, administrative), project implementation (i.a. financial and technical) and commercialisation phase (i.a. demand and consortiumrelated). As main recommendations, the data collection, selection and monitoring mechanisms should be improved. Recommendation: The data collection, selection and monitoring mechanisms in the ESE Database should be improved. 43

46 8. POST 2020: FUTURE OPTIONS AND MODELS The Expert Group s terms of reference request specific advice on the lessons learned from the Article initiative and the options for future European programmes in this area. In light of recent developments concerning the participation of States and the experience gained with the implementation of the programme, the issue arises whether the current model of EU participation in the Eurostars-2 Programme is still the most effective and best suited one in order to meet the EU s goals and objectives. Several instruments exist at the EC to support the co-ordination of national policies and programmes with different intensities of convergence and coordination. The choice of instrument in a given case should help maximise the European value-added of the EU s contributions under the specific framework conditions of national and co-operation activities among the participating States. Adapting and modifying Article 185 Article 185 provides the Commission with reasonable flexibility in terms of structuring its participation and with regard to the rules and procedures for the provision of financial support to national R&D activities. The current Eurostars-2 implementation model calculates the EU contribution based on the funds spent (costs incurred) by the project partners in the Participating States on a Call by Call basis. The Commission then transfers the EU contribution to the ESE which serves as the implementation structure for the Eurostars-2 Programme. The ESE distributes the EU funds to the national funding bodies that manage the funding agreements with the national project partners involved in Eurostars-2 projects. As discussed in the main report, in the past this financing model was prone to significant levels of underspending of planned EU contributions as some Participating States failed to commit sufficient resources to the Calls in order to fund all proposals selected whilst other States with budget commitments had no national applicants among the selected projects. 21 Article 185 TFEU states that "In implementing the multiannual framework programme, the Union may make provision, in agreement with the Member States concerned, for participation in research and development programmes undertaken by several Member States, including participation in the structures created for the execution of those programmes". 44

47 Other policy instruments The European Commission can support co-ordination activities of national research policies through other instruments than those based on Article 185. Other policy instruments like Joint Programming Initiative (JPI) to address grand societal challenges for Europe, ERA-Net, ERA-Net Co-Fund Actions, JTI have clearly been assessed as being not relevant to the specificity of Eurostars. At a practical level, the scope for action comes down to a handful of policy options each of which more or less maps to a specific programme model or policy instrument as described in the previous sub-section. These are summarised in Table 7 here below. Table 7. Four policy scenarios for future Eurostars Policy Approach Policy Definition and Associated Instrument 1) No EU participation No Art. 185 on a coordinated European initiative for marketoriented research cooperation of R&D-performing SMEs. Implemented through: EUREKA Network. No dedicated European policy instrument is implemented. Implications Established networks under Eurostars-2 need to be differentiated (and if not merged) from the EUREKA ones in terms of beneficiaries, governance and financing instruments. Loss of Eurostars-2 as a flagship initiative on marketoriented research cooperation of R&D-performing SMEs. 2) Continue the Programmebased national collaboration A successor programme to the current Eurostars-2 programme is established on the same/similar basis. Implemented through: Article 185 initiative More of the same, potentially continuing both the positive and negative features of the current Programme. Overcoming negative aspects would require major changes in the current structure of the consortia; more promotion of the programme at national level, set-up of national mechanisms to improve the commercialization of projects outcomes; etc. EU added value would be limited if participation continues at current levels and risk on EU's underuse of financial commitments will be significant. To mitigate these risks, a coordination between Participating States and the future EIC policy might be envisaged 3) Design a new Programmebased national collaboration Participating States work together to align their policies and increase funding, with the EC providing glue financing for the sole administrative expenditure. Convergence of National Policies will attain its momentum. Less accumulation of EU and National rules for implementation. Collaboration continues in a more informal basis: no bilateral agreements 45

48 Implemented through: Article 185 initiative between ESE and each Participating State are necessary for the allocation of the EU contribution. Less administrative burden for the EU, ESE and the Participating States. 4) EU-funded collaboration European Union programme funded in the frame of the 9 th Framework Programme under the umbrella of the future European Innovation Council pilar ("EIC") Implemented through: EU Framework Programme. Collaboration continues under the umbrella of the future EIC under the 9 th FP, not the EUREKA Framework. No more coordination of National Policies in this field. The future Eurostars will be part of the panoply of marketcreating innovation instruments under the EIC. Risk of developing "grey zones'" with other existing activities addressing innovative SMEs. The risk could be mitigated if the "uniqueness" of the programme is demonstrated with success. These options are not mutually exclusive and at this stage it would be premature to recommend one over the other. A new Article 185 might be appropriate, provided sufficient countries commit to fund such a programme over seven years (or however long the next Framework Programme will run) and agree major changes in funding and coordination (as listed in Table 7). 46

49 9. CONCLUSIONS AND RECOMMENDATIONS The major benefits of Eurostars rest in several niche features of the programme: Bottom up approach; Strengthening transnational cooperation among R&D performing SMEs; Division of work between central structure (ESE) and decentralised structures (NFBs); Introduction of new products, processes and services within two years of projects' completion; Targeting of SMEs without any previous international experience in transnational industrial R&D collaborations. On the other side, some weaknesses could also be identified: High heterogeneity of times to contract; Insufficient accuracy and up-to-date information in the ESE database; Uncertainty to get funding for selected projects when the contribution of participating States has been exhausted by other projects; Lack of synchronization of procedures for implementing the programme; Not enough Participating States at the EU level are active in the Eurostars-2 jointprogramme. Based on these conclusions and the overall analysis, the expert group summarises the following recommendations: 9.1 Key Issues In order to strengthen the EU added value, at least two R&D performing SMEs from two different Participating States should be mandatory in the proposed project consortium with a fair distribution of activities and a good balanced budget (20% minimum for each R&D performing SME, i.e. 70% minimum for both). This is important to respect the main feature of Eurostars-2 Joint Programme to go to the transnational market with innovative products within two years after projects completion. (short term) The core objective of Eurostars-2 is to introduce into the market two years after the project completion the new products, services or processes: this should be organised and effectively implemented at programme level as well as project level. The two years requirement should be a clear criterion to take into consideration in the evaluation process. Follow-up and market readiness measures should be monitored by the Participating States which have a much better knowledge of their beneficiaries and their market potential. (short term) Recommendation: R&D performing SMEs as eligibility criteria should be assessed at national level as it is done for the SME self-declaration. Today, this action is pursed by ESE. (short term) The uniqueness of the Eurostars programme in the European panoply of market creating innovation tools to support SMEs should be preserved and re-strengthened. (short term) 47

50 9.2 Programme implementation The EC should consider in the short term granting the EU top-up to those Eurostars projects for which grant agreements have not been signed within one year of the cutoff period. (short term) Shorter time to contract will permit to avoid jeopardizing the innovation potential of projects. Time to contract should be calculated at the project level when the consortium agreement is signed and not at the participant level when they sign their grant agreement with their respective NFB. (short term) The gathering of information from the centralized ESE database which should be constantly updated would constitute an advantage to monitor the overall implementation of the Joint programme. (short term) The data collection, selection and monitoring mechanisms in the ESE Database should be improved. (short term) Participating States should first receive MIRs and assess them on efficiency, effectiveness and pragmatism to-go-to-the-market. Their assessment per project should be addressed to the ESE as an element for the payment of the EC top-up. It will allow creating an instrument for assessment of the bottlenecks for commercialisation to build new go-to-the-market and financial tool boxes. (long term) Structure and content of MIR reports should be improved in order to assess projects results on efficiency, effectiveness and pragmatism with regards to-go-to-the-market. (short term) Put in place a follow-up of the results of funded projects (i.e. FiRs) and their introduction into the market within two years after projects completion (MIRs) to better inform Participating States and the European Commission. (short term) The management of the programme should foresee the total annual EU funding as requested in the Annual Work Plan (AWP). Losses of EU annual funding should be avoided. The increase of the EU top-up rate to the NFBs in order to use the left-over EU contribution is a counter-productive option because this will definitively decrease the total budget (and impact) of the joint programme. (long term) R&D performing SMEs as eligibility criteria should be assessed at national level as it is done for the SME self-declaration. Today, this action is pursued by ESE. (short term) 9.3 Governance The Eurostars budget is around 50% of the total administrative budget of ESE including EUREKA activities. This is not perceived in the institutional governance of EUREKA. A distinctive (from EUREKA) Eurostars governance agenda should be set up. (short term) An in depth and representative survey of the barriers encountered by underrepresented Participating States should be carried out following two vectors: exogenous barriers and endogenous barriers. A particular emphasis will be put on the role of NPCs of under-represented Participating States. (short term) The call for convergence of Art.185 creates difficulties and tensions with Participating States that are reluctant to further harmonize the national rules and procedures and funding rates for Eurostars-2. Article 185 TFEU may be not the most appropriate instrument in absence of further convergence potentialities. (long term). 48

51 The current evaluation process is too fragmented and the evaluation criteria are too numerous and narrow to have a realistic overview of the project's quality. More feedbacks should be given to the evaluators about the outcomes of the submitted projects they examined. (short term) 49

52 ANNEX 1: EUROSTARS-2 JOINT PROGRAMME LEGAL BASIS DECISION No 553/2014/EU OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 15 May 2014 on the participation of the Union in a Research and Development Programme jointly undertaken by several Member States aimed at supporting research and development performing small and medium-sized enterprises (Text with EEA relevance) THE EUROPEAN PARLIAMENT AND THE COUNCIL OF THE EUROPEAN UNION, Having regard to the Treaty on the Functioning of the European Union, and in particular Article 185, and the second paragraph of Article 188, thereof, Having regard to the proposal from the European Commission, After transmission of the draft legislative act to the national parliaments, Having regard to the opinion of the European Economic and Social Committee (1), Acting in accordance with the ordinary legislative procedure (2), Whereas: (1) In its Communication of 3 March 2010 entitled Europe 2020 A Strategy for smart, sustainable and inclusive growth (the Europe 2020 strategy ) the Commission emphasised the need to develop favourable conditions for investment in knowledge and innovation so as to achieve smart, sustainable and inclusive growth in the Union. Both the European Parliament and the Council have endorsed this strategy. (2) Regulation (EU) No 1291/2013 of the European Parliament and of the Council (3) established Horizon 2020 The Framework Programme for Research and Innovation ( ) ( Horizon 2020 ). Horizon 2020 aims at achieving a greater impact with respect to research and innovation by contributing to the strengthening of public-public partnerships, including through Union participation in programmes undertaken by several Member States in accordance with Article 185 of the Treaty on the Functioning of the European Union. (3) Public-public partnerships should aim to develop closer synergies, increase coordination and avoid unnecessary duplication with Union, international, national and regional research programmes, and should fully respect the Horizon 2020 general principles, in particular those relating to openness and transparency. Moreover, open access to scientific publications should be ensured. (4) By Decision No 743/2008/EC of the European Parliament and of the Council (4), the Community decided to make a financial contribution to Eurostars, a joint research and development programme undertaken by all Member States and five participating countries in the framework of Eureka, an intergovernmental initiative established in 1985 with the objective of promoting cooperation in industrial research ( Eurostars ). (5) In April 2012, the Commission communicated to the European Parliament and the Council a report on the interim evaluation of Eurostars carried out by a Group of 50

53 Independent Experts two years after the beginning of the programme. The overall opinion of the experts was that Eurostars meets its objectives, adds value to European research and development performing small and medium-sized enterprises ( SMEs ) and should be continued after Eurostars is also considered to meet a number of genuine needs of SMEs engaged in research and development; it has attracted a large number of applications, with the budget for projects eligible for funding exceeding the initial budget. A number of recommendations for improvement were made, mainly addressing the need of further integration of national programmes and improvements in the operational performance in order to reach shorter time-to-contract and more transparency in the procedures. (6) The definition of SME provided for in Commission Recommendation 2003/361/EC of 6 May 2003 concerning the definition of micro, small and medium-sized enterprises (5) should apply. (7) In accordance with Council Decision 2013/743/EU (6), support may be provided to an action building on Eurostars and reorienting it along the lines stated in its interim evaluation. (8) The second Research and Development Programme jointly undertaken by several Member States aimed at supporting research and development performing small and medium-sized enterprises ( Eurostars-2 ), aligned with the Europe 2020 strategy, its flagship initiative Innovation Union and the Commission Communication of 17 July 2012 entitled A Reinforced European Research Area Partnership for Excellence and Growth, will aim to supporting research and development performing SMEs by co-financing their market oriented research projects in any field. As such, and in combination with the activities under the Leadership in Enabling and Industrial Technologies objective set out in Horizon 2020, it will contribute to the goals of the Industrial Leadership part of that programme to speed-up development of the technologies and innovations that will underpin tomorrow s businesses and help innovative European SMEs to grow into worldleading companies. As part of the improvements from the previous Eurostars programme, Eurostars-2 should head towards shorter time-to-grant, stronger integration and lean, transparent and more efficient administration to the ultimate benefit of research and development performing SMEs. To keep the bottom-up nature and the business-driven agenda with its main focus on market potential from the previous Eurostars programme is key to the success of Eurostars-2. (9) In order to take into account the duration of Horizon 2020, calls for proposals under Eurostars-2 should be launched at the latest by 31 December In duly justified cases calls for proposals may be launched by 31 December (10) The Eureka Ministerial Conference on 22 June 2012 in Budapest endorsed a strategic vision for Eurostars-2 ( Budapest Document ). The ministers committed to support the continuation of Eurostars after its termination in 2013 for the period covered by Horizon This will consist of a reinforced partnership addressing the recommendations of the interim evaluation of Eurostars. The Budapest Document sets out two main objectives for Eurostars-2. Firstly, a structural-oriented objective to deepen the synchronisation and alignment of the national research programmes in the field of funding, which is a central element towards the realisation of the European Research Area by the member countries. Secondly a content-related objective to support research and development performing SMEs engaging in transnational research and innovation projects. The Budapest Document invites the Union to participate in Eurostars-2. (11) For the purpose of simplification, administrative burdens should be reduced for all parties. Double audits and disproportionate documentation and reporting should be avoided. When audits are conducted, the specificities of the national programmes should be taken into account, as appropriate. 51

54 (12) Audits of recipients of Union funds provided under Eurostars-2 should be carried out in accordance with Regulation (EU) No 1291/2013. (13) The Participating States intend to contribute to the implementation of Eurostars-2 during the period covered by Eurostars-2 ( ). (14) Eurostars-2 activities should be in line with the objectives and bottom-up principles of Horizon 2020 and with the general principles and conditions laid down in Article 26 of Regulation (EU) No 1291/2013. (15) A ceiling should be established for the Union s financial contribution to Eurostars-2 for the duration of the Horizon Within the limits of that ceiling, there should be flexibility regarding the Union s contribution, which should be at least one third but no more than half of the contribution of the Participating States in order to ensure a critical mass necessary to satisfy the demand from projects eligible for financial support, to achieve a high leverage effect and ensure stronger integration of national research programmes of the Participating States. (16) In accordance with the objectives of Regulation (EU) No 1291/2013, any Member State and any country associated to Horizon 2020 should be entitled to participate in Eurostars-2. (17) Any Eureka Member or country associated to Eureka that is not a Member State or a country associated to Horizon 2020 may become a Eurostars-2 partner country. (18) The Union s financial contribution should be subject to formal commitments from the Participating States to contribute to the implementation of Eurostars-2 and to the fulfilment of those commitments. Financial support under Eurostars-2 should mainly take the form of grants to projects selected following calls for proposals launched under Eurostars-2. In order to meet the objectives of Eurostars-2, the Participating States shall ensure sufficient financial contributions to fund a reasonable number of proposals selected through each call. (19) The joint implementation of Eurostars-2 requires an implementation structure. The Participating States have agreed on designating the Eureka Secretariat ( ESE ) as the implementation structure for Eurostars-2. ESE is an international non-profit association established under Belgian law in 1997 by the Eureka countries and, since 2008, is responsible for the implementation of Eurostars. ESE s role goes beyond the implementation of Eurostars, being at the same time the secretariat of the Eureka initiative, with its own governance linked to the management of Eureka projects outside of Eurostars. The Union, represented by the Commission, is a founding member of the Eureka initiative and full member of the Eureka Secretariat association. (20) In order to achieve the objectives of Eurostars-2, ESE should be in charge of the organisation of the calls for proposals, the verification of the eligibility criteria, the peerreview evaluation and the selection and the monitoring of projects, as well as the allocation of the Union contribution. The evaluation of proposals should be performed centrally by independent external experts under the responsibility of ESE following calls for proposals. The projects ranking list should be binding for the Participating States as regards the allocation of funding from the Union s financial contribution and from contribution from Participating States. (21) Overall, Eurostars-2should demonstrate clear progress towards further alignment and synchronisation of the national research and innovation programmes as a truly joint programme featuring stronger scientific, management and financial synchronisation. Stronger scientific integration should be achieved through the common definition and implementation of activities and should ensure the excellence and the high impact of the projects selected. Management integration should ensure further improvement of 52

55 operational excellence and accountability for the programme. Stronger financial integration should be based on overall and yearly adequate financial contribution by the States participating in Eurostars-2 and a high degree of national synchronisation. This should be achieved through a progressive harmonisation of national funding rules. (22) The Union s financial contribution should be managed in accordance with the principle of sound financial management and with the rules on indirect management set out in Regulation (EU, Euratom) No 966/2012 of the European Parliament and of the Council (7) and Commission Delegated Regulation (EU) No 1268/2012 (8). (23) In order to protect the Union s financial interests, the Commission should have the right to reduce, suspend or terminate the Union s financial contribution if Eurostars-2 is implemented inadequately, partially or late, or if the Participating States do not contribute, or contribute partially or late, to the financing of Eurostars-2. Those rights should be provided for in the delegation agreement to be concluded between the Union and the ESE. (24) Participation in indirect actions funded by Eurostars-2 is subject to Regulation (EU) No 1290/2013 of the European Parliament and of the Council (9). However, due to the specific operating needs of Eurostars-2, it is necessary to provide for derogations from that Regulation in accordance with Article 1(3) of that Regulation. (25) In order to facilitate the participation of SMEs which are more used to national channels and which would otherwise carry out research activities only within their national boundaries, the Eurostars-2 financial contribution should be provided in accordance with the well-known rules of their national programmes and implemented through a funding agreement directly administered by the national authorities, combining Union funding with the corresponding national funding. A derogation should therefore be made from Article 15(9), Articles 18(1), 23(1), (5) to (7), 28 to 34 of Regulation (EU) No 1290/2013. (26) Calls for proposals by Eurostars-2 should also be published on the single portal for participants as well as through other Horizon 2020 electronic means of dissemination managed by the Commission. (27) The Union s financial interests should be protected by means of proportionate measures throughout the expenditure cycle, including the prevention, detection and investigation of irregularities, the recovery of funds lost, wrongly paid or incorrectly used and, where appropriate, administrative and financial penalties in accordance with Regulation (EU, Euratom) No 966/2012. (28) The Commission, in cooperation with the Participating States, should conduct an interim evaluation assessing in particular the quality and efficiency of Eurostars-2 and progress towards the objectives set, as well as a final evaluation and prepare a report on those evaluations. (29) Upon request from the Commission, ESE and the Participating States should submit any information which the Commission needs to include in the reports on the evaluation of Eurostars-2. (30) Since the objectives of this Decision, namely to support transnational research activities performed by research-intensive SMEs and to contribute to the integration, alignment and synchronisation of national research funding programmes cannot be sufficiently achieved by the Member States due to lack of transnational dimension and of complementarity and interoperability of national programmes but can rather, by reason of the scale and impact of the action, be better achieved at Union level, the Union may adopt measures, in accordance with the principle of subsidiarity as set out in Article 5 of 53

56 the Treaty on European Union. In accordance with the principle of proportionality, as set out in that Article, this Decision does not go beyond what is necessary in order to achieve those objectives, HAVE ADOPTED THIS DECISION: Article 1 Subject matter This Decision lays down rules on the participation of the Union in the second Research and Development Programme jointly undertaken by several Member States aimed at supporting research and development performing small and medium-sized enterprises ( Eurostars-2 ) and the conditions for its participation. Article 2 Definitions For the purpose of this Decision the following definitions apply: (1) SME means a micro-, small- and medium-sized enterprises, as defined in Recommendation 2003/361/EC; (2) research and development performing SME means an SME which meets at least one of the following conditions: (a) reinvests at least 10 % of its turnover to research and development activities; (b) dedicates at least 10 % of its full-time equivalents to research and development activities; (c) has at least five full-time equivalents (for SME with no more than 100 full-time equivalents) for research and development activities; or (d) has 10 full-time equivalents (for SME with over 100 full-time equivalents) for research and development activities. Article 3 Objectives Eurostars-2 shall pursue the following objectives: (1) promote research activities that comply with the following conditions: (a) the activities are carried out by transnational collaboration of research- and development performing SMEs among themselves or including other actors of the innovation chain (e.g. universities, research organisations); (b) results of activities are expected to be introduced into the market within two years of the completion of an activity; (2) increase the accessibility, efficiency and efficacy of public funding for SMEs in Europe by aligning, harmonising and synchronising the national funding mechanisms of Participating States; 54

57 (3) promote and increase the participation of SMEs without previous experience in transnational research. Article 4 Participation in and partnership with Eurostars-2 1. The Union shall participate in Eurostars-2 jointly undertaken by Austria, Belgium, Bulgaria, Croatia, Cyprus, the Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Israel, Italy, Latvia, Lithuania, Luxembourg, Malta, the Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and the United Kingdom, (the "Participating States"), in accordance with the conditions laid down in this Decision. 2. Any Member State other than those listed in paragraph 1 and any other country associated to Horizon 2020 may participate in Eurostars-2 provided it fulfils the condition laid down in point (c) of Article 6(1) of this Decision. If it fulfils the condition laid down in point (c) of Article 6(1), it shall be regarded as a Participating State for the purposes of this Decision. 3. Any Eureka Member or country associated to Eureka that is not a Member State or a country associated to Horizon 2020 may become a Eurostars-2 partner country provided it fulfils the condition set out in point (c) of Article 6(1). Those Eureka Members or countries associated to Eureka that fulfil the condition laid down in point (c) of Article 6(1), shall be regarded as partner countries for the purposes of this Decision. Legal entities from those partner countries shall not be eligible for the Union s financial contribution under Eurostars-2. Article 5 Union s financial contribution 1. The Union financial contribution, including EFTA appropriations, to Eurostars-2 shall be up to EUR The Union s financial contribution shall be paid from the appropriations in the general budget of the Union allocated to the relevant parts of the Specific Programme, implementing Horizon 2020, established by Decision 2013/743/EU, in accordance with point (c)(vi) of Article 58(1), and Articles 60 and 61 of Regulation (EU, Euratom) No 966/2012, and in particular from appropriations under the heading "Innovation in SMEs" under Part II. 2. Without exceeding the amount laid down in paragraph 1, the Union s contribution shall be at least one third of the contributions of the Participating States referred to in point (a) of Article 7(1). It shall cover operational costs, including the costs of the evaluation of proposals, and administrative costs. Where during the lifetime of Eurostars- 2 the rate of the Union s contribution needs to be adapted, the Union s contribution may go up to a maximum of half of the contributions of the Participating States referred to in point (a) of Article 7(1). 3. An amount not exceeding 4 % of the Union s financial contribution referred to in paragraph 1 may be used to contribute to the administrative costs of Eurostars-2. Participating States shall cover the national administrative costs necessary for the implementation of Eurostars-2. Article 6 Conditions for the Union s financial contribution 1. The Union s financial contribution shall be conditional upon the following: 55

58 (a) the demonstration by the Participating States that they have set up Eurostars-2 in accordance with the objectives laid down in Article 3; (b) the designation by the Participating States or organisations designated by Participating States, of the ESE, as the structure responsible for implementing Eurostars- 2 and for receiving, allocating and monitoring the Union s financial contribution; (c) the commitment by each Participating State to contribute to the financing of Eurostars-2; (d) the demonstration by ESE of its capacity to implement Eurostars-2, including receiving, allocating and monitoring the Union s financial contribution, in the framework of indirect management of the Union budget in accordance with Articles 58, 60 and 61 of Regulation (EU, Euratom) No 966/2012; and (e) the establishment of a governance model for Eurostars-2 in accordance with Annex II. 2. During the implementation of Eurostars-2, the Union s financial contribution shall also be conditional upon: (a) the implementation by the ESE of Eurostars-2 s objectives set out in Article 3 and activities set out in Annex I, in accordance with the rules for participation and dissemination referred to in Article 8; (b) the maintenance of an appropriate and efficient governance model in accordance with Annex II; (c) compliance by ESE with the reporting requirements set out in Article 60(5) of Regulation (EU, Euratom) No 966/2012; (d) the effective payment by the Participating States of the financial contribution to all participants in Eurostars-2 projects selected for funding following the calls for proposals launched under Eurostars-2, fulfilling the commitments referred to in point (c) of paragraph 1 of this Article; (e) allocation of the funding from the national budgets for Eurostars-2 projects and the Union s financial contribution in accordance with the ranking lists of the projects; and (f) the demonstration of clear progress in the scientific, managerial and financial cooperation by the establishment of minimum operational performance targets and milestones for the implementation of Eurostars-2. Article 7 Contribution from Participating States 1. Contribution from the Participating States shall consist of the following financial contributions: (a) co-financing of the selected Eurostars-2 projects through relevant national forms of funding, mainly through grants. The Commission may use the established grant equivalence rules for the valuation of the contributions from the Participating States in forms other than grants; (b) financial contribution to the administrative costs of Eurostars-2 not covered by the Union contribution as set out in Article 5(3). 56

59 2. Each Participating State shall designate a National Funding Body (NFB) to administer financial support to the national participants in Eurostars-2 in accordance with Article 8. Article 8 Rules for participation and dissemination 1. For the purposes of Regulation (EU) No 1290/2013, the ESE shall be considered to be a funding body. 2. By way of derogation from Article 15(9) of Regulation (EU) No 1290/2013, the NFBs, under the coordination of the ESE, shall verify the financial capacity of all applicants for funding under Eurostars By way of derogation from Article 18(2) of Regulation (EU) No 1290/2013, grant agreements with beneficiaries of indirect actions under Eurostars-2 shall be signed by the NFBs concerned. 4. By way of derogation from Article 23(1), (5), (6) and (7) and Articles 28 to 34 of Regulation (EU) No 1290/2013, the funding rules of the participating national programmes shall apply to Eurostars-2 grants administered by the NFBs. Article 9 Implementation of Eurostars-2 1. Eurostars-2 shall be implemented on the basis of annual work plans. 2. Eurostars-2 shall provide financial support mainly in the form of grants to participants following calls for proposals. Article 10 Agreements between the Union and the ESE 1. Subject to a positive ex-ante assessment of the ESE in accordance with Article 61(1) of Regulation (EU, Euratom) No 966/2012, the Commission, on behalf of the Union, shall conclude a delegation agreement and annual transfer of funds agreements with the ESE. 2. The delegation agreement referred to in paragraph 1 shall be concluded in accordance with Article 58(3) and Articles 60 and 61 of Regulation (EU, Euratom) No 966/2012, and in accordance with Article 40 of Delegated Regulation (EU) No 1268/2012. It shall also set out the following: (a) the requirements for the ESE regarding the performance indicators set out in Annex II to Decision 2013/743/EU; (b) the requirements for the ESE s contribution to the monitoring referred to in Annex III to Decision 2013/743/EU; (c) specific performance indicators for the functioning of the ESE in respect of Eurostars-2; (d) requirements for the ESE regarding the provision of information on administrative costs and of detailed figures concerning the implementation of Eurostars-2; (e) arrangements regarding the provision of data necessary to ensure that the Commission is able to meet its dissemination and reporting obligations; 57

60 (f) an obligation for the ESE to sign bilateral agreements with the NFBs before any transfers of the Union s financial contribution take place, such bilateral agreements laying down the minimum operational performance targets and milestones for the implementation of Eurostars-2; (g) provisions for the publication of calls for proposals by Eurostars-2, in particular on the single portal for participants, as well as through other Horizon 2020 electronic means of dissemination managed by the Commission. Article 11 Termination, reduction or suspension of the Union s financial contribution 1. If Eurostars-2 is not implemented or is implemented inadequately, partially or late, the Commission may terminate, proportionately reduce, or suspend the Union s financial contribution in line with the actual implementation of Eurostars If the Participating States do not contribute, contribute partially or late to the financing of Eurostars-2, the Commission may terminate, proportionately reduce, or suspend the Union s financial contribution, taking into account the amount of funding allocated by the Participating States to implement Eurostars-2. Article 12 Ex-post audits 1. ESE shall ensure that ex-post audits of expenditure on indirect actions are carried out by the respective NFBs in accordance with Article 29 of Regulation (EU) No 1291/ The Commission may decide to carry out itself the audits referred to in paragraph 1. In such cases, it shall do so in accordance with the applicable rules, in particular the provisions of Regulations (EU, Euratom) No 966/2012, (EU) No 1290/2013 and (EU) No 1291/2013. Article 13 Protection of the financial interests of the Union 1. The Commission shall take appropriate measures ensuring that, when actions financed under this Decision are implemented, the financial interests of the Union are protected by the application of preventive measures against fraud, corruption and any other illegal activities, by effective checks and, if irregularities are detected, by the recovery of the amounts wrongly paid and, where appropriate, by effective, proportionate and dissuasive administrative and financial penalties. 2. The ESE shall grant Commission staff and other persons authorised by it, as well as the Court of Auditors, access to its sites and premises and to all the information, including information in electronic format, needed in order to conduct their audits. 3. The European Anti-Fraud Office (OLAF) may carry out investigations, including onthe-spot checks and inspections, in accordance with the provisions and procedures laid down in Council Regulation (Euratom, EC) No 2185/96 (10) and Regulation (EU, Euratom) No 883/2013 of the European Parliament and of the Council (11), with a view to establishing whether there has been fraud, corruption or any other illegal activity affecting the financial interests of the Union in connection with an agreement or decision or a contract funded under this Decision. 58

61 4. Contracts, grant agreements and grant decisions resulting from the implementation of this Decision shall contain provisions expressly empowering the Commission, the Court of Auditors, OLAF and the ESE to conduct such audits and investigations, in accordance with their respective competences. 5. In implementing Eurostars-2, the Participating States shall take the legislative, regulatory, administrative or other measures necessary to protect the Union s financial interests, in particular to ensure full recovery of any amounts due to the Union in accordance with Regulation (EU, Euratom) No 966/2012 and Delegated Regulation (EU) No 1268/2012. Article 14 Communication of information 1. At the request of the Commission, the ESE shall send any information necessary for the preparation of the reports referred to in Article The Participating States shall submit to the Commission, through the ESE, any information requested by the European Parliament, the Council or the Court of Auditors concerning the financial management of Eurostars The Commission shall include the information referred to in paragraph 2 of this Article in the reports referred to in Article 15. Article 15 Evaluation 1. By 30 June 2017, the Commission shall carry out, in close cooperation with the Participating States and with the assistance of independent experts, an interim evaluation of Eurostars-2. The Commission shall prepare a report on that evaluation which includes the conclusions of the evaluation and observations by the Commission. The Commission shall send that report to the European Parliament and to the Council by 31 December The result of the interim evaluation of Eurostars-2 shall be taken into account in the interim evaluation of Horizon At the end of the Union s participation in Eurostars-2, but no later than 31 December 2022, the Commission shall conduct a final evaluation of Eurostars-2. The Commission shall prepare a report on that evaluation which is to include results of that evaluation. The Commission shall send that report to the European Parliament and to the Council. Article 16 Entry into force This Decision shall enter into force on the twentieth day following that of its publication in the Official Journal of the European Union. Article 17 Addressees This Decision is addressed to the Member States. Done at Brussels, 15 May For the European Parliament 59

62 The President M. SCHULZ For the Council The President D. KOURKOULAS (1) Opinion of 10 December 2013 (not yet published in the Official Journal). (2) Position of the European Parliament of 15 April 2014 (not yet published in the Official Journal) and decision of the Council of 6 May (3) Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 December 2013 establishing Horizon 2020 the Framework Programme for Research and Innovation ( ) and repealing Decision No 1982/2006/EC (OJ L 347, , p. 104). (4) Decision No 743/2008/EC of the European Parliament and of the Council of 9 July 2008 on the Community s participation in a research and development programme undertaken by several Member States aimed at supporting research and development performing small and medium-sized enterprises (OJ L 201, , p. 58). (5) OJ L 124, , p. 36. (6) Council Decision 2013/743/EU of 3 December 2013 establishing the specific programme implementing Horizon 2020 the Framework Programme for Research and Innovation ( ) and repealing Decisions 2006/971/EC, 2006/972/EC, 2006/973/EC, 2006/974/EC and 2006/975/EC (OJ L 347, , p. 965). (7) Regulation (EU, Euratom) No 966/2012 of the European Parliament and of the Council of 25 October 2012 on the financial rules applicable to the general budget of the Union and repealing Council Regulation (EC, Euratom) No 1605/2002 (OJ L 298, , p. 1). (8) Commission Delegated Regulation (EU) No 1268/2012 of 29 October 2012 on the rules of application of Regulation (EU, Euratom) No 966/2012 (OJ L 362, , p. 1). (9) Regulation (EU) No 1290/2013 of the European Parliament and of the Council of 11 December 2013 laying down the rules for participation and dissemination in Horizon 2020 the Framework Programme for Research and Innovation ( ) and repealing Regulation (EC) No 1906/2006 (OJ L 347, , p. 81). (10) Council Regulation (Euratom, EC) No 2185/96 of 11 November 1996 concerning onthe-spot checks and inspections carried out by the Commission in order to protect the European Communities financial interests against fraud and other irregularities (OJ L 292, , p. 2). (11) Regulation (EU, Euratom) No 883/2013 of the European Parliament and of the Council of 11 September 2013 concerning investigations conducted by the European Anti-Fraud Office (OLAF) and repealing Regulation (EC) No 1073/1999 of the European Parliament and of the Council and Council Regulation (Euratom) No 1074/1999 (OJ L 248, , p. 1). 60

63 ANNEX I Implementation of Eurostars-2 1. The ESE shall organise continuously open calls for proposals, with cut-off dates for the award of financial support to indirect actions. 2. Applicants shall submit project proposals to the ESE as a single entry point. 3. After the closure of a call for proposals, a central eligibility check shall be carried out by the ESE on the basis of the eligibility criteria set out in the annual work plan. No different or further eligibility criteria may be added by the Participating States. 4. The NFBs, under the coordination of the ESE, shall verify the financial capacity of the participants according to common, clear and transparent rules. 5. Eligible proposals shall be evaluated centrally and ranked by a group of external independent experts in accordance with the criteria set out in Article 15(1) of Regulation (EU) No 1290/2013, on the basis of transparent procedures. 6. The ESE shall provide an evaluation review procedure in accordance with Article 16 of Regulation (EU) No 1290/ The ranking list, approved as a whole by the Eurostars-2 high-level group referred to in Annex II, shall be binding for the allocation of funding from the national budgets for Eurostars-2 projects. 8. Once the ranking list is approved, each Participating State shall finance its national participants in those projects selected for funding through the designated NFB, making all possible efforts to ensure that the top-50 ranked projects and at least 50 % to 75 % of the projects above thresholds are funded. The financial contribution to the participants shall be calculated according to the funding rules of the national programme of the relevant Eurostars-2 Participating State. The Union s financial contribution shall be transferred by the ESE to the NFBs provided that the NFBs have paid their financial contribution to the projects. 9. All eligible participants in projects selected centrally shall be funded. The granting of financial support by the NFBs to project participants selected centrally shall be subject to the principles of equal treatment, transparency and co-funding. 10. The ESE shall be responsible for evaluating proposals, informing NFBs, coordinating the synchronisation process, monitoring projects through project reporting and audits carried out by NFBs, and reporting to the Commission ensuring a short timeto-grant. It shall also take appropriate measures to encourage recognition of the Union s contribution to Eurostars-2, both to the programme itself and to individual projects. It shall promote appropriate visibility for the Union s contribution through the use of the Horizon 2020 logo in all published material, including printed and electronic publications, related to Eurostars ESE shall conclude Eurostars-2 bilateral agreements with the NFBs of Participating States. Those Eurostars-2 bilateral agreements shall set out the responsibilities of the contracting parties in accordance with the Eurostars-2 rules, objectives and implementation modalities. The Eurostars-2 bilateral agreements shall include the rules governing the transfer of the Union s contribution and the minimum operational targets and national progressive milestones for further integration and synchronisation of national programmes, including a shorter time-to-grant in accordance 61

64 with Regulation (EU, Euratom) No 966/2012 and Regulation (EU) No 1290/2013. Those targets and milestones shall be agreed by the Eurostars-2 high-level group in consultation with the Commission. The signature of the Eurostars-2 bilateral agreement and compliance with the operational targets and milestones shall be a pre-condition for NFBs to receive the Union s contribution. 12. The ESE may conclude Eurostars-2 bilateral agreements with the NFBs of partner countries. Those Eurostars-2 bilateral agreements shall set out the responsibilities of the contracting parties in accordance with the Eurostars-2 rules, objectives and implementation modalities, specify the conditions under which partnership with Eurostars-2 shall take place, and include the minimum operational targets, including a short time-to-grant. 13. Networking activities and exchange of best practices shall also be organised amongst the Participating States in order to promote stronger integration at scientific, managerial and financial level. 14. Other activities shall also include brokerage, programme promotion and networking activities with other stakeholders (investors, research and innovation providers, intermediaries) mainly to widen participation from beneficiaries in all Participating States and to involve SMEs with no prior experience in transnational research projects. ANNEX II Governance of Eurostars-2 1. The ESE shall manage Eurostars-2. The Head of the ESE, as the legal representative of the ESE, shall be in charge of implementing Eurostars-2 by: (a) preparing the annual budget for the calls, central organisation of joint calls for proposals and reception of the proposals as single entry point; the central organisation of the eligibility and evaluation of proposals, according to common eligibility and evaluation criteria, central organisation of the ranking and selection of proposals for funding, and project monitoring and follow-up; the receipt, allocation and monitoring of the Union contribution; (b) collecting the necessary information from the NFBs for the transfer the Union contribution; (c) promoting Eurostars-2; (d) reporting to the Eurostars-2 high-level group and the Commission on the Eurostars-2 programme; (e) informing the Eureka network about the activities of Eurostars-2; (f) signing the delegation agreement with the Commission, the bilateral agreements with the NFBs and the contracts with the experts assessing Eurostars-2 applications; (g) adopting the Eurostars-2 annual work plan following the prior agreement of the Eurostars-2 high-level group and of the Commission. 62

65 2. The Eurostars-2 high-level group, composed of the national representatives in the Eureka High Level Group of the Eurostars-2 Participating States, shall supervise the operations of the ESE on Eurostars-2 by: (a) (b) (c) supervising the implementation of Eurostars-2; appointing the members of the Eurostars-2 Advisory Group ( EAG ); approving the annual work plan; (d) approving the ranking list of Eurostars-2 projects to be funded and taking the award decision. The Union, represented by the Commission, shall have the status of observer in the Eurostars-2 high-level group. The Commission shall be invited to participate at the meetings, shall receive all meeting documents and may take part in the discussion. Any partner country shall have the right to send representatives to meetings of the Eurostars-2 high-level group as observers. 3. The EAG shall be composed of Eureka National Project Coordinators (persons in the national government or agency dealing with the operational level of the management of Eureka/Eurostars programme and in charge of the promotion of Eurostars-2 in the Participating States) from the Participating States. The Commission and partner countries shall have the right to send representatives to the EAG meetings as observers. The EAG meetings shall be chaired by the ESE. The EAG shall advise the ESE and the Eurostars-2 high-level group on the arrangements for the implementation of Eurostars The NFB shall be in charge of the administration of financial support to the national participants. 63

66 ANNEX 2: MANDATE OF THE INTERIM EVALUATION EXPERT GROUP The interim evaluation shall be carried out by the Commission in accordance with article 15 of the Eurostars-2 Decision 22. The evaluation will start by finding out how the situation has evolved since the Programme began and how the Programme has been implemented. The interim evaluation should examine the Eurostars-2 Joint Programme at programme and project's level. For the project level assessment of the impact of the Eurostars-1 ended projects will be taken into account as well. Special emphasis will be put on the implementation and the economic impact of the programme and if the original objectives of the Programme and their evolvement in its first two-years of implementation. The evaluation shall also audit the governance of the programme and access its evolution in particular as far as its impact in the market is concerned. The results of the interim evaluation of Eurostars-2 Joint Programme will be included in the Horizon 2020 Interim Evaluation. Terms of Reference are as follows: Purpose and Scope The purpose of the Expert Group ("the Group") is to assist the European Commission in carrying out the interim evaluation of Eurostars-2 Joint programme. The Group is required to prepare a report and to make recommendations based on available relevant evidence, covering all aspects of the interim evaluation as set out in the legal basis (see Annex 1) and respecting the better regulation Guidelines on evaluation. The Group will synthesise the information included in wide range of inputs studies (both horizontal and thematic) together with other relevant documents to be provided by the Commission. These documents comprising the evidence-based will be structured according to the five main evaluation criteria and related evaluation questions prescribed by the Better regulation guidelines in order to ensure a coherent approach and to facilitate the aggregation of information. The evaluation should provide comprehensive answers to the following five main evaluation criteria and accompanying evaluation questions: a) Relevance and appropriateness: To what extent is the Eurostars-2 Joint Programme relevant with respect to the demands of the involved Participating States and of the beneficiaries? Do the objectives still correspond to the needs of the Horizon 2020 Framework Programme? To what extent is Eurostars-2 appropriate to support the realization of the EU policy objectives especially ERA, Innovation Union, the three "O", thematic sector policies including 'Innovation in SMEs'? 22 According to Article 15 (1) of Eurostars-2 Decision "by 30 June 2017, the Commission shall carry out, in close cooperation with the Participating States and with assistance of independent experts, an interim evaluation of Eurostars-2. The results of the interim evaluation shall be presented to the European Parliament and the Council by 31 December 2017." 64

67 b) Efficiency and use of resources: a. Efficiency with respect to the specific and operational objectives of the initiative as laid down in its basic act [Decision No 553/2014/EC of the European Parliament and of the Council of 14 May 2014] and the corresponding Impact Assessment (IA), including: a.1. Progress made towards achievement of the following specific objectives: - Promotion of transnational market-oriented research activities for R&D performing SMEs in any field, leading to the introduction of new or improved products, processes or services in the market by the participating SMEs; - Contribution to the completion of the ERA and increasing the accessibility, efficiency and efficacy of public funding for R&D performing SMEs in Europe by aligning, harmonising and synchronising the national funding mechanisms. a.2. Progress made towards achievement of the following operational objectives taking into consideration the indicators and targets as mentioned in Chapter 8 of the IA: - Three years after the end of each project, for each 1 M of public funding (from EU and participating Eurostars countries), on average, the turnover of the participants should increase by at least 10M, at least 25 new jobs should be created and three new or improved products, processes or services should be on the market; - Scientific integration of national programmes: Ensure excellence and impact of the projects selected through international (EUREKA initiative) competition and the application of a single evaluation and selection process; - Management integration of national programmes: Further improve operational excellence and accountability for the programme by reducing the time to contract while maintaining an optimal frequency of calls per year; - Financial integration of national programmes: Harmonisation of national funding rules and application of a binding ranking list; - Facilitate the participation of R&D performing SMEs without previous experience in transnational R&D activities. b. Efficiency with respect to the implementation structures of Eurostars-2, including b.1. How efficient is the programme implementation by ESE in collaboration with the National Funding Bodies. What were its main limitations in relation to National Funding Bodies (NFBs) during the execution of the Programme? What could be the lessons learnt for the future? b.3. What resources are needed at the different levels (Commission services, Participating States and their NFBs) for the preparation and implementation of the initiative? Are these justified by the scale and scope of the initiative? c. Efficiency with respect to Eurostars-2 as an instrument to foster transnational R&D cooperation within Europe, including: c.1. To what extent has Eurostars-2 been cost-effective? Were the costs involved justified, given the changes/effects which have been achieved? Is the operational performance of EUREKA/ESE and its role proportionate to its allocation of budget? 65

68 c.2. Is the initiative implemented in an efficient way? Have the management aspects been properly addressed? Are effective monitoring and supervision arrangements in place to ensure adequate monitoring of the initiative? c.3. Has the initiative been implemented in accordance with the provisions of the adopted work plans, budgets and the delegation agreement? Are the reporting requirements, including audit provisions efficient? d. What kind of approaches could be considered to generate further efficiency gains? c) Effectiveness: What have been the (qualitative and quantitative) (expected) impacts of Eurostars-2 - At programme and project levels, in particular: At Programme level: - To what extent is Eurostars-2 able to mobilize national and European funding for R&D performing SMEs? How strong is the leverage effect from the Union contribution? Is the programme sufficiently accessible, in particular for the R&D performing SMEs? Are there identified "barriers to entry"? - How does Eurostars-2 compare to the 'mainstream' EUREKA with regard to the participation of SMEs, synchronisation of funding, type of projects etc. - What's the efficiency of the mechanisms and tools ensuring the entry-into-themarket of results/achievements of Eurostars-2 ended projects? - To what extent are the (expected) impacts of Eurostars-2 in line with its objectives? At Project Level: - How effectively are the projects managed by the participants? - What are the main economic and social impacts for R&D performing SMEs participating in Eurostars-2? What is the impact in terms of new products/processes/services or significant improvement of existing ones? What is the impact in terms of competitive position, company profile, employment, qualification of staff, R&D investments and attitude towards transnational collaboration? - What is the added value for an R&D-performing SME to participate in Eurostars-2 (i.e. what is the return on investment)? What is the motivation of other participant types to participate? Do the benefits from participating outweigh the costs? - What improvements/modifications are proposed to enhance the participation of SMEs to the programme and maximise the benefits they can get from their participation? - How are the SMEs using the IPR resulting from the project? - Is the current project format (at least one R&D performing SME and another partner from a different country) appropriate with regard to Eurostars-2 objectives? - Does Eurostars-2 play an adequate role in promoting excellence in scientific and technological research, development and demonstration in the considered field and impact of the projects selected? - Does Eurostars-2 play an adequate role in supporting innovation in the considered field? - Does Eurostars-2 play an adequate role in positioning Europe on the global map of science and technology in research in the concerned field? 66

69 d) Coherence: a. External coherence: To what extent is Eurostars-2 coherent with other EUlevel interventions which have similar objectives? What are the relations (i) complementarity, (ii) synergies including with other funding programmes such as the ESIF Regional Funds, collaborative projects were SMEs are participating in the frame of bilateral S&T&I agreements between the Participating States (iii) and potential overlaps (iv)? b. Is the performance of Eurostars-2 in line with the spirit of Article 185 TFEU? Are the procedures implemented at national level, coherent and aligned to ensure the best performance of the Programme? c. Internal coherence: How was the interaction between the different bodies intervening in Eurostars-2 governance (EUREKA High Level Group (HLG), Eurostars HLG, Eurostars-2 Advisory Group, EUREKA Secretariat, NPCs, National Funding Bodies, European Commission)? Did their role evolve during the execution of the programme? Did the role of the Commission change? e) European Added Value 23 : a. What is the additional value resulting from the EU intervention(s), compared to what could be achieved by Member States at national, regional and/or local levels? b. What is the added-value of the Eurostars-2 Programme compared to other forms of support to R&D&I in Horizon 2020 Framework programme (in particular SME-Instrument, collaborative projects in the Thematic priorities of LEITs and Societal Challenges, joint technology initiative, ERA-net)? Any other question that the independent experts feel necessary to address for accomplishing their task can be added by the Group after consultation with the European Commission. 23 Concerning the aspect of EU-added value, the findings of a currently on-going horizontal study dedicated to analyzing the EU added value of FP interventions will be made available as input for the Group. 67

70 ANNEX 3: ANALYSIS OF IMPACT OF COMPLETED EUROSTARS- 1 PROJECTS The scope of the study consisted in the 400 Eurostars-1 ended projects that have been funded during the period (i.e. from cut-off 1 to cut-off 5) and their impacts on the market in terms of turnover generated, company growth, new market development, increase of employees, innovation, sustainability, economic and social impact, complementarities in terms of impacts with other Horizon 2020 initiatives in particular SME Instrument Phase 2 projects and collaborative projects under the Societal Challenges and Leadership in Enabling and Industrial Technologies. In terms of the methodology implemented, three main tasks have been undertaken in order to achieve the objectives of the study, namely: desk research, interviews and results analysis. First, desk research has been performed and included the following activities: Literature review of policy documents and analysis of the policy papers and other relevant policy literature mainly aimed to investigate specific aspects of the Eurostars- 1 programme such as: addressed needs, objectives, governance, funding mechanism, monitoring setup; Analysis of the contents and of the quality of project reporting questionnaires (Final Report and Market Impact Report) and analysis of existing data concerning Eurostars- 1 implemented projects mainly data extracted from the Final Reports and Market Impact Reports and included in the Eurostars database, in order to identify eventual data gaps, issues of data quality and possible data needs that may affect the results of the impact evaluation. Cross-checks of data declared by Eurostars-1 beneficiaries (in Final Reports, Market Impact Reports and during the interviews performed within the current study) with data extracted from Amadeus database. Two series of interviews were successively conducted in the next phase of the study, namely: Short interviews with main partners (or, if not available, with other appointed partners) aimed to construct an overview at programme level regarding projects completion and commercialisation of project results, the following questions being addressed: o o When did the project end? (Please specify the date.); Have all the initially planned products / services / processes been introduced in the market? (Yes/No. If yes, please specify the date. If no, why?). In-depth interviews with main partners from a representative sample of 30 projects which allowed the Research Team to collect detailed information regarding: added value of the participation in Eurostars-1, success and hampering factors for achieving the project results, potential areas of improvement. The last task consisting in the results analysis focused first on constructing at programme level the profile of submitted and approved applications, implemented projects and corresponding beneficiaries. Descriptive statistics have been created reflecting the following dimensions: distribution by cut-off, size and structure of the consortia, 68

71 geographical distribution, EU commitment, distribution by technological area addressed by the projects. Secondly, the assessment of impacts of Eurostars-1 projects was performed based on three types of analyses: An analysis at commercial level aimed at measuring the effectiveness of the Eurostars-1 projects, in terms of: (1) achievement of the expected results (i.e. developing a product/process/service) and (2) effectiveness in introducing a product/process/service in the market; An analysis at beneficiary level focused on the impacts deriving from the participation in the programme in terms of company growth, jobs created and profitability of the investment. An in-depth analysis has been performed on a representative sample of 30 projects in order to investigate topics such as: reasons for applying, previous experience and collaboration with project partners, project results and benefits on the beneficiaries, sustainability of project benefits and results, hampering and fostering factors for introducing the project results in the market. The study consisted also in identifying complementarities of the Eurostars-1 programme with other Horizon 2020 initiatives through both desk research and interviews. In terms of findings, the quality analysis of available data first focused on the design of the Final Report and Market Impact Report which reflect the projects and beneficiaries achievements at project completion, respectively, after the introduction of the developed products, processes or services in the market. The following main findings emerged: (1) some general and/or unclear questions partially useful for monitoring are included in the reports; (2) insufficient information is requested for monitoring innovation created; (3) some requested economic indicators are not related to the impact of the programme; (4) insufficient information is requested on the project return on investment. The second phase of quality analysis of available data continued with the assessment of the accuracy of the Eurostars Database. Some inconsistencies and missing information were identified, such as: (1) information related to the participants role within the project (leading or partner) and to the situation monitoring (project withdrawn or completed) was in some cases different in various spreadsheets; (2) even if the Final Report form was sent for completion to all Eurostars participants, only 88% of them provided the required data, some of them only partially; (3) impacts on economic indicators, prior and after project completion, can be estimated for 23% of the total number of beneficiaries, as only them submitted complete data both for Final Report and Market Impact Report; (4) the data reported for some economic indicators contain several typos, resulting in severe misunderstandings; (5) the database does not contain information about the number and type of planned products, only related to the launch of products and the code of products is different in each spreadsheet, the status regarding the market launch couldn t be tracked. Considering the above-listed findings on the accuracy of the Eurostars Database which resulted from an analysis of internal logic of data, the Research Team decided to perform further checks with the company data available in Amadeus. Checks on the annual turnover and the number of employees were performed. In more than 50% of cases the deviations were higher than 10%, a lower turnover and a higher number of employees, respectively, being declared by the main partners in the Final Report. 69

72 The accuracy of data included in the Eurostars Database regarding (1) the end-date of the projects and (2) the date of market introduction of developed products/processes/services was also checked based on short interviews aimed to involve main partners of all analysed projects. Only 16% of the main partners were available to participate in the interviews. In 60% of the cases, differences (from 1 month to 2 years) regarding the end-date of the projects were declared. Regarding the submitted and approved applications, only 27.4% of the submitted ones were approved. The success rate of applications submitted decreased from the 1st to the 5 th cut-off. The trend in terms of accepted beneficiaries mirrors the accepted projects. The number of approved beneficiaries was stable only during the first three cut-offs, and then decreased during the 4 th and 5 th calls. Small (i.e. with 2 or 3 participants) and large consortia (i.e. with a number of participants greater than 5) registered similar success rates. The involvement of universities and research institutes did not prove to have a positive contribution to the approval of the applications; the highest success rates corresponded to consortia with only R&D SMEs, with a higher number of R&D SMEs, and with main partners located in the UK, France and Sweden. However, in terms of nominal values, France, Spain, Sweden and Germany had the highest number of selected applications; in fact, these were the countries with the highest number of submitted applications. As regards all project partners, the highest success rates were achieved by applications with partners located in Switzerland, France and Sweden, while in nominal values, Germany, France and Spain had the highest number of submitted and selected applications. Regarding the implemented projects, out of the 400 applications approved, only 370 projects have been implemented, the remaining 30 projects being withdrawn. However, the lower number of projects implemented during the 4 th and 5 th cut-offs is correlated with the success rates registered during the selection process, and not with the withdrawals. The highest number of withdrawals is related to the 2 nd cut-off. Out of the total approved applicants 11% decided to withdraw from the programme; some were related to the 30 withdrawn projects, but also to other implemented projects. The 370 projects were implemented by 1,197 beneficiaries. The size of the consortium varied between only 2 project partners to up to 9 project partners, the majority of the projects having either 2 or 3 partners. Given the nature of the programme, the highest number of participants was represented by R&D SMEs; however, an important number of universities, research institutes, SMEs and large companies participated in the programme. In particular, in 33% of the cases, the consortia were formed only by R&D SMEs, while in 67% of the cases, other types of institutions were also involved. A high number of main partners were located in France, followed by Sweden, Spain, Germany and the UK. On the other hand, when discussing about all project partners, the hierarchy is dominated by Germany, followed by France, Spain, Sweden and Italy. Within the analysed cut-offs, EUR 59.8 million were committed from EU funds to the project implementation. Both the total EU commitment and the average EU commitment per project were decreased towards the last cut-offs. Eurostars projects can address any technological area for any market, but must have a civilian purpose and be aimed at the development of a new product, process or service. Considering the first 5 cut-offs, 86% of the projects addressed three technological areas, namely ICT, Biotech, and Industrial. The commercial analysis revealed that 88% of the analysed projects developed the planned products/processes/services. Around 50% of them were coordinated by R&D performing SMEs based in France, Spain, Germany, Sweden and the Netherlands. The 70

73 successful projects cover all the technological areas. However, 73% of them refer to ICT, Industrial and Biotech. Moreover, 82% of these projects succeeded in introducing the results in the market. The success rate was higher among the projects funded under the first three calls, having smaller consortia including R&D SMEs plus other types of beneficiaries and addressing Environment, ICT and Industrial technological areas. The analysis at beneficiary level highlighted that SMEs that implemented successful projects in terms of commercialising the project results appear to have higher annual turnover, earnings and number of employees, and a lower annual investment in R&D than the other SMEs. In addition, over the monitoring period, they benefited from a higher growth of the turnover and earnings and they faced a lower increase of the investment in R&D. As a general highlight of a study, the projects for which data are available showed a limited profitability of the investment undertaken. The in-depth analysis of 30 randomly selected projects indicated no significant differences between beneficiaries with successful and not successful projects in commercialising the projects results in terms of: reasons for applying to Eurostars-1 programme (i.a. access to funding, entering new markets, development of internal capabilities, creation of long-term partnerships); previous experience in R&D activities, most of them declaring that are experienced; collaboration with project partners, most of them declaring smooth collaboration during the project implementation; project benefits, main declared benefits referring to knowledge, network, reputation, competitive position and internationalisation; sustainability of project benefits and results, most of them declaring they are still active in the market and involved in R&D activities; hampering (i.a. political financial, technical, administrative) and fostering factors (i.a. created partnership, market knowledge, successful research results) for introducing the project results in the market. The main conclusions revealed by the performed analyses are presented below: The data collection mechanism (i.e. design and content of FiR and MIR) in place for the first five cut-offs of the Eurostars-1 significantly limits the monitoring and the measurement of the performance and impacts attributable to the Eurostars projects as: - FiR and MIR contains information related to the evolution of the overall financial situation of the beneficiaries. However this is not directly attributable to the participation in Eurostars; - the design of the FiR and MIR did not differentiate between types of beneficiaries in order to adapt the data collected to the type of beneficiary (i.e. currently universities have to declare the turnover achieved); - the data collected from beneficiaries and included in FiR and MIR did not allow the identification and monitoring of individual products/services/processes that were planned and that have been developed/planned as well as the link between granted patent and products/services/processes cannot be deducted; Reliability of data included in FiR and MIR is rather limited. All information are collected on a declarative base and reliability and coherence of data is not ensured by an effective validation process: - the method and type of documents used to collect and integrate data and information provided by the beneficiaries in FiRs and MIRs might have caused some inconsistencies and errors in the aggregated source, the Eurostars database (i.e. not updated information, contradictory data 71

74 included in different sections of the database), which could have been determined among others by: the use of different units of measurement in reporting quantitative data, different understanding on the requested information, typing mistakes in transferring data from the original document submitted by the beneficiary in the Eurostars database; - The Eurostars database contains information that appears non-consistent with data included in other databases (e.g. Amadeus). In 70% of the cases, we found deviations higher than 10% for the declared Turnover and for the declared Number of Employees. - discrepancies appeared during the cross-checks performed by the Research Team between different beneficiaries declarations (i.e. in some cases the declared end date and first commercialisation date corresponding to the Eurostars-1 projects differed between the Eurostars database and the phone interviews conducted within the study); The submission of FiR and MIR and the completeness of data is not granted: only 21.5% of beneficiaries (i.e. 258 out of 1,197) have submitted completed reports, this allowed the analysis only of around 50% (i.e. 182 out of 370) of the completed projects. This might indicate the reporting activity was perceived as optional by some beneficiaries. Due to the reasons mentioned above (availability, reliability and usefulness of data), measuring the impacts of completed Eurostars project is currently not possible in a comprehensive manner. As a result, this study was based on a sample of 182 projects for which completed information was available from the main beneficiaries. Considering the projects for which data are available, most of the analysed projects (160 out of 182, representing 88% of them) were successful in achieving project results (i.e. developing products/ services/ processes within the project implementation period); moreover, a large part of them (131 out of 160, representing 82% of them) succeeded in commercialising the project results. Consortia that managed the projects that successfully introduced a product/service/process in the market included SMEs that: - have higher annual turnover and have benefited from a higher growth of the turnover over the six-year period considered (including the implementation and postimplementation phases); - have a lower annual investment in R&D as well as faced a lower increase of the investment in R&D over the six years; - have higher annual earnings and they benefited from a higher increase in earnings (5% compared to -4%), both in absolute value as well as in relative terms (i.e. as % of the turnover). - have higher number of employees and created on average three new jobs after the project implementation. However, there is no difference concerning the percentage of employees involved in R&D activities; in general SMEs that participated in Eurostars-1 projects allocated 42% of their FTEs to R&D activities with no significant variation before/after the completion of project activities. Projects for which data are available showed a limited profitability of the investment undertaken. Data available indicated that on average a SME invested more than 1.2 million and benefited from an additional turnover equal to 682 thousand (i.e. less than 50% of the overall investment). All interviewed beneficiaries consider the participation in the Eurostars-1 programme a positive experience and highlighted benefits in terms of knowledge, network, reputation, competitive position and internationalisation, as revealed by the in-depth analysis of 30 selected projects. From the interviews with beneficiaries it emerged that the following factors foster the commercialisation of project results: access to funding for projects with a high-risk and strategic potential, well-established partnerships with a clear division of rights and responsibilities, initial maturity of the planned products/services/processes, market knowledge and strong strategy, support offered by ESE. Some barriers existed in different phases of the Eurostars-1 programme, as perceived by the beneficiaries, them mainly referring to the following: - application phase: political (i.e. at National Project Coordinators level), financial (i.a. insufficient national funds), technical (i.a. disqualification of one project partner), administrative (i.a. different format of the application form applied at European and national level) and partner-related barriers (i.a. lack of experience in applying for European funds); - project implementation: barriers regarding the project setup and management (i.e. importance given to different project activities, difficulties in coordination multiple partners), financial, technical and administrative bottlenecks (i.a. differences between national granting procedures across EU Member States, unexpected extensive period of testing, administrative 72

75 burden existing at national level), R&D results not as expected, reduced support within the participating organisations; - commercialisation phase: external (i.a. changes in the regulatory landscape), demandrelated (i.a. non-receptive European market), internal consortium (i.a. changes in the market strategy), market readiness of the project results, objective and project setup problems (i.a. insufficient funding foreseen). The analysis was also aimed at providing concrete recommendations and any other useful comments on how to better implement the Eurostars-2 programme. Consequently, the following recommendations could be taken into account: R1: Improving the data collection mechanism by: - restructuring the project reporting questionnaires in order to collect information that would reflect the performance and impacts of the participation in the Eurostars programme and not only the overall economic performance of the beneficiaries and tailoring the requested information per type of beneficiary (i.a. requesting specific information in order to clearly understand if the commercialisation took place or is expected to, requesting specific information about the gross earnings generated by the commercialisation of the project results, not asking universities about turnover generated, inquire about types of investments undertaken); - creating a system of associating unique IDs to each planned product /service /process in order to be possible to identify which of them have been developed and commercialised, which intellectual property rights are in place for each of them (i.a. granted patents); - linking the reimbursement of part of the granted funds to the submission of the projects reports (FiR and MIR) in order to increase the number and completeness of submitted FiRs; - using different IT tools that would allow to restrict the completion of FiR and MIR to one common predefined format for all beneficiaries and to automatically centralise the information declared by all beneficiaries; R2: Improving the selection mechanism in order to financially support projects having the greatest market potential in order to reduce the number of withdrawn projects and increase the number of successful projects in terms of commercialisation of products/processes/services. At project selection stage, maturity and potentiality of the product/service to go to market within two years, according to thematic domain and the market targeted, could be included as a selection criteria or could be judged through a market readiness assessment. R3: Making the financing of projects (or at least a percentage) dependent on introduction to market in order to incentivise beneficiaries to commercialise the project results and to dis-incentivise to withdraw from the project. R4: Increasing the rigorousness of the monitoring mechanism by: - checking the data and information declared by the beneficiaries through various non-declarative methods (i.a. by cross-checking the declared financial data with other sources like Amadeus or financial situations submitted to national authorities, requesting supporting documents that could justify the declared data and information, by performing random audits at least for the qualification criteria); - asking clarifications for the observed outliers; - developing a continuous communication with Eurostars-1 beneficiaries and incentivising them to actively participate in the monitoring and reporting phase; - verifying and updating the Eurostars database as the FiR and MIR are received in order to permanently have included in the database the last information and data available; 73

76 ANNEX 4: THE INTERIM EVALUATION EXPERT GROUP Marcel Shaton Chair (Israel). From 1998 till 2015 Marcel Shaton has been the General Manager of I.S.E.R.D. the Israeli Europe R & D Directorate. In this capacity he was responsible in Israel of the implementation of the Framework Programs, the Eureka and R&D binational European activities. He participated in various EU RTD Committees and was the national NCP Coordinator. As such he led the Israeli Delegation for negotiations on Israeli accession to the Framework Programs. From 1993 to 1998 he was a Minister for Economic Affairs at the Mission of Israel to the European Communities in Brussels. From 1985 to 1993 Marcel Shaton had various positions at the Foreign Trade Administration, Ministry or Industry and Trade, Jerusalem up to Deputy Director of the Foreign Trade Administration. Between 1980 and 1985 he was the Economic Counsellor at the Permanent Mission of Israel to the United Nations in Geneva and IL representative to GATT. Previously he worked as a coordinator for International Economic Affairs at the Ministry of Finance and as a commodity trade analyst at the Trade Governmental Administration, Ministry of Industry and Trade. Mr. Shaton holds a M.A. degree (Cum Laude) in Political Science and a B.A. degree in Political Science and Economics, both from the Hebrew University. He graduated from the Commercial Attaches course of the Ministry of Industry and Trade. Axel Lehmann, (Germany). Axel Lehmann is a Full Professor Emeritus of the Faculty for Informatics at the Universität der Bundeswehr München, Germany, where he held a chair for modelling and simulation until Besides his research activities as faculty member, he is also Executive Director of the research Institute for Intelligent Systems (IT IS) at his university. His major areas of research range from computer-based modelling and simulation, knowledge-based diagnosis and decision support systems, to serious games for education and training. He is former President of the Society for Modeling and Simulation International (SCS), Fellow of the German Informatics Society (GI), and of the Federation of Asian Simulation Societies (ASIASIM). He serves on several editorial boards of journals, e.g. Japan Journal of Industrial and Applied Mathematics (JSIAM), Telecommunication Systems Modeling, Analysis, Design and Management. He was and still is active as expert for various institutions, like for the European Commission (FP6- and FP7-REGPOT, COST), for the Spanish Ministry of Science and Innovation (Severo-Ochoa-Centers of Excellence), or for the Minister of Science, Research and Arts in Baden-Württemberg, Germany (IT innovations). Isella Vicini, (Italy). MSc Computing Science University of Milan, ( ) Researcher at the Robotic Department of CNR (National Research Council) where she coordinated and managed targeted activities of research project in ICT area. She is coauthor of more than 30 scientific papers published in national and international journals and conference proceedings. ( ) Project Manager in Think3 Inc. and responsible for European projects management and administration. (2008-onwards) Warrant Group - European Funding Division Director. Project Manager of HELM Project (FP7); Dissemination Manager of EFEVE Project (FP7); Project Manager and Dissemination Manager of NEWSPEC Project (FP7), Poseidon project (H2020), NANOCATHEDRAL Project (H2020) and Partial PGM project (H2020); involved as project manager consultant in other 18 running projects FP7, Life+, CIP, SME, FTI and H2020. Eva Pando, (Spain). She is currently Managing Director of IDEPA, regional development agency of Principality of Asturias. This organization has the role to improve the competitiveness of Asturias companies through innovation, and internationalization, with the goal of creation new jobs and richness in the region. Previously, for 10 years, she was general manager in a Business Innovation Centre (CEEI ASTURIAS) dedicated to the development of new based technology companies in the Principality of Asturias (Spain). CEEI ASTURIAS depends from IDEPA. Previously she worked in the venture capital market for 7 years. During this time she was dealing with two different tasks: the first as investing director, and the second as investing department Coordinator. She was also member of 13 boards of directors from participated companies during this period. 74

77 Currently she hold the position of vice president of the board of directors of ANCES (Spanish Association of Business Innovation Centers), and she is member of the board of directors of ASBAN, a business angels network. Michele Cincera, Rapporteur (Belgium). Michele Cincera is Professor of Industrial economics at the Solvay Brussels School of Economics and Management Université Libre de Bruxelles. Since 2012, he is the Director of the International Centre of Innovation, Technology and Education Studies (icite). In , he was visiting the EC-JRC-IPTS as a senior scientist. His research interests embrace the quantitative assessment at the micro-level of innovative and entrepreneurial activities, their determinants and socio-economic impacts as well as the analysis of National Innovation Systems and policies supporting science, research and innovation. 75

78 ANNEX 5: MID-TERM EVALUATION EUROSTARS-2 TASKFORCE REPORT Introduction The goal of the present report is the comparison between the initial Eurostars-2 targets and the achievements at mid-term of the programme. The results shall demonstrate the areas, where the targets have been fully or partially achieved and highlight areas, where more effort is needed to fulfill all targets of Eurostars-2 until Further improvements will be presented as well. The report also analyses arguments for a continuation of the programme after The report is based on data from the EUREKA Secretariat and on surveys carried out within the EUREKA High Level Group and actors having applied for Eurostars-2. Eurostars-2 is an initiative of the EUREKA member states and is co-financed by the EU in accordance with article 185 of the Treaty on the Functioning of the European Union. The relevant targets of Eurostars-2 are formulated in the following documents: Budapest Document, endorsed during the EUREKA Ministerial Conference on 22 June 2012 in Budapest. General Implementing Guidelines, for the Eurostars-2 programme endorsed by the High Level Group in Ankara in June Decision No 553/2014/EU of the European Parliament and of the Council of 15 May The targets for Eurostars-2 on political and operational level are summarised in the following table: Political targets of Eurostars-2 Operational targets of Eurostars-2 source target source Target 1,2,3 Stimulate economic growth and job creation by enhancing the competitiveness of R&D performing SME Lean administration by deepening the synchronisation of the national funding programmes Opening to Associated and non EU- Member EUREKA countries 1,2,3 Increased number of applications Balanced and flexible financing by securing funding of the 50 highest rated projects and of the top 50 to 75% of the applications above threshold 7 months time to contract Market introduction 2 years after project completion Enhanced promotion 2 Common programme evaluation including: Impact assessment Synchronisation, communication and management process 2 Introduce: Online submission system Common financial viability method standard Unique progress report Redress procedure 76

79 3 Favorable conditions for investment in knowledge and innovation Greater impact with respect to research and innovation Closer synergies, coordination and no unnecessary duplication Participation of SMEs without previous experience in transnational research Common programme evaluation 3 Low administrative burden, no double audits and reporting Progressive harmonisation of national funding rules Main observations on the programme implementation between 2014 and Achievements 1a Political achievements European policy objectives The HLG survey underlines that Eurostars-2 clearly contributes to the establishment of the following European strategies mentioned in Decision No 553/2014/EU of the European Parliament and of the Council of 15 May 2014: The Europe 2020 strategy emphasises the need to develop favorable conditions for investment in knowledge and innovation so as to achieve smart, sustainable and inclusive growth in the Union. Eurostars-2 clearly contributes to the Europe 2020 strategy in SMEs due to its bottom-up nature, business-driven agenda, focus on market and international dimension also beyond EU. Moreover Eurostars-2 facilitates the participation of SMEs which are more used to national channels and is often the first step towards internationalisation. A Reinforced European Research Area Partnership for Excellence and Growth : by reducing the fragmentation of the European research landscape through networking between Member States and the European Commission and coordinating R&D measures on programme and project level. Participating states and partner countries agree that Eurostars-2 is one of the best examples for an ERA instrument as it brings national and EU policies together. Horizon 2020 aims at achieving a greater impact with respect to research and innovation with its Industrial Leadership pillar, including its financial contribution to Eurostars-2. By strengthening P2P partnerships the Union s contribution to Eurostars- 2 has pushed participating states in their common effort to establish clear and transparent procedures which are manageable for SMEs. As a result of the efficient central evaluation and increased coordination of national programmes a high number of high quality projects can be funded. At the same time Eurostars-2 avoids duplication with Union, international, national and regional research programmes. Funding policy objectives Promotion of research activities: Eurostars-2 is designed to explicitly support transnational collaboration of research- and development performing SMEs among themselves or including other actors of the innovation chain. 71% of the applying organisations are R&D performing SMEs and the most popular configuration is a consortium of 2 R&D performing SMEs. The commercialisation of the developments is continuously monitored during the project lifetime and also until 3 years after project completion. The figures from Eurostars-1 are very convincing: 77

80 899 new products, processes and services reported, 693 of them were brought to market Median time to market for the first commercialisation 10 months Promote and increase the participation of SMEs without previous experience in transnational research According to the participant survey launched in December % of the participating SMEs had no previous experience with international cooperation before applying for Eurostars-2. Additionally 27% of webinar participants in 2016 had no international experience. This demonstrates that this group is showing an increasing interest in international R&D cooperation. 1b Opening Eurostars-2 Three out of the 36 Eurostars-2 countries are associated countries to EUREKA: South Korea, South Africa and Canada. South Africa and Canada joined Eurostars-2 in March 2016 (starting from COD 6 in Eurostars-2). South Korea participates in Eurostars-2 from the beginning. Furthermore, Switzerland as a full member of EUREKA was fully associated to the EU only in 2107 but they participated in Eurostars-2 from the beginning. The associated countries and Switzerland fully participate in Eurostars-2 without financially benefitting from the cooperation with the EU. Full participation means, amongst others, following the Eurostars-2 common eligibility and evaluation rules and respecting the international ranking list. Since these countries are neither EU member states nor associated members states to Horizon 2020, they are not entitled to receive top up from the EU nor can they benefit from the financial EU support in the evaluation process. The countries pay a fee to the EUREKA secretariat to cover the costs of e.g. the evaluation of the project proposals. The mechanism integrating countries, which are not eligible to receive EU top up allows an opening of Eurostars to the world (via the association to EUREKA) and to EUREKA member countries, who are not associated to Horizon2020, like Russia and Liechtenstein. The main reasons for those countries to participate in Eurostars-2 are: Eurostars-2 provides opportunities for national SMEs to work on the best joint projects with the best partners possible Joining Eurostars-2 was a request of the national SMEs in order to be able to cooperate with all EUREKA member states using all EUREKA instruments (EUREKA network projects, Cluster projects and Eurostars projects). Eurostars-2 fits the national policy to support SMEs becoming global players. In order to realise this, engaging in collaborative activities with outstanding overseas companies is required. Since Europe is a high-density region of enterprises with innovation and technological capabilities, it is a good environment for the national SMEs to induce innovation The international cooperation through Eurostars-2 helps the national SMEs to overcome the multiple differences in national technology-related legal systems and technology environments through technology joint development, and to expand together to new markets through open technology innovation. 78

81 2 Operational targets and achievements 1a) A balanced budget from the NFB and the European Commission to secure funding for the top-50 projects of the projects above the threshold. 242 projects out of 250 from Cut-off 1 to Cut-off 5 have secured funding (overall, 92 % to 100 % of the top 50 projects have secured funding) % 98 % 94 % 100 % 100 % Cut-off 1 Cut-off 2 Cut-off 3 Cut-off 4 Cut-off 5 Approved for funding Not funded Withdrawn by participants 1b) A balanced budget from the NFB and the European Commission to secure funding for 50-75% of the projects above the threshold At programme level, the target has been reached with funding secured for 79 % to 100 % of the projects ranked above the threshold as a result of the effort of the participating states and the partner countries to provide enough funding to support the projects. 100% 75% Overall results of the funding of the projects above the threshold 100% 90% 83% 80% 79% 50% 25% 0% Cut-off 1 Cut-off 2 Cut-off 3 Cut-off 4 Cut-off 5 24 One project ranked in top 50 projects from Cut-off 2 was withdrawn by all participants due to not having the necessary financial means to carry out their part of the project. 79

82 2a) Time-to-contract of 7 months or shorter on average by Cut-off Time-to-contract is the time elapsed between the Cut-off date and the date of notification of the grant decision from the NFB to the participant, or the date of signature of the grant agreement. Clear improvements of Time-to-contract can be observed in comparison to the first Eurostars Programme ( ) but also over the Eurostars-2 Cut-offs. The average Time-to-contract in Eurostars-2 has decreased from 8,9 months in Cut-off 1 to 7,1 months in Cut-off 4 reaching the target of 7 months or shorter as set in Eurostars-2 Bilateral Agreement. Some delays can be observed for the Cut-off 1 as NFBs were advised not to conclude any grant decision/ agreement during the summer 2014, due to clarifications required for the eligibility of the Union financial contribution. Other key structural reasons for delay for the fist Cut-offs: for 4 NFBs, the Bilateral Agreement needed to be signed before any grant decision could be finalised and 2 NFBs underwent a change in their national RDI framework Average Time to contract in months Average Eurostars 1 ( ) Cut-off 1 Cut-off 2 Cut-off 3 Cut-off 4 This aim could be reached among others because countries have accepted the English Eurostars-2 application form in their national applications and others aligned their national procedure to reach time- to-contract of 7 months 2b) High degree of national synchronisation The aim of the national synchronisation is to allow the consortia to start with the project as soon as possible and at the same time. By shortening the time to contract the national synchronisation is fulfilled to a high degree as well, as for all participants in one project the contracts are more or less signed at the same time. Thus a common project start is enabled. 3a) A common financial viability methodology defined, based on a proposal accepted by the National Implementation Group The financial viability is defined by the capacity to finance the project development, taking into account the potential access to public funds. The financial viability assessment is not an eligibility criterion, therefore negative assessments do not result in the ineligibility of a project. The verification of the financial capacity to carry out the proposed project is an integral part of the evaluation stage and must be completed before the Independent Evaluation Panel meeting. The idea behind this methodology is to safeguard the public money not only in the country where there are doubts on financial 80

83 capacity of one specific partner, but on the rest of the countries participating in the project. A common financial viability methodology has been defined and presented for a discussion to the NIG in June It was agreed to the general principle of the financial viability methodology. The proposal will be further finalised with the comments received and the final financial viability methodology proposal will be presented to the NFBs represented at the NIG during the next meeting in June b) A common reporting system accepted, based on a proposal accepted by the National Implementation Group and taking into account national requirements A Standard and unique common progress report in English was accepted by the NIG in June The report aims to collect information on the level of each participant about technical progress, the quality of cooperation in the project, deviations from the original project plan and any other unforeseen changes in the implementation of the plan. The new IT system is being developed for Eurostars-2, to ensure participants and NFBs use an efficient online reporting mechanism ensuring that each participant s data remains available to the participants and their NFB. After the central implementation of the common progress report in 2017, the ESE will monitor and support the implementation on the national levels. 3a HLR survey and SWOT Analysis In HLR survey a SWOT analysis on Eurostars-2 was included. The main results are summarised in the graph below. The large majority of the participating states and member countries are satisfied with the programme. Eurostars-2 is regarded to be a very relevant programme. Often it is the only SME funding programme or compared to other national funding programmes the biggest one with respect to the funding. 81

84 The results underline the achievements mentioned above. Eurostars-2 has a highly recognised European added value. With regard to the EU objectives Eurostars-2 is consistent with them as it supports cross border operations of SMEs, decreases the fragmentation of the ERA and is the best example for an ERA as it brings national and EU policies together. By addressing intermediate TRL levels (4-6) Eurostars-2 is complementary to EU programmes, especially the SME instrument. The possibility for the project participants to cooperate internationally and with partners beyond the EU allows access to international markets. Specifically for the SMEs it is often the first step towards internationalisation. The HLR have a strong interest to continue the joint programme. The strong and long lasting commitment from participating states and partner countries and the allocation of dedicated national budgets is a strength compared to other programmes. The co-funding of the Commission is an incentive to agree on the central procedures and accept the ranking list. Additionally it helps to make progress towards a synchronised/harmonised programme. Also the leverage effect of a joint programme should not be underestimated. The collaboration of participating states, partner countries and the Commission enables us to support more SMEs than we can separately. In addition the Eurostars-2 label raises the visibility of the funded SMEs. However room for improvement is given e.g. with regard to the budget. In the beginning of Eurostars-2 all participating states have committed themselves to participate in the programme and to provide a certain budget. Due to several reasons like the economic crisis some countries unfortunately cannot live up to their original financial commitments. This sometimes causes problems in other countries, as they cannot spend their budget. Eurostars-2 is not everywhere well known and some countries would like to receive more applications with a higher quality. Compared to Eurostars 1 the administrative burden is even higher under H2020. Specifically the requirements regarding audits, selfassessment and ethics are considered to be too high. 82

85 3b Participant survey With the participants survey no complete SWOT analysis was performed but it was asked for the main strengths and weaknesses (see graph on the next page). The participants often are of the same opinion as the HLRs. Eurostars-2 fits to their needs, is being regarded to be easy, fast and compared to other programmes the administrative burden is much lower. They appreciate the very good support at national level and the transparency of the programme. The possibility to cooperate internationally is a big advantage for them. The lack of budget in some countries leads to not funded projects although they are above the threshold. Nevertheless Eurostars-2 has a high success rate compared to other programmes. Again room for improvement is still given and the participants request a better alignment between the central and national processes. In conclusion also the participants think that Eurostars-2 is a great programme for the support of R&D in SMEs, more than two third of them will or would apply again and they would welcome the continuation of Eurostars. Strengths Small projects, i.e. small number of partners (at least two) Public funding rate Straightforward application process Bottom-up principle, with no thematic restriction Transparent, central evaluation with the feedback received National contact point (support received in national language, national funding contract/procedure, ) Possibility to cooperate internationally with a large number of members states, even outside European Union/Europe Fast evaluation and grant agreement signature Two Cut-off days per year Comprehensive feedback on the evaluation Success rate Weaknesses Dualism: Central evaluation and monitoring / National contact point (national funding rules, reports, ) Process of selecting funded projects (e.g. some projects on the ranking list are not funded, because at least in one country the national budget is exhausted) Public funding rate (e.g. different rates in different countries) Success rate is too low 83

86 4) IMPROVEMENTS FOR EUROSTARS The overall performance of the Eurostars-2 programme in the period is considered satisfactory, but some specific improvements to increase the scope of its results are going to be implemented from now to a) Balanced participation of all countries In terms of number of proposals Some countries receive very few applications, making it difficult for them to fund as many projects as expected, and globally the programme was designed to receiving more applications. The implementation of specific national roadshows, with the support of the EUREKA Secretariat and with the cooperation of other countries more experienced in the programme, and the general dissemination of the specificities of Eurostars-2 will contribute to increase the number of applications. In addition, cooperation between national and regional authorities will lead to a more effective knowledge of the programme by the SMEs. With a targeted approach even more SMEs without previous international experience could become involved in Eurostars-2 projects. In terms of quality of the proposals Certain countries receive a good number of applications, but the quality is not that high and as a result the ratio of approved projects is low, hampering the allocation of committed funds. As a whole, Eurostars-2 is a competitive programme aimed at financing the best close to market projects performed in international cooperation and led by R & D performing SMEs. The training of the Eurostars National Project Coordinators and Project Officers is crucial, so more specific training sessions about how proposal are evaluated will be organised by the EUREKA Secretariat. Knowing more about the evaluation can give inference how a good application should look like. The assistance to the eligibility sessions is also an important possibility to gain experience on the same topic. Best Practices will be shared by visits of experienced officers to other countries upon request, and the review of draft proposals will be done in cooperation between the officers of the different countries involved in the consortium. Additionally National Offices will organise workshops for participants focused on how to write competitive applications, and also will provide support services to the participants in the preparation of proposals by reviewing draft applications prior to their submission. This will be supported by experienced offices. 4b) Increased commitment of some countries Given that only guaranteed-funded projects are approved in the Eurostars-2 programme, and this is done with national funds, there is a correlation between the commitment of each country and the number of projects approved. In a scenario of global financial crisis, some countries cannot live up to their original financial commitments, which reduces their number of applications approved, and also causes problems in other countries having sufficient funding available. A detailed analysis will be made to determine the reasons of the reduced commitment in those countries, and bilateral discussions will take place in order to determine what support can be provided by the network in order to raise the funding and explore the availability of national and regional funding programmes. 4c) Better promotion of the programme The visibility of Eurostars-2 programme and the brand Eurostars will be raised to facilitate the previous goals: 84

87 A specific study about how to reach the targeted clients of the programme, the R&D performing SMEs, will be conducted in order to review the current communication strategy and tools Every year the Chairmanship of EUREKA will organise an international Eurostars-2 event directed to promote the participation of all participating states and partner countries in the Eurostars-2 programme The cooperation in communication activities with the European Enterprise Network and the European Commission will be enhanced, given their key role as promoters of the innovation and internationalisation between the SMEs The protection of the current figurative trademark and the word trademark will be extended for another ten years An improved homepage will be in place and the presence in social media will be increased. The webinars organised before each Cut-off date, which are already well accepted, will be more widely promoted The visibility of National Contact Points will be increased These measures will be applied to all the countries participating in the programme with special attention to those that need an increase in the quantity or the quality of the proposals. 4d) Decreased administrative burden The Eurostars-2 programme is characterised by being centrally evaluated and financed in a decentralised manner. Currently the administrative burden in the ESE and the participating states and partner countries is too high. We ll strive for a leaner internal administration by improving the current methodologies of ethics assessment, internal audits and measures while preserving the EU financing and regulations. Duplicate requests for information to participants at national and international levels will be avoided by the implementation of the common reporting system. Again, learning from Best Practice developed by the members of the EUREKA Network is a great opportunity to improve national procedures and rules and make them more innovative and appropriate for SMEs. 4f) Impact Assessment According to the General Implementing Guidelines, for the Eurostars-2 programme endorsed by the High Level Group in Ankara in June 2013 a mid-term as well as an expost evaluation shall be carried out in close cooperation between the Commission and the Member States. Furthermore an objective impact evaluation of the Eurostars programme, based on both internal (collected by the ESE) and external sources of information and data should be carried out. The EUREKA network should discuss with the Commission further details and the timing of such an impact assessment. Ideally, the impact assessment should support the discussions and preparation of Eurostars beyond ) Conclusions EUREKA strongly believes in the necessity and opportunities of a joint programme based on the well-established and reliable model of Eurostars-2. Based on the interest, commitment and demand of the SMEs, Member States and European Union the excellent and strong programme Eurostars should be continued beyond 2020 and we should start with the discussions now. The programme will be an important instrument of an overall innovation chain within the European Research and Innovation Area and national innovation policies to support SMEs in developing innovative and competitive products that will impact job creation and economic development. Eurostars will still have a unique position in the funding landscape in the future. 85

88 Eurostars stimulates and finances R&D activities in SMEs in an international environment. Thus Eurostars contributes to and strengthens the Industrial Leadership Pillar of H2020. With the co-funding of the Commission, the participating states and partner countries Eurostars is one of the best examples of the ERA. To continue with the joint programme will create a win-win situation for the participating states, partner countries and the Commission as well. Together we can effectuate more than each of the parties alone. 86

89 ANNEX 6: PUBLIC CONSULTATION QUESTIONS Section D - EUROSTARS2 Introduction to Eurostars2 Eurostars supports international innovative projects led by research and developmentperforming small- and medium-sized enterprises (R&D-performing SMEs). With its bottom-up approach, Eurostars supports the development of rapidly marketable innovative products, processes and services that help improve the daily lives of people around the world. Eurostars has been carefully developed to meet the specific needs of SMEs. It is an ideal first step in international cooperation, enabling small businesses to combine and share expertise and benefit from working beyond national borders. Eurostars is a joint programme between EUREKA and the European Union, co-funded from the national budgets of 36 Eurostars Participating States and Partner Countries and by the European Union through Horizon In the period it has a total public budget of 1.14 billion. Additional information can be found at D.1: What is your level of familiarity with the Eurostars2 Joint Programme? Very good Good Fair Low D.2: Have you participated in an action under Eurostars-1 and/or Eurostars-2 (several answers are possible)? Yes, in funded project of Eurostars1 Yes, in non-selected project of Eurostars1 Yes, in funded project of Eurostars2 Yes, in non-selected project of Eurostars2 No D.3: If you are not involved in a Eurostars project, how did you find out information about the Joint Programme? In a conference At a scientific workshop or training event Through media (Internet, national information channels, newspapers, specialised press, etc.) Through national networks (NPS, NCPs, EEN, KAM, Regional authorities, national or regional Innovation Agencies, national or regional Chambers of Commerce, etc.) Other Please specify (maximum 100 characters, mandatory for "Other"): 87

90 Objectives C.4: To which extent is the Eurostars programme likely to achieve the following objectives? Promote research activities that are carried out by transnational collaboration of research- and development performing SMEs among themselves or including other actors of the innovation chain (e.g. universities, research organisations) Promote research activities where results are to be introduced into the market within two years of the completion of an activity Increase the accessibility, efficiency and efficacy of public funding for SMEs in Europe by aligning, harmonising and synchronising the national funding mechanisms of Participating States Promote and increase the participation of SMEs without previous experience in transnational research Fully To a large extent To a small extent Not No opinion Comments (maximum 600 characters) D.5: A major objective of the Joint Programme is to introduce the results of projects into the market within 2 years of the completion of the project. Does the present design of the Eurostars Joint Programme sufficiently support such a target, do you see any possibilities to improve this? Fully To a large extent To a small extent No No opinion Comments (maximum 600 characters) D.6: In the absence of a Eurostars-2 grant, would R&D performing SMEs have undertaken their projects by their proper or other means? (maximum 600 characters) Yes To a large extent To a small extent No No opinion Comments (maximum 600 characters) D.7: Is there sufficient budget from Participating States to achieve the objectives of the Eurostars2 Programme? 88

91 Sufficient budgets from all Participating States Sufficient budgets from some Participating States Insufficient budgets from some Participating States Insufficient budgets from all Participating States No opinion Comments (maximum 600 characters) Relevance D.8: Does the Eurostars-2 Joint Programme in its design and implementation contribute to the general objectives of making Horizon 2020 more oriented towards innovation and economic impact and support the holistic approach to innovation taken by Horizon 2020? Yes To a large extent To a small extent No No opinion Comments (maximum 600 characters) D.9: Is the design of the Eurostars-2 Joint Programme (minimum 2 participants from 2 different Eurostars-2 participating States, R&D performing SME as leading partner in the consortia, 3 years project duration, project results to be introduced into the market after to 2 years of the project completion, etc.) an adequate response to the observations on SME innovation support in FP7 and H2020? Yes To a large extent To a small extent No No opinion Comments (maximum 600 characters) Coherence D.10: Does the Eurostars-2 Joint Programme complement other interventions / instruments from Horizon 2020 (SME Instrument, Fast Track to Innovation, Collaborative projects, Marie Skłodowska-Curie actions) or from other EU programmes (COSME) and realise synergies where possible? Yes To a large extent To a small extent No No opinion Comments (maximum 600 characters) D.11: Are the resources mobilized by the Participating States and the European Union justified by the scale and scope of the initiative? Yes To a large extent To a small extent No 89

92 No opinion Comments (maximum 600 characters) D.12: How do you assess the efficiency of the mechanisms and tools ensuring the entryinto-the-market of results/achievements of Eurostars-2 ended projects? Very efficient Efficient Partially efficient Not efficient No opinion Comments (maximum 600 characters) Effectiveness D.13: In your opinion is the Programme sufficiently accessible in particular for R&D performing SMEs? Yes To a large extent To a small extent No No opinion Comments (maximum 600 characters) D.14: What is the benefit for an R&D-performing SME to participate in Eurostars2 projects? (maximum 600 characters) European Added Value D.15: Do you think that the total amount of EU financial contribution (i.e. max 287 million EUR) is appropriate in order to achieve the objectives of the Eurostars2 Programme? Too high Adequate Too small No opinion Comments (maximum 600 characters) D.16: What is in your opinion the additional value resulting from the EU intervention in the Programme compared to what could be achieved at national or regional level? (maximum 600 characters) D.17: Is the design and performance of Eurostars2 in line with the spirit of Art.185 TFEU and with the requirements of Art.26 of Horizon 2020, in particular concerning financial, managerial and scientific integration? Yes To a large extent To a small extent 90

93 No No opinion Comments (maximum 600 characters) Strengths, weaknesses and the future D.18: In your opinion what are the strengths, weaknesses, opportunities and threats regarding the management of Eurostars2 Joint Programme? (maximum 600 characters) D.19: What are the lessons learnt for the future? (maximum 600 characters) D.20: Would you be in favour of a future Eurostars initiative? Yes, as a joint programmes with the participation of both Participating States and the EU Yes, as a joint programmes, but only with the Participating States No, I would prefer community support in the context of a future Framework Programme No, only national programmes are relevant in this domain No opinion Other Comments (maximum 600 characters, mandatory for "Other") D.21: Do you have any further comments (maximum 1200 characters)? 91

94 ANNEX 7: RESULTS OF THE PUBLIC CONSULTATION OF EUROSTARS-2 PROGRAMME This section presents an overview of the outcomes of the Public Consultation related to the Evaluation of the Eurostars-2 Art.185 initiative. The online survey was open for submission for more than 13 weeks (27 January 30 April 2017). 1. Profile of the respondents to the public consultation As shown in Figure.1, about two thirds of the 93 respondents belong to a single institution or company and one respondent is an individual. Figure.1. Capacity of respondents As an individual On behalf of a single institution/company On behalf of an umbrella organisation of EU interest Figure.2. shows that 58% of respondents are business organisations (among which two thirds SMEs), 22% public authorities and 15% universities. Figure 2. SMEs In terms of geographic coverage, Figure 3 indicates that nearly 50% of respondents are originating from 5 countries: Belgium (16.1%), Norway (9.7%), Spain, Greece (7.5%) and Italy (6.5%). This geographical snapshot is of importance once related to the total Eurostars-2 contribution to Research & Development and innovation activities in Europe and partner countries across participating countries. Germany, The Netherlands, France and Switzerland, identified amongst the Top 5 countries in this matter, have showed limited interest in the consultation. The attractiveness in responding to such survey from SMEs is strong and the ratio Companies- Institutions is quite balanced. Figure 3 adds an interesting aspect of the disproportion of interests in responding to such public consultation, more importantly in Partner Countries and Moderate Innovator 92