This section of the report summarizes the main findings. The summary is organised along the subthemes target group and scope of the programme, governance, management and operations, funding, and outcomes and impacts of Eurostars.
Target group and scope of the programme
According to the participating SMEs and the governing and administrative bodies of Eurostars, the programme is relevant to the needs and objectives of the target group. The high level of demand from SMEs is considered as an indication of this “strategic fit”. Almost all participating SMEs plan to apply to Eurostars again, as the programme corresponds to their needs and objectives. Research performing SMEs are the overwhelming majority of participants, and
received about 75% of the money spent by Eurostars. In addition, the programme was successful in addressing young and small and micro firms. Participating SMEs already have experience in national and international
funding programmes, and the majority already have pre-existing international links. Most probably, it is because of this experience that the programme documents are found easy to understand by SMEs, and project proposals are prepared without almost any external assistance.
However, it is found that applications which involve university and/or PROs have a higher chance of being funded. This may be because the presence of a university or public research organisation signals the high quality of the proposal, or that those organisations are more experienced in preparing application documents, or that those applications provided the best balance between scientific excellence and market knowledge.
Based on its bottom-up nature, Eurostars is able to address a wide-variety of innovative technologies, ranging from ICT and its applications, and medical biotechnology, to (renewable) energy and steering and control technologies. Governance
In general, ESE is considered a successful Dedicated Implementation Structure. However there is a need to improve managerial and administrative practices,
November 2014 Page 102 of 139 enhance the qualifications of staff, develop and implement a marketing and promotion strategy, and develop and maintain a well-structured information management system and database for the use of ESE and the other governing and administrative bodies of Eurostars.
The governance system of Eurostars functions properly and the interaction between the national and central administrative bodies is effective. However, the international dimension of Eurostars required a complex governance system to be set up.
In HLG meetings, various issues critical to the success of Eurostars are regularly discussed without reaching either a consensus for action or rules binding for all countries. Synchronisation and harmonisation of funding, eligibility rules and the adequacy of national budgets were intensely analysed and discussed. However, a consensus was reached on the number of cut-offs per year and improved information flows between the ESE and NFBs.
There is a need to increase the ownership and involvement of HLG towards Eurostars. This is seen as critical in order to facilitate synchronisation and harmonisation. Similarly, NFBs’ involvement remains limited in the governance structure; they should be more intensely involved to achieve a better synchronisation and harmonisation.
Eurostars strengthened the SME component within EUREKA and contributed to
a much higher visibility of the contribution of SMEs to European competitiveness and innovation. In this way, Eurostars had a significant and long-lasting positive effect on the EUREKA network.
Management and Operations
The virtual common pot worked well and is the preferred mechanism, since it attracts countries to contribute to Eurostars. Based on the success of the programme, several countries increased their shares significantly beyond the initially agreed level. However, the full benefits of the virtual common pot have not yet been exploited in Eurostars, since some of the countries, in particular Germany (despite an enormous increase of funds while the programme was running) and Spain, do not allocate a sufficient level of funding for the programme. Because of this, some of the good projects selected drop out of the ranking list, which in turn leads to frustration among applicant SMEs. Synchronisation of funding is still the most important challenge facing
Eurostars, although some improvements have been achieved. In some cases, participating SMEs face great difficulties because of the longer time-to- contract. There are also parallel application and evaluation requirements as well as dual reporting obligations applied by some of the NFBs.
The other major challenge of Eurostars is the harmonisation of funding rules. The representatives of HLG think that this challenge should be addressed through a top-down process with an initiative by national ministries and the EU. The NPCs think that full harmonisation is not possible (for example they cannot change the percentage of funding and the upper limit for finance per SME). Another issue is the need for a better harmonisation of subcontracting rules for universities and PROs.
November 2014 Page 103 of 139 The process of central evaluation in Eurostars worked well. However, there is a
need to improve the transparency and feedback mechanism and to streamline the process in order to decrease the time from application to the availability of IEP evaluation.
The time-to-contract, e.g. the time span from application to the signature of the funding contract between participants and NBFs, was rather long at the start of the programme, but went down considerably during the first phase of Eurostars. However, the time-to-contract is fairly long in a significant number of countries. Improvement of the time-to-contract for the programme is urgently needed, as the existing large difference between participating countries calls the whole system into question.
Although the number of technical experts as well as those with market experience has increased considerably since 2012, there is still room to improve geographical and gender balance. Also, the NPCs would like to be involved in the process of improving the experts’ database.
SMEs are satisfied with the level of information received about the evaluation of their projects. The reporting workload and the monitoring process in Eurostars are found appropriate by them. They also find the evaluation process clear, transparent and timely.
Funding
The auditing is considered by NPCs as well as NBFs as harder in Eurostars compared to other EU programmes. NBF auditing within the framework of Eurostars should be abolished and substituted by the use of reports by national independent audit offices as they already audit the NFBs.
One of the indicators of the success of Eurostars was the significant increase in the number of applications and hence the increasing demand for public funding by this programme. Hence the share of submitted applications funded declined throughout the lifetime of Eurostars. Similarly, the share of budget demanded by approved projects in relation to the budget demanded by submitted projects went down from 42% in cut-off 1 to 18% in cut-off 10. The relation between the number participants (members of consortia of approved projects) and the number of applicants in submitting consortia moved, in a similar fashion, from 42% to 17%.
The increase of the number of applications is especially remarkable when the number of applications per cut-off is mapped onto the number of applications per year. As national budgets is allocated per year, then cut-off 1 correspond to 2008, cut-off 2 to 2009, cut-off 3 and 4 to 2010, cut-off 5 and 6 to 2011, cut- off 7 and 8 to 2012, and cut-off 9 and 10 to 2013. This resulted in 90 applications for 2008, 90 for 2009, 149 for 2010, 145 for 2011, 139 for 2012 and 179 for 2013. This is an impressive increase in the number of applications per year.
As a consequence, the share of projects meeting the quality threshold which were approved for funding declined from 68% for cut-off 1 to 55% for cut-off 10; i.e., an increasing share of high-quality projects were not funded. However,
November 2014 Page 104 of 139
in terms of budget the slowdown between cut-off 1 and cut-off 10 was less dramatic, going down from 62% to 54%.
The size of the national budgets for Eurostars still differs enormously. This refers to both the relative size of the country and the development stage of their national public funding system, and the demand from national SMEs for Eurostars funding. This difference had a significant negative impact on the functioning of the virtual common pot system.
The size of funding per participant is judged as adequate by most participants and programme administrators.
The financial and economic crisis in Europe had a negative effect on national funding for Eurostars in the majority of countries. The number of projects above threshold decreased considerably due to lack of funds, and for this reason some countries did not participate in the last calls of Eurostars. The countries which did not have shortage of funds have also reported that they were affected negatively by the crisis, since the SMEs they support partnered with their counterparts in countries having funding difficulties.
Outcomes and impacts of Eurostars
The EU funding has a leverage effect on Eurostars in a way that stimulated national policy-makers to have clear goals and allocate enough funds for the programme. It also created a strong marketing effect towards potential beneficiaries by increasing the prestige and attractiveness of the programme to SMEs.
Eurostars stimulated new partnerships of young and small SMEs between participating countries. It is foreseen that nearly all consortia will result in intensive R&D links between firms that will survive in the future without public funding. The results pointed to some long-lasting impacts of the programme, as about 90% of the firms intended to continue the initiated partnerships with other R&D-performing SMEs in Europe whether the consortium was funded or not.
Eurostars contributed to employment generation by R&D-performing SMEs in Europe. The causal impact of Eurostars funding on employment growth was calculated using different regression designs analysing the growth-rate differentials between funded and similar but non-funded firms. This difference- in-difference estimation is a rigorous way to test for causal programme effects between funded and similar non-funded applicants to the same programme. These results are fairly robust against changes in the underlying assumptions
of the evaluation model. The econometric evaluation approach used was tested for heterogeneity of the size of treatment effect with respect to the size of funded SMEs measured in terms of employment, the composition of consortia, the rank of an application in the ex-ante evaluation by technical experts, the size of the Eurostars grant, or whether the SMEs applied and / or received funding for more than one Eurostars project. In addition, the matching approach implies that the employment growth rate of funded firms was nearly twice the employment growth of observably equal non-funded firms.
November 2014 Page 105 of 139 The identified differentials in growth rates are even more striking as there were
hints that the programme still has not reached its full effect, as projects need more time to develop.
Eurostars also had a significant positive impact on the technological potential of funded firms. The estimated number of patent applications by funded firms exceeded the number of patent application of non-funded programme applicants.
November 2014 Page 106 of 139
7 Assessment of findings
Eurostars is a unique programme to support bottom-up, near-market-research of SMEs in participating countries. Overall, the results of this evaluation provide evidence for the success of the programme and the worthwhile nature of the public money from the EU and the participating countries. The programme contributed to employment generation, improved the competitive position of participants in the market place and contributed to the development and roll-out of new and improved products and services. There is room for improvement in the implementation of the programme so that the positive overall impacts of the programme will become even larger in the future.
This chapter assesses the findings of this review under the same subthemes as in Chapter 6.6, Summary of findings: Target group and scope of the programme, Governance, Management and operation, Funding, Outcomes and impacts of the programme.