EL GATEKEEPER

7.1 Conclusions

We present the conclusions of this evaluation of the NFP (2002-2010) in line with the evaluation questions that are linked to the main evaluation criteria: relevance, effectiveness, impact and efficiency. We also present options for the future.

Relevance

The core question relating to the relevance of the NFP is whether there is still a need for the external support the NFP is providing in order to alleviate ‘quantitative and qualitative shortages of skilled manpower at mid-career level, in the context of capacity building within the framework of poverty reduction in developing countries’.

The conclusion regarding relevance is that from an individual employee perspective as well as from an employers perspective the NFP education and training are (still) seen as relevant for increasing knowledge, skills and competences as well as career opportunities of individual employees, and in the view of employers and fellows are also expected to contribute to the quality of the services of the organisation and innovation of the organisation. The relevance of the NFP was underpinned with concrete information from the country studies, although in most cases the shortages could not be documented.

It is noted that the perceived relevance of the NFP programmes as expressed by the employers should be considered with care. First of all, because a relatively large number of employers (about one third) were a NFP fellow in the past, possibly having a positive bias towards the NFP.

Furthermore, the country studies indicated that if given the choice fellowships would not always be the first instrument selected for capacity building.

The relevance of the NFP is also clear in the context of local (higher) education systems which to a large degree does not allow for appropriate responses to demands of the labour market, leading to structural mismatches between the education system (qualifications taught) and the labour market (skills and knowledge requested by employers. Not surprisingly, the NFP is especially highly valued by alumni and employers because of its combination of theoretical and practical content and the high quality of its courses. From this it is also concluded that the assumed causal relations which are part of the intervention logic seem to hold. In practice, however, there might obstacles and hindering factors limiting the perceived effectiveness of individual education and training in the organisation, which in general are not given serious consideration in the implementation of the programme.

It is also concluded that the NFP can be of relevance for the Dutch HE institutes for its potential to generate knowledge products (research output, publications, PhDs, new educational programmes, etc.), although additional information would be needed to underpin this claim. The same applies for the Netherlands as a state which might potentially benefit through improved bilateral relation from the large network of alumni that followed a course in the Netherlands with NFP funding.

Finally, also earlier evaluations of other fellowship programmes indicate empirical evidence that individual fellowship programmes like the NFP can benefits to individual fellows in terms of increased competences and improved career opportunities. Also, there seems to be a potential for

106 Evaluation of the Netherlands Fellowship Programme (NFP) 2002 - 2010

stimulating organisational development provided that the organisation is open to new skills and new knowledge. On the national level, however, also based on earlier evaluations, there is little formal evidence of the impact of training programmes on poverty alleviation or improvements in sectors and the economy or society.

Effectiveness

Outputs

During the period 2002 and 2010 the NFP succeeded in establishing about 12.750 fellowships for mid-career professionals. The main focus of the NFP has been on Short Courses (58% of the NFP fellows) and Master Courses (38% of the NFP fellows). With 47% of the participants being female, the programme target of 50% female participants was almost met. There are, however, large differences in the share of female fellows per country, obviously reflecting different social positions of women in these countries.

Although about 60 countries have been eligible for the NFP programme, the participation in the NFP show a skewed distribution towards a limited number of (mainly low income) countries with almost 50 % of the grants being awarded to participants from eight countries, with Ethiopia (1.147 fellows) and Tanzania (1020 fellows) being the largest beneficiary countries.

Although the total number of NFP fellows is quite substantial also in comparison with other

fellowship programmes of other donors, it seems obvious that even with a concentration on a small number of countries the number of fellows is limited at national level. This sets clear limits to the ambitions of capacity building and alleviating skills shortages.

Outcomes

At individual level the NFP fellows clearly benefitted from their participation in NFP. Benefits were reported regarding increased personal and professional competences, career perspectives, increased responsibility and network opportunities. About 70% of the alumni reported a higher job positions and/ or increased responsibility after they obtained a higher degree of education.

Alumni are of the opinion that they to a large extent have been able to put (part of) their new expertise and skills to work. Findings from the field visits confirm that organisations have benefitted from the improved technical skills of the alumni and the exchange effect on colleagues. This provided opportunities for innovation and improved working methods and quality of services and products. Nevertheless the examples provided show that the possibilities to utilise the knowledge and skills gained vary greatly across organisations and are influenced by factors and conditions such as the extent to which organisations and superiors are receptive to new ideas, the position of the alumni upon return and the availability of resources.

At organisational level the benefits noticeably have led to better qualified staff for the employers. This created the conditions for, and according to the employers actually improved the quality of products and services of the organisation. It should be noted however, that these effects are difficult to measure. The effects on the organisation were reported subjectively by the employers, of which a considerable number themselves participated in the NFP, which might have influenced their opinion and view on the effectiveness of the NFP.

Furthermore, various serious obstacles were (subjectively) reported by employers and alumni, hindering an optimal use of learned skills and knowledge at the level of organisations. Organisational barriers were found to be lower in organisations where more fellowships were awarded. This could indicate more “readiness” within the organisation or be the result of a larger

mass of employees having been trained. Another factor influencing organisational performance is the integration of the training or education of staff in a staff development plan of the organisation.

The number of alumni that do not return to their employers or leave the country is relatively low but differs per country. Notwithstanding the fact that most alumni do return to their employer after completing the NFP education, there is evidence of occasional considerable brain drain at the level of the organisations. The loss of trained employees to other institutions can limit the momentum for institutional development while possibly being a brain gain for other organisations.

The overall conclusion is that there is more evidence for positive effects of the NFP at the individual level than at the organisational level.

Impact

There is little formal evidence of contribution of the NFP to socio-economic developments in the regions or countries the fellows reside. Nevertheless, some anecdotal evidence was collected during the country case studies on wider impact.

Based on the long NFP history it is plausible that the programme has benefitted bilateral relations of the Netherlands with the NFP countries. This was also indicated in interviews with policy officers of the Dutch of Ministry of Foreign Affairs. The country cases showed that alumni information is not actively used to develop relations between the Netherlands and the NFP countries. Especially the latter can be seen as a missed opportunity of the Programme to contribute to positive public relations for the Netherlands. The limited budget and the lack of a clear strategy for alumni activities is not supportive to a sustainable alumni network.

Efficiency

The total budget for the NFP programme during the evaluation period 2002-2010 amounted to 244 million euro, which is on average approximately 31 million euro per year, excluding the management and implementation costs which for the implementation period amounted to 12.5 million, which is about 5% of the programme costs.

The conclusion regarding efficiency is that, contrary to the policy of the Ministry of Foreign Affairs, tuition fees have increased considerably in the period 2002 – 2010. This increase in tuition fees, an important part of the NFP grants, is (mainly) due to policies of the Dutch Ministry of Education regarding higher tuition fees for non-EER students, so including the NFP fellows, and the possibility given to the education institutes to charge cost-effective tuition fees for NFP fellows. Nevertheless, the NFP tuition fees are still below the ISPAC tuition fee level, which is the fee Dutch HE institutes charge for foreign (non-EER) students. From 2010 onwards tuition fees were settled for a period of four years, in the mean time only allowing price indexation (inflation). The effectiveness of this measurement is outside the scope of this evaluation.

It is also concluded that alternative supply structures could reduce the costs of the programme, and should be seriously considered. Options for cost reduction which could be considered are co- financing, using regional institutes and allowing students to enrol in foreign/regional fellowships. Also the possibilities of e-learning could be more explored, including the reasons why the available options on the NFP course list have not or have only very limited been used so far. In general the NFP could use good practices related to a more cost-efficient supply of education and training.

108 Evaluation of the Netherlands Fellowship Programme (NFP) 2002 - 2010

7.2 Options for the future

The current NFP is torn between various policy objectives related to development cooperation, foreign policy and Dutch education that hinder the effectiveness and efficiency of the programme. This is especially the case because the intervention logic shows flaws. Also the changing international context asks for changes in intervention strategies. While in the Netherlands it is discussed whether Dutch students should receive any study grants at all, a financial contribution of NFP fellows to their education or training could be considered.

On the basis of this evaluation clear policy choices should be made regarding the future of the programme in relation to the main objectives, the number of countries, types and location of courses, and funding options. Of course different combinations are possible.

 Main policy objective: this choice relates to the relative emphasis on the various objectives. The ministry should decide on the importance of the various policy objectives related to the

improvement of bilateral relations (foreign policy), capacity building (development cooperation) and the internationalisation of Dutch education and to what extent these objectives can be combined in a single programme. This choice very much influences the three other choices.  Number of countries: this choice relates to the number (concentration vs. enlargement) and type

of countries (low income vs. middle income) and who to focus on in these countries.

 Types and location of courses: this choice relates to the course supply (restricted or almost no restrictions) and location where the courses are provided (the Netherlands, in partner countries or third countries).

 Management and cost reduction: this choice relates to the way the programme is managed and the attention given to economy and cost reduction.

The evaluation team has identified three ‘extreme’ options:

1 Continuation of the programme with an emphasis on improvement of bilateral relations and internationalisation of Dutch higher education

This option takes the Dutch bilateral relations and the internationalisation of higher education in the Netherlands as its point of departure. To a large extent the NFP would be used as a strategic instrument to improve bilateral relations (economic and public diplomacy) in the long run. At the same time, the programme would put a greater emphasis on the interests of the Netherlands education institutes

The focus of the programme will be on capacity building on individual level, but there would not be an explicit focus on capacity building of organisations. Specific organisations which are of interest to the Netherlands (Foreign Policy) could be approached to propose staff for the programme.

The consequence of this would be that the NFP would remain a relatively open programme with the aim of reaching a large variety of potential candidates from countries that are of interest to the Netherlands. It could even be decided to increase the number of countries, for instance also involving upcoming market countries. Also a larger focus of the course supply to areas which are of specific interest to the Netherlands business community, such as water management could be considered.

Whether such a programme should be completely funded from the development cooperation budget is an issue. Considering the main policy objectives (to promote political and economic interests and internationalisation of Dutch education) it would not be logical to finance the programme from Development Cooperation funds . l. Also the funding policy should be revised,

because next to grants also loans could be considered. If the programme is to serve also the interests of Dutch HE institutes, it should be considered to look for synergies in policy and budgets between the NFP and the internationalisation policy of the Dutch ministry of Education. In this option the embassies should play an active role in advising on selection and in making use of alumni networks. But also tracking and informing the alumni of the programme.

Clear choices at the start should be made, in principle, management costs and programme costs may be considerably lower than at present is the case. In fact, this option could be considered as a return to the origins of the programme when there was no explicit focus on capacity building, but it emphasises more the Dutch interests.

2 Continuation of the programme with a clearer emphasis on capacity building (development cooperation)

In this option, the main focus of NFP on capacity building of organisations and development cooperation policies is further enhanced. This would require that fellowships would be combined with other capacity building activities. The organisations would become the main focus rather than the individuals, as is currently the case. Therefore, front end mechanisms to link fellowships to other capacity building activities in line with well-known capacity building strategies should be envisaged.

The present evaluation has shown that such a focus cannot realistically be realised in a large number of countries and a limitation of the number of countries (e.g. to partner countries) would be required. The course offer should not be limited to courses in the Netherlands, but the possibility of also having offering them in country courses and in the region should be considered. The use of more local training and education opportunities will provide the opportunity to align more closely to the capacity needs and capacity development policies of the countries and limit the costs of the fellowships provided

The development sections of embassies would need to see the NFP as an integral instrument in their development cooperation strategies. A pro-active role of the embassy would be required The programme would need to make a link to the organisations involved (universities, government organisations or business). This can either be upfront (similar to MYA), involvement of

organisations in selection process (agreed number of fellowship to be awarded to the organisation- sector) or through follow up after the return of the alumni (JICA, Danida)

Even with a limitation to considerably less countries, the management costs of the programme are likely to increase, because selection requires considerable attention, just as more guidance during implementation, follow-up and strengthened monitoring and evaluation. Even with a focus on low income countries, contributions of fellows and/or organisations should not be excluded beforehand. This option is a step further on the way towards more focus on capacity building. The main

challenge is to reorient an essential supply-driven programme to a real demand-oriented programme that is still manageable.

3 Phasing out of the programme

With a strong focus on capacity building and development cooperation policies it might be

questioned whether a separate fellowship programme should be maintained. Fellowships can also be provided through other capacity building programmes such as NICHE but also through sector support programmes.

110 Evaluation of the Netherlands Fellowship Programme (NFP) 2002 - 2010

As a consequence of the phase-out of the programme possible benefits for Dutch economic and public relations will diminish. It will also have consequences for Dutch HEI specifically for those institutes for which NFP educating and training is their core business. On the other hand continuation of the Niche programme will still further the internationalisation of Dutch education.