Mecanismos que marcaron el acceso a la propiedad agraria durante la Reforma y

For the future, the European Voluntary Service (EVS) and from 2018 the European Solidary Corpse (ESC) could be suitable for a counterfactual impact evaluation. EVS allows young people aged 17-30 to engage in unpaid and full-time voluntary activity in a host organisation within or outside the EU.12 _{The volunteer service involves many} different kinds of activities in various areas (e.g. youth work, cultural activities, social care, environmental protection, civic engagement, development cooperation, non-formal education programmes). Volunteers can engage in short-term or long-term service. Short-term EVS last between 2 weeks and 2 months, so that effects might be small and as a consequence more difficult to be identified as statistically significant (in contrast to being due to sampling variation). By contrast, in long-term EVS the individual service can last between 2 months and one year, which is likely to render bigger sized effects. As part of Key Action 1 of Erasmus+, the individuals are the action’s beneficiaries.

Organisations which need to hold an EVS accreditation are in charge of managing the service for the beneficiaries. In addition, they apply to the call for expression of interest proposing an EVS project. An EVS project can include just one to up to 30 volunteers. During the application process, the presented EVS projects are assessed against the relevance of the project, the quality of its design and implementation, impact and dissemination. The project itself can last from 3 to 24 months.13 _{Once a project is} awarded, eligible youths can apply directly through the participating organisations. The objectives of ESV are to reach out to marginalised young people and promote the following principles: diversity, intercultural and inter-religious dialogue, common values of freedom, tolerance and respect of human rights literacy, critical thinking and sense of initiative of young people. Appropriate outcomes for evaluating the impact of this action

12 _{Travel expenses, accommodation, food, local transportation, health insurance,} language lessons and monthly allowance are covered and financed by the European Commission (the volunteer contributes only a 10% of the travel expenses).

13 _{EVS can be also involved in large-scale EVS events and strategic EVS projects. Large-} scale EVS events entail at least 30 volunteers and must include complementary activities (i.e. conferences, seminars, meetings and workshops) in addition to the EVS activities. Strategic EVS projects are granted to experienced EVS coordinating organisations for projects aiming at generating solid and systemic impact. They implement standard EVS activities involving several volunteers and may include complementary activities.

29

could be measures of tolerance and attachment to European values. It is unlikely to find measures for the objectives in administrative data. As a consequence, measures would need to be collected through surveys.

According to the 2017 annual work program of DG EAC (2016a), 3 Million Euros are allocated to the EVS insurance,14 _{while 400,000 Euros are allocated to large-scale EVS} events. The EVS budget has been increased in 2017 so as to launch the implementation of the ESC with the aim to further strengthening the impact of the action, as well as to open ESC to an even broader range of young people and organisations (DG EAC, 2016a p. 16). This witnesses the growing interest for EVS and calls for the need of evaluating the action through counterfactual impact methods.

5.2.1 Existing research

Since volunteering entails the provision of services for free, its value is not captured by official GDP statistics. According to an article from ‘The Economist’ the economic value of volunteering is often ignored by statisticians but can be estimated with data at hands and under some assumptions.15 _{In addition, the article claims that volunteering produces} also private benefits (in terms of improving health and employment prospects of volunteers) and social positive externalities (e.g. decreasing crime activity). Below we review a number of scientific studies providing some empirical evidence in favour of these claims. Descriptive evidence is based on self-assessments collected though surveys on volunteers and voluntary organisations on the impact of volunteering. For a review of this literature, see Huiting Wu (2011). In addition, few studies provide causal evidence of the impact of volunteering on the volunteers’ well-being, or on economic development and social inclusion.

A study by Corporation for National and Community Service (2008) evaluates the impact of AmeriCorps volunteering programme on the career prospects, civic engagement and well-being of volunteers. The study is based on longitudinal data and implements a Diff- Diff approach by comparing a group of individuals who participated in AmeriCorps in 1999-2000 with a similar group of individuals who expressed interest in joining AmeriCorps but did not enrol. The results show that, due to participation in the programme, AmeriCorps members are more likely to pursue public service career. In addition, results suggest positive long-lasting effects on civic engagement, and life satisfaction (measured 8 years after completion of voluntary service). Fujiwara and Kawachi (2008) study the causal impact of volunteer activity and community participation (as proxy for social capital) on physical and mental health in the U.S. This study is cross-sectional. In order to disentangle causal effect of participation in volunteering activity from unobservable individual characteristics such as personality or early childhood environment, the authors compare adult twins with different levels of social capital (proxied among others by volunteering). The causal effect is identified under the assumption that twins are similar in terms of personality and family background so that any difference in the outcomes is due to social capital. They find evidence of positive causal effect of social capital on physical health. In addition, the positive association between social capital on health is discussed in Kawachi and Berkman (2000).

Another strand of literature provides evidence in favour of the negative association between social capital and crime rates (e.g. Kawaguki et al. 1999). These associations are not necessarily causal. It is notable that the discussed evidence is restricted to the

14 _{The EVS insurance scheme aims at covering the risks run by the volunteers taking} part in the EVS.

15 _{See `The Economist’ (12th September 2014). The economic value of volunteering can} be estimated by calculating how much it cost to society to get these voluntary services produced if the volunteers had to be paid under regular contracts.

30

US. To the knowledge of the authors, causal impact of volunteering has not been investigated in the European context yet.

While the evidence discussed above refers to volunteering in general, to the knowledge of the authors the causal impact of EVS has not been examined so far. The EVS action has been evaluated only by participants’ self-assessments of the programme using surveys. Based on the answers of the respondents, EVS has a positive influence on volunteers’ lives. They gain self-confidence, independence, increase their capability of dealing with life difficulties and cultural awareness (e.g. Structure for Operational Support, 1999; DG EAC, 2016b).

In contrast to participants’ assessment of the EVS, the ongoing EAC evaluation of Erasmus+ aims at evaluating the causal impact of EVS by means of counterfactual impact evaluation. To do this, the tender of this evaluation of Erasmus+ is intending to draw a sample of participating and non-participating young people in EVS.16 _{The tender} foresees running the survey twice, so to measure the outcomes of interest before and after participation. This would allow for conducting a Diff-Diff approach in order to evaluate short-term EVS. Since short-term effects – as discussed above – are likely to be rather small and difficult to find to be significant, another approach could be to evaluate the causal impact of long-term EVS.

5.2.2 Data

To the knowledge of the authors, survey data containing measures of the EVS objectives are not available for a big enough sample of European youths. Even if this were the case, the survey should contain information on young people who participated in EVS mobility. Therefore, DG EAC would need to conduct its own survey in order to analyse the causal impact of EVS on a number of attitudes of beneficiaries. An advantage of this approach is DG EAC’s choice of variables of interest for the analysis. The disadvantage is that collecting data is generally costly. It needs time to plan, organise and implement a survey design and collect and edit the data for analysis.

The duration of EVS can take up to 12 months, as such its impact on competences and career prospects might be more sizable then those found for youth exchanges.

In terms of survey design, it is feasible to first randomly draw a sample of organisations from the population of participating EVS organisations. Selected organisations would need to provide a list of all EVS applicants. Using this list a random sample of applicants (participants and non-participants) would be selected in each EVS organization of the sample. Because of the cluster design, the sample size of applicants needs to be increased compared to random sampling. It is important that once the sample is randomly selected, the survey to be conducted would need to include in-depth information on ability levels and socio-economic background to take selection bias into account.

5.2.3 Outcome variables

Given the major objectives of the EVS action, the following outcomes could be considered:

(i) measures of competences: e.g. competences in communication skills and languages, digital competences;

16 _{Currently there is no documentation available discussing how random samples of} young people are secured (see Section 3) and clustering of individuals in organizations is taken into account for the calculation of the sample size.

31

(ii) attitudes: e.g. towards European values and cultural diversity, degree of tolerance and curiosity, participation in social and political issues, solidarity, increasing concern about inequalities and

(iii) measures of job search attitudes, career aspirations, interest in achieving higher education, entrepreneurial skills.

5.2.4 Methodology

Two alternative methodologies could be used in order to measure the impact of the action on the aforementioned outcomes. If pre- and post-survey data is available, a Diff- Diff approach could be employed. The Diff-Diff approach would allow comparing the average change in relevant outcomes (i.e. interest in European values) before and after participation between participants and non-participants. The corresponding average change for non-participants represents the baseline evolution of the outcomes – that is it represents how the outcomes of participants would have evolved over time if the individuals had not participated in the EVS (the counterfactual). This requires that non- participants should be as similar as possible to participants. To ensure this, it is important that the questionnaire reports information on the socio-economic and educational background of applicants. (For more information see Section 2.)

If instead only post-survey data is available, the analysis could rely on PSM. The causal effect of participating in EVS would then be identified by comparing outcomes between participants and non-participants matched on relevant background characteristics that are likely to drive the participation choice. The quality of the approach depends on the richness of the survey data on individuals’ background information.

5.2.5 Value added

Evaluations of the impact of EVS/ESC are drawn from descriptive evidence while a causal impact has not been shown yet. It is therefore not known whether EVS works and achieves its objectives. Given the growing emphasis attributed to EVS within Erasmus+ and the budget allocated, the importance of causal analysis on EVS has grown too. Therefore, a sound and robust analysis on the causal impact of EVS is needed to complement existing self-assessments of future ESC participants. In addition, the empirical analysis proposed in this chapter would represent added value to the existing literature on the impact of volunteering service, complementing the existing evidence on the causal impact of US volunteering programmes.

5.2.6 Time frame

Given the lack of individual level data on EVS (and future ESC) participants and non- participants, the research project illustrated above cannot feed into preparations for Erasmus+ post 2020. By contrast, the aforementioned analysis requires collecting individual data beforehand, which in turn requires time and resources for the planning and the implementation of the survey design. However, results could be provided in time for the final evaluation of Erasmus+ due by 30 June 2022. If this is of interest for DG EAC, the data collection should be planned as soon as possible.

The data collection should last at least 3 years in order to measure the outcomes of interest at most 2 years after the completion of the EVS / ESC service. Given the importance of the quality of the data for the final results, considerable amount of time and resources should be invested into the planning and design of the survey (at least 6 months).

5.2.7 Risk assessment

The risks associated with the above research project are mostly associated with the difficulties of ensuring high quality data collection through surveys. It is important that

32

selected EVS organisations can provide a list of EVS applicants that include contact details.

In particular, the JRC will take care about the following issues, so as to ensure the quality of the collected data:

• Decide the country coverage by maximising the representativeness of the country-specific samples with respect to their population

• Minimise low-response rate

• Design the questionnaires so as to maximise high quality responses.

Once the data collection procedure is sound and can be carried out properly, there are no risks associated to the analysis.

5.3 Pupil mobility

With more than 103,500 participants in 2014, this action represents a substantial area of Erasmus+ that accounted for an overall spending of 124 million euro (DG EAC, 2015b). Although mobility of pupils is not a mobility project per se but rather an activity embedded in strategic partnerships (Key Action 2), the fact that it targets pupils makes it very interesting for an impact evaluation: mobility at the early stage of the educational pathway may potentially contribute to lessening educational inequalities set during the early years and thereby improving children’s educational outcomes and future opportunities.

Pupil mobility is organized by schools in the framework of Strategic Partnerships, which aim at supporting innovation and exchange of good practices. ‘The Erasmus+ Key Action 2 Strategic Partnerships’ support the development, transfer and/or implementation of innovative practices and promotes cooperation, peer learning and exchanges of experience at European level.’ (DG EAC, 2015a). Given that pupil mobility is based on different strategic partnerships between schools, the aim of specific pupil mobility depends completely on the aim of its overall Strategic Partnership. Typically, Strategic Partnerships aim to improve inclusion and basic skills of pupils, so that pupil mobility is employed as a means for achieving these aims. Some Strategical Partnerships might also aim at decreasing early school leaving rates and improve cultural awareness and understanding.

As such, in contrast to Key Action 1 mobility, there is no overall aim that can be attributed to all pupil mobility. Hence, any kind of counterfactual impact analysis needs to assume a common aim across all Strategic Partnerships initiating pupil mobility. There are three types of exchanges that involve pupils for Strategic Partnerships in the field of school education. First, within ‘blended mobility’ physical and virtual mobility are combined. Second, short term exchanges of groups of pupils cover pupils’ mobility at host partner schools abroad. Both, blended mobility and short term exchange have durations between 5 days and 2 months. Third, long term study mobility regards pupils aged 14 years and older and covers durations of 2 to 12 months. Participation in long term pupil exchanges is more difficult to achieve within Strategic Partnerships, so that pupil numbers are very low (DG EAC, 2017. pp. 124-129)

Pupils participating in mobility are selected by the school, which participates in a Strategic Partnership.

5.3.1 Existing research

To the best of the knowledge of the authors, the impact of this action has not been evaluated yet. One relevant caveat in studying the impact of pupils’ mobility is its heterogeneity ranging from short-term group exchanges to individual long-term stays in

33

host schools abroad. Short-term experiences might have an impact, but this impact is probably small and as such only possible to detect by using population data or investing into large expensive survey data collection. It could therefore be argued, that impact evaluation should generally focus on individuals’ long-term mobility. However, the number of pupils taking part in long term activities is relatively low.

A second difficulty is the lack of individual data on which to evaluate the programme. Third, given that the aim of pupil mobility depends on the ‘Strategic Partnership’ within which it is organized, the choice of outcome variable for causal impact analysis is not straightforward.

The ongoing evaluation of Erasmus+ plans to carry out a counterfactual impact evaluation of pupils mobility based on survey data that will be collected by the contractor in the current academic year. The proposal aims to collect the individual data at two different points in time (before and after the mobility, for a group of participants and a group of non-mobile pupils) which allows using a Diff-Diff approach in which the before- after programme differences between mobile and non-mobile pupils’ outcomes are computed. This is an interesting approach which however will need to address a number of limitations on data compilation.17

5.3.2 Data

Any kind of data collections would depend on an agreement of the outcome variable to measure. Given that different Strategic Partnerships pursue different aims this is not straightforward to do.

For future evaluations, three different kinds of data sets could be considered:

i) Exploit national administrative pupil data sets including information on students’ educational achievement.

As discussed above, students’ basic achievement might - but not necessarily is - the aimed outcome of specific strategic partnerships. The following possibility of data access is however only of importance, if DG EAC would like to measure the impact of pupil exchange on achievement. In addition, educational achievement is unlikely to change with short term pupil exchanges. As a consequence, the following data collection should only be considered if the focus is on long term pupil exchanges (2 to 12 months) of pupils aged 14 years or older.

A number of countries conduct regular national exams for pupils at different ages and collect information on pupils’ results and other characteristics in a data base covering the entire population. The UK is an example, but also Scandinavian countries and some other central European countries have similar administrative data.

17 _{The average length of the considered mobility is very short (10 days). The estimated} average effects, if any, will be necessarily very small. As such, the sample size would need to be very big to show that the small effects found are not just by chance of drawing a specific sample. This is however the major limitation of the planned evaluation. The contractor proposes a so-called ‘cluster design’, collecting first information on 50 schools and then students within schools. Cluster sampling lessens the precision of the estimates (increases the standard error) especially if the schools differ from one another (so-called intracluster correlation) and the school sample is small. This is the case with just 50 schools planned to be sampled. Given the combination of both, short mobility and lack of precision due to small samples, it will be very difficult to find a significant effect. It is also important to note that any kind of analysis would also need to