3.1. The major source of information: students from the target countries
The most important target group of the study were students from the six target countries, as they would also be the target of a marketing campaign. Students from the target countries took part in the study either by filling in a paper questionnaire or by responding to the online survey. Data from both sources were put together in order to create a common data pool for the statistical analysis.
It is noteworthy that a substantial proportion of online-respondents had completed a Bachelor or Master degree and was already employed at the time the study was carried out. This group of potential future mobile students could only be reached with the help of the online survey and would have been “lost” if only paper based surveys had been carried out in secondary schools and universities. Pooling together the respondents to both surveys therefore led to a broader representation of relevant target groups.
3.2. Widening the perspective: experts, triangulation and comparison groups
Including groups other than students from the target countries served several purposes:
(1) Broadening the range of questions: Questions to which a valid answer could not be expected from the students, e.g. general views of internationalisation of higher education, student mobility and its role in education policy in the target countries, etc. were included in the interviews with representatives of public authorities and rectors of universities;
(2) Triangulation: Addressing different stakeholders enabled the project team to compare the students’ responses with the views of other important actors, e.g. parents or lecturers. The method of comparing the views of different groups on similar issues is known as "triangulation" in social sciences. It is used to increase the validity of results and conclusions. Information was gathered with the help of interviews (parents, rectors, etc.) and standardised questionnaires in the case of education staff;
(3) Global context: Responses to the global online survey allowed to situate the results from the target countries in a global context, and to address students from other world regions, e.g. Africa or North America, not covered by the target countries;
(4) Comparison with students who had decided against Europe: How do the views and perceptions of students in the stage of planning to study abroad compare with those of students already studying abroad, namely in the US? Why have these students decided against studying in Europe? International students in the US were used as a comparison group to students from the target countries and respondents to the global online survey.
The results presented in chapter IV of this report were based on both, responses of students from the target countries and information provided by parents, experts or students from other countries and regions.
3.3. Representativeness and validity of results
For various reasons the students participating in the surveys do not fully match the (hypothetical) total target population of the study, i.e. all students from the target countries with a minimum interest in studying abroad. Major reasons for discrepancies are:
(1) The creation of stratified samples for the paper based surveys in the target countries: the introduction of quota criteria (type of institution, geographical spread of institutions, level of study and field of study) was seen as the only way to ensure the participation of a broad range of students with different socio-economic and educational backgrounds and, at the same time, obtain sub-groups of participants with similar characteristics of sufficient size to allow for the application of multivariate statistical methods. These methods were necessary to identify the decisive factors influencing the students’ reasons to study abroad, the criteria for selecting the destination country, the perception of European higher education etc.
(2) Distribution of paper questionnaires through the channels of European scholarship organisations: The network of local offices of ACA members made it possible to reach a considerable number of students interested in European higher education or in pursuing an international education in general. However, the use of this network is likely to have resulted in an overall trend favouring European destinations in the study, i.e. non-European destination countries (including the US) were underrepresented. In addition, for every target country, the coordinating organisation’s country was overrepresented, i.e. Germany for Mexico and Brazil, the UK for India and Thailand, the Netherlands for China.
(3) Access to the global online survey via the websites of European scholarship organisations: Placing a link to the survey on the websites of ACA member institutions allowed bringing the survey to the attention of an important proportion of students seeking information on education opportunities abroad. Yet, the use of these channels probably also resulted in a large proportion of scholarship seekers among online respondents. Most online respondents were at an advanced level of study or already employed. Furthermore, more respondents to the online survey than to the paper based surveys favoured European destinations.
Hence, the samples of respondents are on the one hand highly representative for the student population of every target country: they cover students from different parts of the country and from different types of institutions, levels and fields of study. On the other hand, the relative weight of sub-groups of respondents with distinct profiles (field or level of study, destination, etc.) does not exactly match the relative weight of these sub-groups in the total target population (of potential international students).
How do the differences between the samples and the total target population impact on the validity of responses? In order to assess to which extent the results of the study are biased by the composition of samples or can be considered as robust and valid, two types of analyses were used:
(1) Identification of major factors explaining the variation of responses: Multivariate statistical methods were used to identify the impact of major factors on the variance in responses to individual questions and items (variance analysis). In addition to possibly biased criteria, i.e. level of study, field of study and destination country, the country of origin of students was included in the analysis.
The multivariate analysis clearly showed that the students’ country of origin explained most of the variance in responses to questions concerning motivations to pursue an international education, criteria for the selection of a destination country, importance of different sources of information, means to enhance the attractiveness of European higher education and differences between the member states of the European Union. Thus, the possible bias of the samples with respect to other criteria could not substantially impact on the results deriving from these questions.
Some of the questions and items were strongly correlated with the selected or preferred destination country of students, e.g. characteristics of the destination country or the ranking of major destination countries for a range of aspects. In order to control the possible destination country bias, the statistical analysis was done separately for students aiming to study in Europe and students aiming to study in the US, and presented in different tables.
The level or field of study of respondents influenced to some extent the students’ motivations to study abroad and the selection of universities. However, the respective correlations were limited to a small number of aspects only. These are described in the report but they did not require special analysis techniques.
(2) Measuring of composition effects on totals: The composition of samples (determined by quota criteria like the country of origin, the type of institution and level and field of study, or by factors like the students’ preferred destination country) could impact on the averages calculated on the basis of all responses to individual items. The stronger the correlation between different criteria and the item in question, the more the totals could be influenced by the composition of a sample and the higher the risk that it did not correspond to the "real" average in the total target population. In order to assess the impact of the composition of samples on the totals, respondents were weighed in a series of tests:
– balancing the proportions of respondents within the sub-groups defined by the combinations of quota criteria and destination country;
– adjustment of the proportions of sub-groups of respondents for every target country to the relative weight of these sub-groups in the overall sample;
– adjustment of respondents from individual target countries to the relative size of the total population of the country and to their relative importance within the total international student population, i.e. respondents from China and India got a much higher weight than students from other target countries.
By and large, the experimental variation of the composition of samples only showed a very limited impact on the totals for individual items or questions, resulting in variations ranging between zero and three percent. Furthermore, for most questions, there were no or only slight changes in the rank order of importance of items/responses. Only for questions strongly correlated with the destination country the impact was stronger, and special measures had to be introduced to control the “destination bias” (see above).
Overall, it can be concluded that the survey results described in this report are carefully analysed and only presented if the authors were convinced that they were valid and could be generalised for potentially mobile students from the target countries.