Validación de los resultados

In this chapter I have presented and explained the aims and rationale for the research and presented the plan which was based on those considerations. I have also, for reasons of clarity, given an overview of how plans were changed.

In the following two chapters, each covering a distinct period in the research, I describe in chronological order how the research actually progressed. In the first of them I describe the part of the research based around the Long List of items, which includes showing how the items were designed to cover a broad range of areas of autonomy, and how care was taken with the wording of the items. This Chapter also deals with the unforeseen issue of translation (which had far-reaching consequences on the timescale of the research). In the subsequent chapter I deal with the Short List, including how it was selected from the Long List, and I look at the issues involved in presenting it as a questionnaire.

153

Step Description of original step Notes Section

1 Survey the literature of autonomy to find the main areas.

The range of areas to be covered was very large and this meant that selection had to be more focused on those which appeared to relate to the practical aims of the eventual questionnaire. This meant deciding which areas were peripheral to these aims and eliminating them. This meant that the ―objectivity‖ was reduced.

3.2.2, 3.2.3, 3.7.1.1.1, 3.9.1.1

2 Compose items which cover the areas of autonomy. Cover each area with multiple items worded differently so that a very large number of items, perhaps as many as 200, would result.

Scales to cover each area and sub-area were found to be impractical for the present research due to the great number of items which would be necessary. Respondent fatigue would be a major limitation. Items had therefore to be chosen more subjectively with the aims of the eventual questionnaire in mind.

4.2

3 Compose a Likert scale; upload the list of items to the Internet as a form in a webpage with each response coded with a ―score‖ to gather responses to the items. Respondents would give their feedback on the items to highlight any which they found unclear.

Access to the Internet was an issue in China. In other places students did not have access to computers in class. This meant that paper versions had to be printed and distributed and the papers returned by post and the data digitised. This severely slowed down the research. The issue of translation emerged as a major issue. The research was intended to be international, but items were not being understood as intended, and consequently it was decided to translate the items into a number of different languages. Quality of translation became a major issue. This became time-consuming as the items were translated into Mandarin Chinese and checked. Most of the respondents were Chinese, but more translations would have been done if time had been available.

4.3.3 4.3.4

3.13

4 Having amassed at least 200 responses from a wide range of respondents, items which had caused confusion would be removed. The data would then be factor analysed. The results of this would be used to eliminate items.

The number of respondents fell short of the numbers hoped for, which slowed down the research further. It meant that the numbers of responses required for factor analysis would be reached much later than envisaged. In order to proceed with the research it was necessary to reduce the number of items to make the ratio of respondents to items acceptable for factor analysis. This meant reducing the number of items to 50. This item reduction was carried out by statistical means

5.1

5 The remaining items would be used to form a well- designed questionnaire.

6 The questionnaire would then be administered over the Internet to as broad a sample of respondents as possible, numbering at least 200.

154

Step Description of original step Problems which emerged during the research Section

7 Items which had caused confusion for respondents would be eliminated and the data would be factor analysed to eliminate items which did not contribute to the factors found.

5.1

8 The questionnaire would again be formatted in accordance with guidelines in the questionnaires literature.

9 Data would be gathered from a large number of respondents. Teacher estimates of autonomy would be gathered longitudinally, ideally at beginning and end of a course.

At the large scale the stage of data gathering was not carried out. Only one

independent group (i.e. in addition to my own class) was found. This was partly due to the delays caused by translating the questionnaire, but mainly due to lack of responses to my requests for volunteers.

5.2

10 Small scale data gathering in tandem with the questionnaire. Learners would be observed, interviewed, and feedback sought from their teacher to gather data on the respondents. I would teach one class and find another class and teacher at Warwick so that I could interview students who were not from my own class.

Qualitative data gathering in tandem with the questionnaire was carried out on a presessional course and on a third year undergraduate course. The presessional was my own class and the undergraduate class was that of one of the lecturers at Warwick University.

Only one student who had agreed to be interviewed actually came for interview. This was caused by the earlier delays in the research caused by translation and by the difficulty of finding sufficient respondents to the Long List. The delay meant that interviews clashed with end of year assessments.

5.2

11 Comparison of the questionnaire and the qualitative data to gain insights on its viability and validity.

Comparison with the literature: the data so far gathered were standardised so that they could be combined to carry out factor analysis. The results of the factor analysis would be used for construct validity.

Comparison was made of the small scale qualitative and quantitative data which had been gathered to see whether the questionnaire was functioning well for individual learners and classes to provide a qualitative indication of validity.

3.12.1

155

4 THE LONG LIST STAGE

4.1 Overview

In this chapter I describe the compilation of the Long List and give details of the data gathering which was carried out using it. The purpose of this stage was to gather data in order to carry out item reduction to select the best-performing items. As

previously stated, factor analysis had originally been envisaged for this, but

insufficient returns were obtained (the actual procedure used is given in Section 5.1).