Funcionalidades del rol Experto

The pilot study was conducted only in primary schools, in two phases, focusing on

the generation of a sample and on the piloting of the teacher questionnaire and pupil

questionnaires. This consisted of piloting the research instruments and the methods

of data analysis and was considered a formative process. It was designed to provide

feedback on the research design and field procedures and to generate – and respond

to – any potential issues with the data collection tools. The tools which were trialled

and evaluated in the pilot study were the pupil questionnaire, pupil interview, teacher

questionnaire and the teacher interview. Firstly, this section provides an overview of

the instruments that were piloted and this is followed by discussion of the impact of

the pilot study on the final study method.

3.10.7.1 Questionnaire pilot

A postal questionnaire was used to gather data quickly and easily (Appendix 19).

Online questionnaires have a low administrative cost (Gray, 2004) but the teachers

consulted advised against this approach, to ensure a good return. The highly

structured questions and the clarity of the instructions were intended to reduce the

number of incomplete responses (Denscombe, 2003) and increase the response rate

(Burton et al., 2008) through making the questionnaire easier and quicker to

complete.

3.10.7.2 Pilot Interviews

Although there is evidence to suggest that responses in telephone interviews can be

as honest as in face-to-face interviews (Denscombe, 2003), due to the close

geographical proximity of the participants, the length and small number of interviews,

150 were selected for the pilot to assist the interviewer to establish a rapport with the

interviewee and to read the interviewee’s body language (Bell, 2001; Drever, 2003 and Richie and Lewis, 2006). The semi-structured interview format was selected to

provide flexibility to adapt the questions to the responses given (Cohen et al., 2003)

(Appendix 20). Audio recording was used to capture and analyse the responses.

3.10.7.3 Participants

Quota sampling was used to ensure that participants were from schools teaching

Primary Languages in one of the two selected local authorities. The sample

contained schools from a variety of socio-economic areas and with different levels of

attainment (using Ofsted reports and KS2 SATs results). The sample for the pupil

interviews was selected through negotiation with the class teacher to ensure that the

sample met the requirements and, importantly, that the pupils would not be distressed

or anxious in any way about participating in the study. As purposive sampling

lessens the external validity, there was a need to ensure internal validity. For

example, more than ten questionnaires were posted to lessen the impact of

experimental mortality (Cohen et al., 2003) and two teacher interviews were

arranged.

3.10.7.4 Pilot data analysis

The questionnaire responses were checked for completeness and accuracy (Cohen et

al., 2003). The responses were collated (in Excel) and blank rows and columns were

inserted to make the data easier to read and analyse, as advised by Munn and Drever

(2004). The closed questions were pre-coded but for open-ended questions a coding

151 The open-ended questions were used to gather qualitative data, attitudinal responses,

rich data and thick description set in context (Merriam, 1988; Stake, 1996) whereas

closed questions were used to gather information quickly and easily. The

quantitative data were scrutinised to identify patterns and irregularities and data from

the two local authorities were compared.

Due to the time constraints and the nature of the research questions, the interviews

were recorded but not transcribed and the recordings were used to identify key points.

The qualitative data (from the questionnaires and interviews) were used to identify

the range of experience and find examples and explanations to give depth and

meaning to the points made; whereas the quantitative data were used to identify the

range of distribution i.e. how typical the experience described is amongst the sample.

Cross case analysis was used to compare the findings arising from each of the four

cases and to identify similarities and differences which emerged. Furthermore, cross

case analysis could potentially contribute to the robustness of the study (Yin, 2009),

particularly when compared to single case study design. The reason for carrying out

an analysis of all of the four cases together was to provide a background picture and

to identify anomalies. The reason for conducting a case by case analysis was to

show how each case worked.

An atomistic approach was used to analyse the data from the questionnaire (Burton et

al., 2008). The mapping of the questionnaire and interview questions against the

research questions was then used to identify sources to address each research

152 responses - to identify comparisons and contrasts and to address each of the research

questions- and with the literature review to relate the outcomes to the field of inquiry.

3.10.7.5 The findings of the pilot study and their impact on the final study method

The findings related to each of the research questions were analysed but are not discussed below, as they were broadly consistent with the full study. This chapter concentrates on the impact of the pilot study on the final method used.

Overall, the approach used was an effective means of yielding appropriate and

sufficient data to answer the three research questions within constraints (both

temporal and financial). To avoid ambiguity, clear questions were used (Muijs,

2006) and both confusing jargon (Bell, 2001) and leading questions were avoided.

This appeared to be successful, with only one respondent not answering one

question. Another strength of the approach was the high response rate (9/10) for the

teacher questionnaire which was not anticipated as response rates to postal

questionnaires can be low (Muijs, 2006). This may be linked to the inclusion of a

personalised covering letter and a stamped addressed envelope with the

questionnaire; or as Muijs (2006) argues the questionnaire format is familiar to

respondents and can be completed at the respondent’s convenience. However, despite this success, for the final study, the teachers involved suggested the

questionnaire should be administered face-to-face.

The pilot sought to rehearse and critically evaluate the methods and processes used,

153 unless they were significant in re-shaping the method. The pilot was very small in

scale and, as the main study was on a much larger scale involving a much larger

sample, some elements of the approach trialled in the pilot were adapted for the

main study. These include the decision to administer the teacher questionnaires in

person (where possible) and to use specialist software for the analysis of the data, as

discussed below.

The pilot study identified potential time management issues. Firstly, the focus of the

study was on answering the research questions, yet many of the questionnaire and

interview questions in the pilot questionnaire aimed to gather contextual information

(such as who teaches languages, how often, teacher qualifications etc.) rather than

information relating directly to the research questions. As a result, the data entry

and data analysis processes took longer than anticipated, especially as an atomistic

approach was used, and for which there was an opportunity cost. This is consistent

with the advice by Burton et al. (2008). Therefore, the main study was more

focused and it can be seen that the final questionnaires reflect this.

The follow-up of the questionnaires resulted in a high return rate, but was time

consuming. This is not uncommon (Muijs, 2006). Also, the amount of time required

to prepare the questionnaire and collate responses was underestimated in this pilot –

as was the time required to analyse data as it is gathered - rather than after it has been

collected, as advised by Silverman (2005). Following the pilot study, estimations of

the time required were revised and, as mentioned above, the decision was made to

use specialist IT packages for the data analysis; namely the IBM Statistical Package

154 As a result of the pilot, greater consideration was given to the timing of the data

collection, particularly for the interviews, as pre-Christmas is a busy time in primary

schools. Fortunately, the timing did not appear to affect the response rate for the

questionnaire though it did appeared that the participants were less willing to be

involved in interviews. For the main study, the Year 6 questionnaire and interview

data were collected in the summer term and for the Year 7 data there were two

collection points: autumn and summer. It was also recognised that it would be

essential to liaise closely with schools to negotiate mutually convenient times for the

data collection. This was an important consideration in shaping the final structure of

the study.