Importancia de la evaluación de los resultados operativos

Before setting out and justifying the methodological approach and research design used in the study, this section provides a brief description of the participants (teachers and students), the context they worked and studied in, and how they were recruited for the

collected and the analytic procedures used. I begin with the four teachers, some of their characteristics, the context they worked in, and how they were recruited for the study.

The four teachers who agreed to take part in the study (three female, one male) all worked in the bilingual department of a secondary school which had been participating in the BEP since 2004. They were all content teachers, although two (the geography and the history teachers) were also qualified as EFL teachers. They had no previous

experience of working in bilingual education prior to joining the department, and all had between two and four years’ experience in teaching their subjects in English. The subjects taught were biology, technology, history and geography. In initial interviews, all the teachers expressed high degrees of motivation and satisfaction to be working in the Bilingual Project. They shared a view that teaching their subjects in English had energised their teaching careers, and all felt that they were in an intensive period of learning and professional development. This was in spite of a relative lack of formal teacher training in CLIL or related methodologies. They had all had short visits to the UK which involved brief periods of in-service training and school visits, as well as in- service sessions, some held at a nearby teacher-training centre. All four teachers, in common with the other teachers in the BEP, had to have provided evidence of a sufficient level of language proficiency (around C1 in the CEF) to teach on the programme, as well as being certified teachers of their subjects.

The teachers taught their subjects in English to students in the school’s bilingual section. These students were taught up to 40% of the curriculum in English for the first four years of secondary education, that is, until the end of compulsory secondary

education. This usually involved studying three subjects in English (science, technology and social science) as well as having an additional two hours of English, bringing the total of hours of English lessons to five per week. English lessons did not use an EFL methodology, but rather what was described as a ‘literacy’ approach, in which students followed a curriculum similar to that taught in English subject lessons in the UK. The students in these classes mostly came from a primary school in the catchment area which also belonged to the BEP, so that these students had been taught content through English since they were five or six years old. Although early observations produced evidence that there were quite wide variations in the students’ English proficiency, it was clear that they were confident in using English as a medium of instruction for a

wide range of classroom purposes, including both procedural matters and talk about curricular content. The classes in the study ranged from year 7 to year 10, so that the students ranged in age from 12 to 16.

My access to the field was gained through a long process of becoming familiar with these teachers and others in bilingual schools in the Madrid area through my

involvement in a Comenius project on CLIL teacher education (Hansen-Pauly et al. 2009). I had also led in-service training sessions on aspects of CLIL methodology at the local teacher training centre, which had been attended by three of the four teachers. It had become clear that sufficient trust to be allowed into these teachers’ classrooms could only be gained through a gradual process of establishing my own credibility as someone who had something to offer them. Through my involvement in the Comenius project, I was able to carry out a small pilot study in which I got access to one recording of a lesson by each teacher, and was able to carry out informal interviews. After two years of intermittent contact with the field, I gained their assent to take part in the final study, and data collection took place over two periods - November-December 2008, and April-May 2009. I also gained informed consent from the head of the school and the parents of all the children in the classes.

On agreeing to take part in the study, the four teachers were asked to choose a topic from the curriculum which they normally teach. I explained to them that they would have to fill in a sheet (the CoRe instrument) in which they would reflect on their approach to teaching the content topic in the unit, and how they approached language issues in teaching this topic. We then arranged for two consecutive lessons in which they taught aspects of this topic to be video recorded. Because of the study’s focus on teacher cognition, in which data sources other than classroom interaction would be used, I decided that two lessons (as well as a third lesson from the pilot study) would provide enough data for the study. In any case, as Seedhouse (2004) points out, in most CA-informed studies of classroom interaction, a corpus of five-ten lessons is usually considered to be sufficient. The teachers decided on the following topics and classes:

TOPIC CLASS

Teacher B (technology) Materials (making a wooden toy)

Year 7 (20 students)

Teacher C (history) Medieval art (iconography) Year 9 (18 students) Teacher D (geography) Development Year 9 (16 students)

Table 5.1 Teaching topics and classes chosen by teachers in the study

The teachers were given a few days to read and complete the CoRe sheet, which they brought to the pre-teaching interview. The interview schedule followed the structure of the CoRe instrument, with questions focusing on how the teachers saw the topic from the students’ point of view, both in terms of dealing with the content and any language- related issues. These interviews were audio-recorded and transcribed for analysis.

Within a week of carrying out the CoRe interview, the two lessons from the unit in question were observed and video recorded. The video recordings were made with a single, hand-held video camera. As Heath et al. (2010: 53) point out, ‘In many cases a single video camera will suffice. Indeed, multiple cameras tend to complicate data collection and analysis’. This was the decision taken in this study, as the focus was mostly on the teachers’ practices, it was sufficient to have one camera focused on him/her, but including students he/she was interacting with. As well as the sound captured by the camera, a good quality digital sound recorder was used as back-up and to have access to better-quality audio files for transcription.

Within a week after the two video recordings, the teachers participated in individual video-based comments sessions. In the intervening week, I had chosen four or five vignettes from the two lessons, in which issues pertaining to the teachers’ language practices were salient. As this was not an action-research project, I decided that it was better for me to choose the vignettes, in order to keep the study focused and to ensure a dialogue between the ideas and frameworks of interest to the study and the evidence as seen in the video data, and the teachers’ responses to it. These video comment sessions were also audio recorded and transcribed. Table 5.2 shows the different stages in the study, the different data collection methods used at each stage, the raw data produced, and the amount of data produced.

STAGE OF STUDY DATA COLLECTION METHOD TYPE OF DATA PRODUCED AMOUNT OF DATA 1. Focus on teachers’ pre-active constructions of language-related practices based on CoRe task. Semi-structured task- based interviews

Audio recordings and transcripts of interviews 4 x 1 hour interviews (approx 28,000 words) 2. Focus on teachers’ language- related practices in the classroom. Video recordings of classroom interaction.

Video and sound files of classroom interaction and transcripts. 12 x 1 hour lessons. (approx 60,000 words) 3. Focus on teachers’ post-active constructions of practice. Videoclip-based stimulated comment procedure. Audio recordings of teachers’ comments and transcripts. 4 x 1 hour interviews. (approx 12,000 words)

TOTAL 20 hours of recorded

interaction. (Approx. 100,000 words). Table 5.2 The data collected for the study

As can be seen, a very substantial amount of data was collected. This has important consequences (and dangers!) for the methodological approach taken in this study, which relies on fine-grained analysis of interaction rather than, for example, content analysis or grounded theory. Potter and Wetherell warn of the danger of ‘getting bogged down in too much data and not being able to let the linguistic detail emerge from the

mountains of text’ (Potter and Wetherell 1987: 161). To avoid this problem, they suggest that discursive studies can use small sample sizes:

Because one is interested in language use rather than the people generating the language and because a large number of linguistic patterns are likely to emerge from a few people, small samples or a few interviews are generally quite adequate for investigating an interesting and practically important range of phenomena. For discourse analysts the success of the study is not in the least dependent on sample size. The crucial determinant of sample size, however, must be, here as elsewhere, the specific research question. (Potter and Wetherell 1987: 161)

In the case of this study, the research questions relate to the teachers’ language-related understandings and practices as they are produced during interaction. This means that in an initial coding phase, segments of interaction that related to these constructions were identified for closer analysis. As Wiggins and Potter (2008: 84) explain, ‘The coding stage is the precursor to the analysis and involves sifting through the larger data corpus for instances of a phenomenon.’ In this way, the unmanageably large data corpus (for a study using micro-analysis of interaction) was reduced to focus on those stretches of interaction in which the phenomenon of interest in the study (teachers’ language-related practices and understandings) were seen to be salient.

The data analysis of the classroom interaction took place in three phases according to the three relevant research questions (1, 3 and 5). Thus, for research question 1, which focused on the relationships between the teachers’ pedagogical goals and the

organization of interaction in their classrooms, the corpus of twelve lessons was divided into segments in which the interactional organization displayed participants’ orientation to a particular pedagogic agenda. These segments were assigned to emerging categories, with the categories changing as new stretches of interaction were added, until the

analysis ‘settled’ with any segment of interaction being clearly allocated to one of the five categories, or micro-contexts. For research question three, all examples of a pre- emptive focus on language in the corpus were identified, and a collection built. This collection was then sub-divided into the different interactional ways of dealing with language pre-emptively which are analysed in chapter seven. For research question five, a similar process was used, with a collection of all examples of a reactive focus on language being built, which was then subjected to a more fine-grained analysis, the results of which are presented in chapter eight.

The analyses of the verbal commentaries followed a different trajectory. For question two, the post-teaching video comments were examined for ways in which the teachers provided accounts or explanations of the pedagogic goals they pursued in classroom interaction. These accounts were possible to identify as they were situated in the interaction just after the playing of the video, and my request for the teacher to say ‘what was going on’. As each account was produced after viewing an example of interaction in one of the micro-contexts, the accounts were examined for ways in which

the teachers’ descriptions of their pedagogic goals aligned or disaligned with the pedagogic foci as demonstrably oriented to in the emic analysis of the interaction. In responding to research questions four and six, the teachers’ verbal commentaries in the CoRe interviews and the post-active comments were examined for examples of

constructions relating to language as a curriculum concern, and as a matter of learners’ competence. Collections were built of segments of interaction where these concerns were topicalized.