Sectores de población más vulnerables

Retrospective thinking, also known as ‘stimulated recall’ or ‘aided subsequent verbal protocol’ (Van Den Haak, De Jong, & Schellens, 2003), has been widely used and is considered an effective way to study participants’ cognition and thought processes (Goo, 2010; Janssen, van Waes, & van den Bergh, 1996; Sasaki, 2008). Retrospection is based on information processing (IP) theory, which claims that information is stored in people’s short- term memory (STM) and is available for retrieval through verbal reports (Sasaki, 2008). This involves verbal reporting or commenting on people’s cognitions with reference to prior behaviours (Borg, 2006). This definition highlights two main characteristics of retrospection: first, that it is verbal, and second, that it is asynchronous, as it occurs only after a task is finished (Janssen et al., 1996).

Originally, I intended to use concurrent think-aloud to gather data. Unfortunately, during the pilot phase of the study (see below) this method to elicit a direct representation of the

teachers’ cognitive processes while they were constructing their grammar assessment tasks, I encountered rejection from not only the administration party but from the teachers

themselves. This was due to the fact that constructing assessment tasks, especially in the form of written exams, is subject to confidentiality and secrecy. As a result, I resorted to an

alternative, retrospective thinking. Retrospection, therefore, was employed in this study as a means to allow participants to explain/describe what went on in their minds while they construct grammar assessment tasks.

4.4.3.1 Retrospective thinking design.

A s described above, retrospection is one of the main techniques used to capture teachers’ interactive thoughts and decision-making processes and it was thus considered a good alternative to think aloud to gather information with regard to constructing classroom-based assessments (RQ 1 and RQ 2). Retrospective sessions, in general, are unstructured in order to encourage the participants to take the initiative and talk about any aspect of the phenomenon under study (Borg, 2006; Meijer, 1999; Woods, 1996). However, as Woods notes, the researcher should sometimes play the role of a facilitator or a prompt, because teachers vary greatly in the extent to which they take the initiative to comfortably identify episodes and comment on their own practices.

In this study, retrospection was based on the participants’ classroom-based assessments. I used samples of the participants’ written exams to encourage the teachers to walk me through the cognitive process, guide them in writing their exams and let them reflect on the factors that would influence their practices or prevent them from enacting their beliefs.

Conducting retrospection proved to be a very appealing method for two reasons. First, retrospective sessions were time-efficient, since the participating teachers were able to execute a task in their own manner and pace. Second, since the retrospection was carried out in a bilingual context and English was used as the means of communication, it was probably

less difficult for the participants to verbalise their thoughts in English after they had completed their written exams rather than while they worked on them.

However, there was a major concern regarding the fact that the participants might have produced biased accounts of the thoughts they had while performing the tasks, i.e., talking about how they wrote their exam. They might, for instance, have forgotten specific things that had occurred when they were writing their exams. Ericsson and Simon (1993) emphasise that vital information may be lost in the case of retrospective research, which is confirmed by several studies (e.g. Russo et al., 1989; Teague et al., 2001). Bias might have also risen as a result of the participants deciding to conceal certain thoughts they had, invent thoughts they did not have, or modify their own thoughts, perhaps due to reasons of self-presentation or social desirability.

Despite this drawback, retrospection proved to be a valuable tool to gain access to teachers’ mental representations of writing exams. Furthermore, the retrospections were one of the various tools for data gathering in this study, and thus facilitated the triangulation process to ensure better validity of the research design.

4.4.3.2 Retrospective thinking data collection procedure.

Retrospective sessions took place after the semi-structured interviews were conducted (see Table 4.4). All such sessions were conducted in English. As with the interviews, the female participants had the retrospection sessions in the offices in their respective educational facilities, while the male participants had theirs consecutively with their interviews at the Executive Hotel, because it would have been inconvenient for them to make the trip twice to meet me. All the retrospective sessions were audio-recorded digitally. Session durations

varied from 10–20 minutes, depending largely on the teachers and the time available. A total of five-and-a-half hours worth of data were collected.

Each participant presented a sample of her/his written exam during the retrospection, which was used to stimulate and prompt her/him to talk about what went on inside their minds during the construction and writing of the exams.

4.4.3.3 Retrospection participants’ demographics.

20 EFL teachers volunteered for the retrospection, including both females (N = 15) and males (N = 5). Half of the participants’ (N = 10) were MA holders while the other half were PhD holders. Their teaching experience varied from 4 years to over 15 years. The majority of the participants were Saudis (N = 19). Table 10 summarizes the participants’ information.

Table 10.

Retrospection Participants’ Demographics University/

Institution

Gender Qualification Origin

Male Female Masters PhD Saudis Non-

Saudis A 4 8 7 5 11 1 B Ø 1 Ø 1 1 Ø C 1 4 3 2 5 Ø D Ø 2 Ø 2 2 Ø Total number (% of sample) 5 (25%) 15 (75%) 10 (50%) 10 (50%) 19 (95%) 1 (5%)

It is important to emphasise here that the participants who agreed to the retrospection

constitute a part of the interview participants’ sample and are the one who provided a sample of written exams for the document analysis (see section 4.5.4 below).

4.4.3.4 Retrospective thinking data analysis procedure.

The transcription procedure of the retrospective audio data was identical to the interview procedure. With the aid of Dragon, I was able to transcribe all my audios to texts in a matter of a few hours. Again, I relied on oTranscribe to transcribe (verbatim) the text into MS Word documents in English. Once all the audios were transcribed and checked, I imported them to MAXQDA to begin the reading and coding processes (Figure 21).

Figure 21. Coding procedure for the data gathered from the retrospection.

This coding process involved identifying the relevant segments to describe teachers’ practices and the related factors. The next step was to go through the coding scheme again in order to put them together and organise them into categories. Thereafter, the categories were

reorganised to generate broader themes. Figure 22 shows the categories and themes which emerged from the analysis process.