El Grupo Técnico de Género como herramienta al primer

Diaries were chosen as the first method of qualitative data collection because, as mentioned in chapter three, they can give us ‗internal‘, participant data and as such offer an insight into the learning process which quantitative data cannot. They also give us access to introspective data which we may not be able to obtain through other qualitative methods such as observation (Bailey and Ochsner 1983:189). Nunan (1992:118) describes diary studies as ‗important introspective tools‘ in qualitative research, not least because they allow researchers an insight into the affective factors surrounding learning, something Dornyei (2007) supports. Krishnan and Hoon (2002:228) suggest that diaries can also work as a ‗powerful tool‘ in allowing students to evaluate courses, giving us an ‗insider account‘ (Dornyei 2007:157) of the

classroom. Diaries can provide useful sources of data from a descriptive point of view because they allow us to see how a learner‘s thoughts change towards a given teaching method and can also suggest useful points of development which we can follow up with more extensive interviews. Halbach (2000:85) summarises these benefits clearly: ‗By reflecting on the processes that go on inside the writers‘ minds, they open up fields that are not normally accessible to researchers‘. They also, of course, provide opportunities for a learner to demonstrate what they may have noticed in the form of a written report, something we

on their learning in a similar sense to the kind of think-aloud protocols employed by Alanen (1995). She argues that this kind of report can provide evidence of noticing or lack of noticing and as such is valuable research evidence. We would also argue that a diary allows learners to reflect on learning in a way which they cannot do during a think-aloud protocol, as they have time to reflect, under considerably less time pressure. In addition, as the data from the pilot study shows, diaries can provide a large amount of data which can contribute to our understanding of how learners evaluate different methodologies.

We have argued previously that many similar studies have tended to ignore such ‗internal‘, qualitative data in favour of one or more tests. As a result, the assumption seems to have been made that superior post-test scores are proof that one type of instruction is more effective than the other. This seems to ignore the fact that learners themselves are recipients of any given methodology and their belief in its effectiveness must play at least some part in how effective it actually is. We have argued that tests can give an objective and reliable measure of scores in each experimental group but of course they do not tell us how learners themselves perceive the different classroom methods. This seems essential if we are to gain a fuller picture of the two teaching frameworks being contrasted and how they affect acquisition of the target DMs: ‗externally‘ in a test score and ‗internally‘ in the learners‘ eyes.

Despite these arguments in favour of using diaries there are, naturally, several threats to the validity of the data and the way the data collection is managed. Each of these will be discussed in turn. The first weakness is that the diaries used in this study were solicited by the researcher. We might argue that this has the potential to make the data unreliable because learners may not have truly reflected on how they felt but attempted to write what they think the researcher wanted, knowing he would read it. This is certainly a possibility but was countered in two ways. Firstly, clear instructions were given in the guidance to students, making it clear that they were not being asked to comment on the teacher themselves or make a judgement on whether the lessons were ‗good or not‘. Secondly, students were assured that the diaries would only be seen by the researcher and that anonymity would be maintained in any subsequent use of the data.

The second potential weakness is asking learners to write a diary in English, the L2 in this case. Intermediate learners, if required to write a diary in the second language ‗may not find the task simple‘ (Krishnan and Hoon 2002:227) and as a result could then lose motivation and interest in keeping their diary. This might lead to diary data only being available from a certain number of subjects in the sample, which in turn could lead us to suggest that the diaries are not representative of the sample as a whole. Whilst this was a possibility, asking the learners to write their diaries in their L1 and then translating them into English was not a realistic option in this study. It would not have been practical or financially viable to arrange for a number of diaries to be translated from Chinese to English. Even if this had been possible, we would then have been faced with the real possibility that the translations were not a ‗true‘ reflection of what learner‘s had originally written. The obvious solution to this was to provide learners with a ‗model‘ diary entry, as we did in the pilot study, to give them an example of what they could write. This did not remove the difficulty of writing the diary in the L2 but it offered essential guidance to learners. Dornyei (2007:158) recommends ‗a detailed training session to ensure that participants fully understand the protocol‘, something which Bolger et al. (2003) also suggest. We provided this with a ‗model‘ entry and an explanation of the instructions before the first input session. The model entry can be found with the instructions given to learners in appendix four.

The third potential weakness of diaries is related to the second and is probably the biggest difficulty. Dornyei (2007) and Bolger et al. (2003) note that diaries demand a lot from the participant in terms of their time and commitment to writing them and this can mean that participation gradually tails off over the course of a study. We helped to reduce this in the following ways:

1. The diaries were made as easy as possible to write by providing a model.

2. The diaries were collected at regular intervals to ensure learners were participating and completing them.

3. It was made clear that each entry was event contingent, i.e. they needed to write an entry after each lesson.

4. Learners were offered an incentive to write their diaries. In this case, I corrected the English used by learners and returned this to them so they could perceive there was a learning ‗pay off‘ for them. Only the original data was used for the study and only spellings were corrected to facilitate analysis with CAQDAS software.

A final problem is one which several researchers have noted; the data is difficult to analyse objectively (for example, Leung 2002, Dornyei 2007). Should the data be analysed

subjectively, it could lead to us finding only what we are hoping to find and not what the data actually tells us. In this study we attempted to overcome this by using CAQDAS software to help analyse the data. We will describe and justify this in the data analysis section of this chapter (4.4.5).