Conformación del Patio: Espacio de Encuentro Social y Control Ambiental

the grades they gave first year doctoral students. Punctuation, spelling and quality of content were some of the features of writing included. The staff members were asked to judge the severity of writing problems of native and non-native speakers using a scale of 1 – 3. It can be seen then, that this type of guided questioning related to perceptions, is entirely appropriate within this type of research. When combined with the opportunity for participants to explain their scores, it can overcome the problem of losing focus within the research and becoming steered-away from the research questions, which is a limitation associated with more interpretivist approaches to education research (Shipman, 1997: 84).

Analysing student perceptions

Nvivo 10 was used to code and theme all the data. Responses and explanations were analysed using different coding approaches. In-Vivo coding was used for some personal, or interpretive meanings (Saldaña, 2015: 61). The coding allowed for a participant-inspired information to be recognised while coding where information related to more conceptual and theoretical level of analysis (Saldaña, 2015: 95). The Figure 10 shows some examples of the codes that were used in relation to the question about whether and why the participants enjoy writing in English.

These codes were themed. For example, a recurrent feature of enjoyability related to success or achievement. This theme included the codes (or NVivo nodes); ‘being able to guess the answer from multiple choice, confidence right answer, good grade, high scores, and (knowing the) incorrect answer’. Figure 10 shows some of the codes related to the challenges reported by participants. These were also themed. An example of a theme related to challenges reported is one related to structuring challenges, at sentence level and whole text concerns. The

following codes were included in this theme: Difficulty ordering objectives, finding grammar mistakes, finding less popular sentences and paragraphs, linking sentences to ‘make it smooth’ and choosing sentence structures. The

109

findings from these ‘perception’ analyses are presented in the findings chapter on page 161 and discussed in more detail in the discussion chapter.

Figure 10. Codes/Nodes for participants’ perceptions of enjoyability and challenge.

This process was repeated a third time when participants were asked what different writing tasks were useful for or what they felt their purpose had been. Examples of some of the initial codes are found in Figure 11. An example of an emerging theme was usefulness to practice for an exam or final assignment. The responses related to the usefulness of written assignments are described in the findings chapter.

110

Figure 11. Codes/nodes for participants’ perceptions of the usefulness of writing tasks for different purposes.

Retrospective Reports: Interview 3

In order to understand the writing processes used by the participants, they were asked to ‘talk through’ an authentic and recent piece of English writing they had created for a university purpose. This activity required them to recollect what they did to complete the writing. They were asked to explain what they did from beginning to end.

111

Stimulated retrospective think alouds often involve showing the participant a video recording of their action, and requires them to remember what they were thinking at the time. The dependence on recall and the ability of participants to explain the iterative nature of writing reduces the validity of this approach. The ability of participants to be able to remember what they were thinking during their actions, is highly questionable and it is more or less accepted that these methods do not provide a complete picture of the cognitions at play during the activity. In response to this, the participants were asked what they did, or the stages they went through as they created the text, rather than what they were thinking about. When the participant had

explained one aspect of the process, they were asked, ‘What did you do next?’ All participants who provided copies of their assignments took part in this part of the research. Only one postgraduate did not provide any assignments and so did not complete this task.

Participants were asked to find a recent assignment in the hope they would have better recall of what they did. The amount of time between the action and the recall is a significant factor within cognitive and retrospective research (Baddeley, 1972), but it is not the only or necessarily most important factor. For example, the topic, the time allowed to complete the assignment, the teaching instruction for the assignment and any personal meanings attached to the assignments are also of great relevance in terms of influencing accurate recall of writing behaviour (Tse, 1999), as well as the linguistic ability to put the recollected processes into meaningful words. As a consequence of these issues, research has often found that retrospective reports are rather incomplete, with significant gaps or lapses in information (ibid). Taken independently, they are of limited validity when capturing the composing activities of L2 writers and there is a high risk that participants offer information they believe the

researcher would like to hear (Shipman, 1997: 86). The stages of composing or the writing activities reported by the participants are outlined in the findings chapter.

112

Concurrent think aloud protocols Interviews 4: Writing-only task

Given the limitations of the retrospective report methods, participants were asked to undertake Plakan’s (2008) writing activities described within the pilot study section above. They were asked to explain what they were thinking about as they completed the writing. The same video and drawing example from the pilot study were used to show the participants what it looks like when people attempt to offer verbalisations of their cognitions during an activity. Having confirmed they felt comfortable and able to complete the task, they did not have the opportunity to practice the procedure, which may have impacted on the content and quality of their verbalisations.

During this session, participants followed the same procedure as the pilot sessions. This involved them undertaking a writing-only task described in the pilot study on page 76. The choice of topic and genre was an important consideration in this activity (Flower and Hayes, 1981). The participants level of confidence or apprehension towards the task was likely based on their prior experience and success of writing on this topic, including the type of

vocabulary required, as well as their ability to generate ideas. Participants’ responses to the task are described in the Findings chapter on page 169. In addition, they had to have some understanding and experience of writing an essay.

Two participants had not experienced writing an essay in English and their English proficiency was significantly lower than other participants, so they were given the option to choose whichever type of writing they wanted to undertake. The readability of the source texts in the reading-writing task may have also been too difficult for their level of English proficiency. This may have led them to feel less apprehensive than other participants who did not have a choice of topic or genre.

113

There was unlimited time given to the task and it could have been continued in a following session, although the participants were aware that the session would last no longer than two hours. They were encouraged to explain as much as possible about their thoughts as they read the question and started the process of answering the question. They were given no other instructions but were told they could use any resource they would normally use to

complete a piece of writing, such as the Internet or a dictionary for example. If the participant was quiet for longer than 30 seconds then they were prompted to explain their thinking by using the statement, ‘What are you thinking?’. Where answers to this prompt were limited, such as ‘I am thinking what to write next’, they were asked whether they could explain it in more detail. This led to more detail on most occasions but not in all cases.

The session was audio-recorded and verbalisations were transcribed. Field notes were taken to record what the participant was doing, including when they were pausing, editing the writing or re-reading material. These also included the behaviours during pauses, such as tapping a pen or looking out of the window. They also captured how participants had laid out the various pieces of paper and when they read back over the task prompts or made changes to their writing. The field notes detailed whether whole sentences were scratched through/ erased, or whether particular words were being changed. They also noted when a dictionary or other resource was used. This level of detail may not have been captured on video, although it would have been more reliable to trace any revisions to texts using Google Docs, S-notation, Trace-It, JEdit and LS graphing. The decision was made not to use these because the tracking process could have distracted the participants further. It was also important that participants had the choice whether to hand-write or type during the tasks. This is a confounding variable and there may be an impact on writing evident within the analysis and findings.

In addition to the behaviours recorded during the writing tasks, the field notes included analytic memos about my own thoughts during the activity. These included questions I planned to ask the participant after the writing, which were related to the writing processes or content. Memos were also written

114

when similarities or differences between participants or processes were noticed. For example, two participants pinpointed that they did not understand the meaning of the word, ‘revitalised’ within a source text. Other memos included how the behaviours or texts being created were related to the individual in question. For example, a Business major tended to give business examples in their writing but a TESOL major offered examples from applied linguistics. These types of differences were recorded when they were noticed.

While the reliability and rigour of field notes and analytic memos are often questioned they are deemed valuable insights for analysis and can be used to help researchers reflect on the data collection events and dynamics (Creswell, 2014; Taylor et al., 2015). They can aid recollection or recall of important connections made during the data collection stage. This supports the notion that analysis occurs concurrently during data collection itself. It is a complex process whereby the researcher’s schemata are equally as active as the participants. This means that the subject knowledge and experiences of the researcher come into play, this steers the focus or attention to particular aspects of events (Creswell, 2014; Taylor et al., 2015). This ‘noticing’- effect can be attributed to what the researcher aims to focus on, their goals or expectations, but also additional subconscious recall of existing knowledge that make synaptic connections with ‘online’ or current activities. It is this complex process of cognitive connection-making and analysis that can contribute to the opaqueness of data analysis (Zohrabi, 2013).

The verbalisation process could have impacted significantly on the participants’ ‘normal’ thinking and composing behaviour. The protocols transcribed are thus incomplete snapshots of some things the participant might think during the writing process (Cooper and Holzman, 1983), or may reflect what they think the researcher wants to hear. The protocols reflect the aspects of cognition or consciousness that the participants are aware of (Cooper and Holzman, 1983) and do not reflect the full range of thinking or ‘meta-cognitive processes’ they are engaging in (Nisbett and Wilson, 1977). The same limitations however can be applied to any qualitative research method. It is accepted, for example, that highly structured interviews or

115

observations can only ever offer an imperfect grasp of the total reality experienced by an individual (Flower and Hayes, 1981). Participants have difficulty answering questions or behaving unauthentically in many research studies and researchers have to be transparent about the limitations of the methods they selected. Given the nature of the research questions however, concurrent think-aloud protocols provided the closest real-time data about some of the considerations participants have while composing. Taken together with the fieldnotes, it provides evidence of the process or stages that the participant engaged in during that writing activity. Given the uniqueness and artificial nature of the writing task however, it is not possible to use this data to say that participants always or often follow the same process for

assignmentwriting.

The act of creating verbalisations in an L2 language also adds an important aspect to the task that could have different effects for each participant, depending on the extent to which they think in L1 or L2 (Hayes and Flower 1983; Cummins, 1981). This is difficult to ascertain and participants in the study were asked to say, during and following the concurrent think aloud activity, whether they thought about any aspect of the writing in Vietnamese.

Despite the limitations of concurrent think aloud protocols, it is possible to compare the activities participants reported to have undertaken within their retrospective reports, with their behaviours and verbalisations during the think aloud process. Although the genres tended to differ for these two research tasks, they can be used to offer some support for the self-reports provided by the participants. An order-effect of the tasks however may have led

participants to engage in the stages or activities they had previously reported to have done, rather than do what they would ‘normally’ do.

116

Interview 5: Reading-Writing Task

The same reading-writing task from the pilot study was used in the main study. Following each writing activity, participants were asked questions about their writing taken from notes made throughout the session. Questions were reserved for the end of the writing so as to avoid disrupting the writing and cognitive processes of the participant. The questions related to how the participant felt about the writing activity, whether it was similar or different to the first writing activity and specific questions such as why the participant had chosen to erase or alter certain aspects of their writing and the reasons for this. Questions were also asked about the content and examples written or excluded within the end product. The findings of the analyses of this data is found in Chapter 4.