Aplicación del procedimiento de Diagnóstico del turismo de eventos en el

The pilot participant was an MA TESOL student recommended by the gatekeeper in Hanoi. The stages of the pilot are detailed below.

Interview 1

The ethical and informed consent form (Appendix 2) was discussed first. The participant had had access to it (in Vietnamese and English) before the first interview so had time to prepare any questions. Each section was explained and the participant was reminded of their rights in relation to confidentiality, right to withdraw and the issue about anonymity, which is outlined later in this

88

chapter. It was translated into Vietnamese to use if any questions were misunderstood. A question schedule was used during the first interview (Appendix 3). The questions were used as a guide to gather background information, including which languages were spoken at home, age when starting school and age when they started to learn English, formally and informally. A Vietnamese version of the questions was available in case the participant did not understand any questions or terms I was using. It was a ‘back-up’ resource and was not given to the participant.

Interview 2

A list of the BAWE genre families identified by Nesi and Gardner (2012) (Appendix 4) was used to focus on types of writing undertaken and to explore these in more detail, including when they were written and for what purposes. The genre labels were created following an investigation of student university assignments in the UK. Within the pilot study, the summarised list- version of the genre families did not provide enough detail about the function of the writing or examples of assignments the participant may have found useful to recollect which writing tasks they had undertaken. The decision was made to provide a fuller account of the 13 genre families in the main study. This would provide detail about the purpose of each genre family and examples of genres that may be included within the genre family (Appendix 5). This was also translated into Vietnamese and made available for the main study. The aim was to focus participants’ attention on the purpose of the writing tasks rather than the labels alone, which may be misleading for participants. The full genre categories by Gardner and Nesi (2013) can be found in Appendix 6. Interview 3

The participant provided assignments to show the types of writing discussed in the previous session/ interview. One recent assignment was selected to use for a retrospective report activity. The participant outlined the major stages of writing such as pre-reading, planning, idea generation, peer feedback and editing. However, the participant spoke abstractly about these processes and did not often refer to the assignment itself. The participant used the pronoun ‘we’ rather than ‘I’ and spoke of best practice in writing tasks from a teacher

89

perspective rather than focusing on the writing processes undertaken to create the assignment of focus. The decision was made to use concurrent think-aloud methods to explore the writing process as it occurs; to have more insight into what the participant physically does and thinks about as they are writing. Some of the limitations of this method are discussed in the next section.

Interview 4

The first half of the session was spent on think-aloud training using a video clip and modelling a think-aloud activity. The video clip was of a think-aloud protocol while cutting an apple into slices. I modelled the method using a drawing example. The modelling of the procedure did not use a writing activity because it may have lead the participant to verbalise certain aspects of their thinking over others. With no experience of this procedure, they would have used the information in the demonstrations to contribute to their schemata for the method and it would have been restrictive to provide them with any

prototype of the content they should or should not verbalise (Flower and Hayes, 1981). The participant then undertook a writing-only concurrent think aloud activity using an adapted version of Plakans’ (2008) method and task prompt,

“Most cultures have borrowed from other cultures. Choose a country that you are familiar with and write about something borrowed from another culture. Consider if this borrowing has been helpful or harmful.”

Plakans had interviewed an English language program coordinator and students enrolled on English language courses to find ideal exam topics. Most popular topics included Technology and Cultural borrowing. Plakans

developed an argumentative task because it is a common genre in academic writing. It was piloted and deemed sufficiently familiar to students. It also provided participants with a clear position to argue from (Plakans, 2008: 115). This task was selected for this study because the topic was useful for capturing culture-related information from the participants. The task was also open enough to give participants the opportunity to write about any culture or country, so they could personalise or depersonalise their answers. The

90

decision to use this topic and task may have steered or guided the type of content given by the participants and this is discussed later.

Plakans had directed participants to plan their answer and described how their writing would be evaluated. For this research, it was important not to lead the participants to undertake writing activities they may not usually undertake so the task prompt did not request them to plan. The writing was not being evaluated in a test approach so this part of the task prompt was also removed.

The participant was asked to explain their thinking at regular intervals with no more than 30 seconds of non-verbalisations. No additional questions were asked during the think-aloud session but notes were taken during the activity and questions were asked immediately following the writing session. Field notes detailed what the participant was doing throughout including where they had paused, edited or asked questions during the process.

Interview 5

The second concurrent think-aloud activity adapted Plakans’ (2008)

readingwriting task. The method and task prompt shows how the same topic (cultural borrowing) was used again. The same techniques were used in terms of eliciting thinking and making notes during the session.

“Globalisation has had a strong impact on the world. One issue of globalisation is cultural borrowing or adaptation. Read the following passages about this issue. Then consider your opinion about the impact of globalization on culture. Write an essay supporting your position and using examples.”

This task specifically asks participants to use examples in their writing. This directive was kept because it was helpful to see how the participant would include examples and how they would use the source texts. The participant was asked to explain their thoughts as they read the two texts and notes were made as they read and organised the texts in relation to the task prompt. The participant could write on the texts and task prompt sheet. There was no time

91

limit for the writing activity and it could have been continued into the next session if necessary.

For the reading-to-write task, Plakans searched for texts in textbooks,

magazines, ESL textbooks, and on the Internet. Plakans wanted to encourage citation by using excerpts from several texts. Plakans own version of the task required participants to cite from the texts (Plakans 2008: 116). The texts can be found in Figure 6 and Figure 7.

Figure 6. Copy of Plakans (2008) Reading-Write Think Aloud, Source Text 1. Plakans assessed the density level, as a measure of readability of the texts using different criteria. The word count is between 80-100, they are Flesch–Kincaid Grade Level 12 and have a Flesch Reading Ease between 30 – 60. The highest score for this index is 100, which is the easiest. As shown in Table 3, this means the texts were generally classed as difficult to read, but appropriate for undergraduate and postgraduates.

92

Table 3. Copy of Flesch Reading Ease table.

Figure 7. Copy of Plakans (2008) Reading-Write Think Aloud, Source Text 2.

To be fully satisfied that the source texts were accessible to the participant, it would have been necessary to assess their reading comprehension and that of the texts. Instead the source texts were used in the pilot to ascertain any comprehension problems or other issues. The pilot participant understood the texts, their main ideas and the task prompt but struggled with some

vocabulary, especially within the examples. These challenges were useful to show how the participant coped with tasks and texts with new vocabulary and non-culturally specific examples. The source texts were used again in the main study.

93

Insights and changes following full Pilot

The pilot was useful to confirm that participants should describe and identify the types of writing they had undertaken via a writing biography without the use of existing genre labels. It also confirmed that the genre family labels should not be used in isolation but that the social purpose and genre examples for each genre family should be given (Appendix 5). During the retrospective report of an assignment, the pilot participant was often vague and abstract in their explanation rather than focusing on the specific assignment . There was a concern that the participant was unable to recall important stages or writing processes so concurrent think aloud methods should also be used. Concurrent methods allow the researcher to witness first-hand the types of writing

activities participants engage in during the writing task by observing and noting the writing behaviours. The method allows for deeper insight into the cognitive processes of participants as they complete writing activities.

There are a number of pitfalls with concurrent think-aloud protocols that are well established in research. The aim to understand the full range of cognitive processes can never be complete because participants may not share all

thoughts or the cognitive processes may be affected by the think aloud process itself. The suggestion is that thinking to explain is cognitively different to thinking ‘intrinsically’. Additionally, think aloud protocols and the quality of the data gathered is largely dependent on the training participants receive

beforehand and on their metacognitive awareness or abilities. Paradoxically over-training for think alouds can be leading and provide participants with an overly structured conceptual framework which they may come to rely on or may restrict their thinking (Plakans, 2008). There is a need for balance of instruction, modelling and practice in think-aloud training.

During the pilot study, the participant talked about their ideas related to content and examples to use in their writing. They also wrote a plan structuring the content to be included in the essay. The participant then chose to erase or ‘scratch through’ some examples and so these did not appear in the final text or product. When questioned about the reasons for erasing or not including