Cohen and Scott (1996) argued that no single assessment method prevails in the field of language learner strategies: some research methods are well established but imperfect, while other methods have not yet been fully explored (p. 89). Chamot (2004) observed that the only way to identify learners’ language learning strategies – a learners’ mental processing, is through self-report (p. 15). She believed that the most frequent and efficient method for identifying students’ learning strategies is through questionnaires; other frequently used methods are retrospective interviews,
20
simulated recall interviews, written diaries and journals, and think-aloud protocols concurrent with a learning task (Chamot 2004, 2005). Adapted from Liu (2005), an outline of data collection methods by L2 learning strategy researchers is shown at the Table 2.1 below.
Table 2.1: Overview of Data Collection Methods by L2 Learner Strategy Researchers (Liu 2005, p. 47)
Researchers Data Collection Method
Rubin (1975) Observation; Intuition
Naiman et al. (1978; 1995) Interview; Questionnaire; Observation
Rubin (1981) Observation; Diary
Oxford and Nyikos (1989) Questionnaire – SILL
O’Malley and Chamot (1990) Interview; Observation; Think-aloud
Green and Oxford (1995) Questionnaire – SILL
Anderson and Vandergrift (1996) Think-aloud
Carson and Longhini (2002) Diary
Halbach (2000) Learning Diary
Chang (2003) Questionnaire – SILL
Gu, Hu, and Zhang (2005) Think-aloud, more precisely, probed
introspective verbal report
However, due to the fact that each individual learner has his/her own approaches to learning and learning strategies, therefore each assessment method has its advantages and disadvantages. The advantages and disadvantages of each method have been discussed extensively in the L2 learning strategy literature (Cohen and Scott 1996; Oxford 1990; O’Malley and Chamot 1990; Cohen and Macaro 2007). In general, questionnaire and the verbal self-report (or think-aloud protocol) have been the two main approaches to strategy elicitation (Macaro 2009). Table 2.2 below shows advantages and disadvantages of using these two L2 learning strategy data collection methods (adapted from Liu 2005, p. 48 – 50):
21
Table 2.2: Advantages and Disadvantages of Using Questionnaire and Think Aloud Protocol, Adapted from Liu (2005, p. 48 – 50)
QU AD 1. May provide a general assessment of strategies used across a wide variety of possible tasks.
2. May enable researchers to generate and test hypotheses through larger scale survey.
3. Representative data.
4. Anonymity assures more reliable data.
5. Uniformly organized data is more easily dealt with statistically. 6. Many convenient ways to administer.
7. The most cost-effective strategy assessment method. 8. Non-threatening under conditions of confidentiality. 9. Each group summary or comparison.
DA 1. Close-ended questions do not provide learners with much freedom to reveal in-depth ideas.
2. May not be able to provide detailed information on strategies used relating to a specific language learning task.
3. Some questions may be left unanswered for no obvious reason. 4. Limited interaction between researcher and subjects makes it difficult
to elicit information in case of ambiguity on the spot which may affect the reliability of the data.
TA AD 1. May provide detailed information on specific learning task.
DA 1. Data only reflects strategies related to the task at hand, not a general portrait of the individual’s strategy use in toto.
2. Subjects may rely on background knowledge and opinions about a topic rather than what they actually did.
3. Unedited data thus difficult to synthesize.
4. Some learners may not be able to articulate the strategies being used. 5. Time consuming and costly.
6. Respondents may produce unreliable verbal reports. 7. Verbal reporting has intrusive effects.
8. Respondents may differ in their ability to verbalise. 9. Weaker students may find it difficult to verbalise in L2.
10. Respondents may be too engrossed in task and forget to verbalise. 11. Respondents may not be able to remember mental events after
performance and may give faulty reporting.
22
This table indicated that questionnaires and think-aloud protocol (TAP) can produce different results. With regard to language learning strategies, TAP can only provide detailed information on the specific learning task at hand, rather than a general portrait of the individual’s strategy use in toto. However, questionnaires may provide a general assessment of strategies used across a variety of possible tasks and may enable researchers to generate and test hypotheses through larger scale surveys. In addition, the uniformly organized questionnaires data enables researchers to deal more easily with statistical data. In consideration of the advantages that may be offered by the use of questionnaires, this method was applied to the current study to investigate students’ LLS, instead of the think aloud method.
Macaro (2009) divides LLS questionnaires into two types: the general questionnaire, designed to assess the overall strategic behavior of the learner, such as SILL: the Strategy Inventory for Language Learning, developed by Oxford (1990); and questionnaires designed to assess specific skills or tasks (p. 18).
The greater numbers of descriptive studies have utilized SILL (Cohen et al. 1998; Macaro 2009; Nyikos and Oxford 1993; Olivares-Cuhat 2002; Oxford 1990, 1996; Oxford and Burry-Stock 1995; Tseng, Dörnyei and Schmitt 2006; Wharton 2000). This instrument has been used in studies that correlate strategy use with variables such as learning styles, gender, proficiency level, and culture (Green and Oxford 1995; Nyikos and Oxford 1993; Oxford and Burry-Stock 1995; Wharton 2000). The SILL instrument has two versions: version 5.1 and version 7.0. Version 5.1
consisting of 80 items, was designed for English speakers learning a new language. Version 7.0 consisting of 50 items, was designed for speakers of other languages learning English. Both versions include six parts that measure the frequency of six types of strategies: memory, cognitive, compensation, metacognitive, affective and social strategy (please refer to the Section 2.1.2 for more information on Oxford’s six classifications).
23 The six parts’ allocations in Version 5.1 are:
Part A, from item 1 to 15, is Memory Strategies, e.g.: I create associations
between new material and what I already know.
Part B, from Item 16 to 40, is Cognitive Strategies, e.g.: I say or write new
expressions repeatedly to practice them.
Part C, from Item 41 to 48, is Compensation Strategies, e.g.: When I cannot
think of the correct expression to say or write, I find a different way to express the idea, for example, I use a synonym or describe the idea.
Part D, from Item 49 to 64, is Metacognitive Strategies, e.g.: I try to notice
my language errors and find out the reasons for them.
Part E, from Item 65 to 71, is Affective Strategies, e.g.: I try to relax
whenever I feel anxious about using the new language.
Part F, from Item 72 to 80, is Social Strategies, e.g.: When I am talking with a
native speaker, I try to let him or her know when I need help.
The six parts’ allocations in Version 7.0 are:
Questions 1 to 9 are memory strategies, e.g.: I use rhymes to remember new
English words.
Questions 10 to 23 are cognitive strategies, e.g.: I say or write new English
words several times.
Questions 24 to 29 are compensation strategies, e.g.: To understand
unfamiliar English words, I make guesses.
Questions 30 to 38 are metacognitive strategies, e.g.: I try to find as many
ways as I can to use my English.
Questions 39 to 44 are affective strategies, e.g.: I try to relax whenever I feel
afraid of using English.
Questions 45 to 50 are social strategies, e.g.: I ask an English speaker to
correct me when I talk.
Each item for either version is scored from one to five: five is the most frequent use indicator and one is the lowest use indicator. Frequency of strategy use has been classified in the SILL inventory according to the following key:
24
Table 2.3: Frequency of Strategy Use (Oxford 1990)
Very high strategy use always or almost always used 4.5-5.0
High strategy use generally used 3.5-4.4
Medium strategy use sometimes used 2.5-3.4
Low strategy use generally not used 1.5-2.4
Very low strategy use never, almost never used 1.0-1.4
Some authors such as Tseng, Dörnyei and Schmitt (2006) have criticized the SILL as fundamentally flawed in design (Macaro 2009), pointing to such problems as the adoption of frequency-of-use scales with highly specific items of a different nature. They claimed that one cannot presume a linear relationship between the individual item scores and the total scale scores as these items are behavioural items (Tseng, Dörnyei and Schmitt 2006, p. 83). In addition, they claimed that the scales in the SILL were not psychometrically justifiable. They argued that it is possible to be adept in one kind of strategy generally even while scoring low on some of the items - one might be a good memory strategy user while being deficient in the use of flash cards, for instance (ibid). Further, they claimed that the SILL was in contradiction with learning strategy theory (ibid.). Macaro (2009) pointed out that debates over strategy size and abstractness contribute to these problems. He identified frequency use of a strategy as another problematic area. When both are an issue, this further compounds the problems (p. 19) (please refer to Section 2.4.1, for more information on strategy size and abstractness).
Despite its potential for inaccuracy, many researchers, Macaro (2009) for instance, still believe that, without any doubt, the best-known general questionnaire is SILL, the Strategy Inventory devised by Rebecca Oxford in the late 1980s (p. 19). The reasons for SILL’s popularity are:
SILL not only provides comprehensive coverage, it has also been submitted to reliability and construct validity measures and performs well (Oxford and Nyikos 1989);
SILL is one of the few instruments to have been tested for social reliability response data, ensuring that it is free from bias and that students do, in fact, answer the SILL honestly (Oxford 1996b);
25
SILL offers a good fit between the six factors originally conceptualized by the author, and the overall data provided by the population of language learners it was tested on (Macaro 2009);
Oxford’s taxonomy of six strategy factors provided the most consistent fit with learners’ strategy use (Cohen and Macaro 2007, p. 95; Hsiao and Oxford 2002; Liu 2005; Macaro 2009).
Many SILL studies allow for comparison (Y. Wu 2007)
Based on the above observations, the current study has adopted SILL to investigate students’ perceptions of learning strategy use. We used version 5.1 for the pilot study (see Section 5.3.2) considering it was designed for English speakers learning a new language; and version 7.0 for the main study according to students’ feedback on the pilot study (see Section 5.4.2).