Reading for a Test: The Effect of High Stakes Exams on Reading Strategies

Texto completo

(1)Monográfico II 2017. 103-115. Reading for a Test: The Effect of High-Stakes Exams on Reading Strategies Andrej Stopar Gašper Ilc. University of Ljubljana, Slovenia Received: 28 March 2017 / Accepted: 21 July 2017 ISSN: 1697-7467. ABSTRACT: This paper presents a study which investigates the reading strategies em-. ployed by Slovenian EFL students whose classroom context is influenced by a high-stakes, curriculum-aligned national examination. The findings show that there are only minor differences between strong and weak students in their use of reading strategies. The results are discussed in light of the washback effect and the development of the reading skill/strategies in the classroom. The discussion suggests that educators should be aware of the limitations of high-stakes examinations, and introduce reading tasks not directly linked to such examinations, in order to minimise their potential negative effect in the classroom. Keywords: reading strategies, national examination, washback effect, EFL reading, reading tasks. Lectura para una prueba: El efecto de los exámenes de alto riesgo en las estrategias de lectura RESUMEN: Este artículo presenta un estudio sobre las estrategias de lectura de los estudiantes eslovenos de ILE cuyo contexto del aula está influido por el examen nacional de alto impacto, añadido al currículo. Los resultados muestran que hay solo diferencias mínimas entre los estudiantes competentes y los incompetentes en el uso de las estrategias de lectura. Los resultados se discuten a la luz del efecto rebote y el desarrollo de la habilidad/las estrategias de lectura en clase. La discusión propone que los educadores deberían ser conscientes de las limitaciones de los exámenes de alto impacto e introducir tareas de lectura indirectamente relacionadas con tales exámenes para minimizar su potencial efecto negativo. Palabras clave: estrategias de lectura, examen nacional, efecto rebote, lectura en ILE, tareas de lectura.. 1. Introduction. and. Literature Overview. It is difficult not to agree with Carrell’s (1998: 1) observation that many EFL students find reading to be the most important among the four skills. For them, solid reading proficiency in English is a prerequisite for their future careers in the world of globalisation and digitalisation. It often seems, however, that both the learners and their instructors are slightly less familiar with the complexity of the cognitive processes involved while reading. Exhaustive studies in the field of reading and cognitive processing (Anderson, 1999;.

(2) Monográfico II. noviembre 2017. Cohen, 1994; Koda, 2005 among others) suggest that developing L2 reading skills is quite challenging, since L2 learners have to process the text and construct a meaning out of what they have read in their non-mother tongue. As a consequence, their reading comprehension performance may be impeded by factors that are not typically present in an L1 situation, for example: weaker linguistic skills, narrow vocabulary and different cultural backgrounds (Hong-Nam and Page, 2014: 196; also see Mijušković and Simović, 2016). To overcome these difficulties, L2 readers may resort to different reading techniques than those that are characteristic of L1 readers (Abbott, 2006). Investigating potential connections between reading abilities, metacognitive awareness, and reading strategies, Sheorey and Mokhtari’s (2001) study establishes that there is a direct correlation between low/high ability readers and their low/high metacognitive awareness, as well as their reading strategy. Drawing on these results, the same authors develop and test a special instrument, the Survey of Reading Strategies (SORS), for measuring the metacognitive awareness of L2 readers, in particular for determining “the type and frequency of reading strategies that adolescent and adult ESL students perceive they use while reading academic materials in English” (Mokhtari and Sheorey, 2002: 4). The SORS is comprised of three groups of reading strategies: (i) global reading strategies (different techniques to keep the purpose in mind, to manage the reading process, etc.); (ii) problem solving strategies (different actions/procedures used while reading, to improve comprehension, for example, adjusting the speed of reading, rereading, etc.); and (iii) support strategies (activities aimed at aiding the reader in comprehending the text, for example, taking notes, underlining, etc.). The complete list of these strategies is given in Appendix 1. Mokhtari and Sheorey (2002: 4-6) point out that the SORS is not only beneficial for L2 learners, but also for teachers, since it can be used as an instrument for identifying the reading needs, as well as making readers more conscientious about the reading process. In fact, the authors (2002: 6-8) accentuate the practical value of SORS, and the importance of teachers in raising learners’ metacognitive awareness, as well as introducing different reading techniques into their teaching practices, inevitably resulting in higher reading abilities among learners. 1 It is noteworthy that Sheorey and Mokhtari’s (2001) and Mokhtari and Sheorey’s (2002) recommendations pertain mostly to teaching reading comprehension. Little reference is made to language testing which has, however, become somewhat dominant and omnipresent in L2 classrooms, a situation triggered by an ever-increasing demand for statistically manageable results obtained from more or less uniform tests that are externally administered, evaluated and validated. Therefore, it is imperative that the question of a potential correlation between the reading strategy use and test performance be adequately addressed, especially in light of the well-reported examination effect on L2 teaching—the ‘washback effect’—i.e. the (positive or negative) impact of high-stakes tests on classroom practices (see, for example, Buck, 1988, and Ahmad and Rao, 2012). It seems, however, that whenever we try to connect reading strategies and testing, we enter a slippery slope. In his ground-breaking and statistically well-supported research on the relationship between the cognitive and metacognitive strategy use and L2 test performance, Purpura (1997) concludes that this relationship is extremely complex, with neither set of strategies bearing direct consequences on the test performance. Rather, it appears that metacognitive processing “exert[s] an executive function over” cognitive For instance, some positive effects of explicit metacognitive strategy instruction have been reported by Tavakoli and Koosha (2016). 1. 104.

(3) Andrej Stopar and Gašper Ilc. Reading for a Test: The Effect of High-Stakes Exams on Reading.... processing (Purpura, 1997: 314), and that the two work in unison. Moreover, Purpura (1997: 315) claims that the test-taking style also influences the test performance: in speeded L2 testing, product-oriented strategies (e.g., using the text only to retrieve information) may increase performance, whereas process-oriented strategies (e.g., using the text to learn) may hamper it. Last but not least, the extent to which a particular strategy is advantageous in a test-taking situation is also determined by the task-type (ibid.). In his book on assessing reading, Alderson (2000: 307) clearly points out that tests cannot be taken as good measurements for reading strategies, mainly because (i) the testtaking process is a stressful situation for test-takers, so they may resort to different reading strategies than they would in a real-life situation, and (ii) the frame of a test is rigid, limiting evaluators to rate responses according to a given scale. In addition, the selection of a reading strategy is, to some extent, subjective: readers should be able and, indeed, allowed to select a strategy that they find useful, rather than being forced to apply a strategy targeted by item-writers/test developers (ibid.). Another potential problem in linking reading strategies with testing is the fact that L2 reading comprehension tests tend to focus more on lower level cues, such as scanning for details and identifying key vocabulary items, which consequently also require less complex, mostly data-/detail-driven reading strategies (Abbott, 2006; Carrell, 1983; Purpura, 1997). As a consequence, a proficient test-taker cannot display the real use of different strategies. Given now the fact that reading strategies cannot be measured effectively and exhaustively in a formal test format and testing situation, a question arises as to what happens in an L2 classroom that is influenced by external examination and its potential washback effect? In particular, do L2 teachers try to minimise this effect and pay more attention to stimulating the use of different reading strategies, which will eventually turn their students into proficient readers, or do they tailor their reading activities to strategies which help their students become good test-takers, but not necessarily proficient readers? In other words, are L2 teachers teaching reading strategies or test-taking strategies? In this paper, we hypothesise that a student’s level of proficiency influences their selection of reading strategies2 when completing various reading tasks. We test the hypothesis on a sample of secondary school students from Slovenia, who have to take the external national school-leaving examination. The study participants were asked to complete three reading comprehension tasks in order to determine their level of proficiency, and the SORS questionnaire to establish which reading strategies they most frequently pursue in general. The results are discussed with regard to types of reading strategies, the frequency of their use, and the differences between strong and weak performers as measured by the reading test employed in the study. Finally, some practical implications are presented.. 2. The Study 2.1. Participants and Context The participants in the study were 115 secondary school students, with Slovenian as Different profiles of strategy use among different types of readers have been previously attested in the literature— see, for instance, Chevalier, Parrila and Ritchie (2017). 2. 105.

(4) Monográfico II. noviembre 2017. their native language. When the experiment was conducted, they were in the final year of their studies, i.e. 17-19 years old. Their expected language proficiency levels ranged from CEFR B1 to B2, which describes the so-called ‘independent users’ at the intermediate and upper intermediate levels and is in accordance with the nationally-prescribed curricular standards (for more on the Common European Framework of Reference for Languages see Council of Europe, 2001). On completing their final year of secondary education, they were to sit the national school-leaving examination, which also serves as an entrance examination for tertiary education, and is thus a high-stakes exam. The sample included six groups of students of approximately equal sizes, from six secondary schools. 2.2. Instruments and Procedure The reading subtest of the General Matura (GM) in English was the basis for the study. The GM in English is a national (Slovenian) school-leaving examination that consists of subtests on reading comprehension (20%), language use (15%), listening comprehension (15%), written communication (two parts, 10% and 20%), and oral communication (20%). The examination is conducted in English. According to the examination specifications, the reading tasks require the task-taker to understand the gist, the main ideas and specific information in various text-types. In the present study, the respondents were asked to complete three tasks from the reading subtest of the GM: a 10-item short answers task, a 10-item gapped sentences task and an 8-item multiple choice questions task; each answer scores one point (see Appendix 2 for examples). In line with Freedle and Kostin’s (1999) classification of reading items, the 28 items can be categorised as targeting explicit details and gist (14 items), text structure and organisation (10 items), and implicit details and gist (4 items). The tasks are based on newspaper articles from a British quality daily newspaper. As the GM is a high-stakes examination, the tasks have been previously administered to thousands of candidates, and have been proven reliable and valid. Additionally, the same tasks have been aligned with the CEFR level B2, with the cut score of 75 percent. The alignment has also been externally validated (Ilc and Stopar, 2015), and the test has also been studied with regard to item and task difficulty (Stopar and Ilc, 2016). The cut score for B1 was not set during the alignment project; therefore, we label the participants below the B2 cut score as ‘sub-B2’ readers. In order to eliminate the affective factors that may influence the test results (see Alderson, 2000 above), the participants were assured that the test results would bear no high-stake consequences for them. The test was administered in accordance with the test specifications (e.g., time limits). After completing the three reading tasks, the respondents were asked to fill out the SORS questionnaire (Mokhtari and Sheorey, 2002) based on their approaches to reading in general. The SORS was opted for over Purpura’s (1997) system of cognitive and metacognitive processing, because it exclusively focuses on reading strategies. For the purpose of the study, the questionnaire was translated into Slovenian. The thirty statements were comprised of thirteen pertaining to global reading strategies, eight to problem solving strategies, and nine to support strategies (cf. Mokhtari and Sheorey, 2002, and Appendix 1).. 106.

(5) Andrej Stopar and Gašper Ilc. Reading for a Test: The Effect of High-Stakes Exams on Reading.... The respondents were asked to indicate how often they use the strategies described by the statements, using the 5-point Likert scale. The results were collected in an Excel file and statistically analysed. The analysis of the reading section relied on simple descriptive statistics: the results were calculated for individual students (mean score) and individual reading items (difficulty and discrimination indices). For the SORS questionnaire, the 1-to-5 Likert scale was analysed, in line with Oxford and Burry-Stock (1995), where the mean of 3.50 or higher counts as high strategy use, a mean in-between 2.50 and 3.40 as moderate, and a mean at or below 2.40 as low. A paired two-sample t-test analysis was employed to compare the data collected from the two groups of students. Obviously, recall bias represents a limitation to the study. 2.3. Research Questions RQ1: Which reading strategies are perceived as useful by the participants? RQ2: Are there any differences in the use of strategies between strong and weak performers (B2 and sub-B2 students)? RQ3: Which are the most/the least frequently used reading strategies? 2.4. Results 2.4.1. The Reading Test The mean facility value (i.e. the proportion of students who answered the reading items correctly) for the twenty-eight reading items is 0.69, with the difficulty of individual items ranging from 0.37 to 0.95. The mean discrimination index (i.e. the index showing the differences between strong performers and weak performers) is at 0.54, with all the items in the positive range. According to the test-provider (cf. Section 2.1.3 of the 2015 Annual Report on the General Matura, published by the National Examinations Centre, 2015), such statistics indicate that the test is of moderate difficulty, and that it consistently shows the differences between strong and weak performers. A comparison of the mean facility values for individual reading items obtained in the present study, and the facility values obtained when the tasks were administered at the national level, shows a high correlation of 0.79, indicating the validity of our sample. The CEFR B2 cut-score of 75 percent was achieved by 59 (51.30%) participants in our study; their mean facility value was 0.88 (24.6 out of a maximum score of 28). In contrast, 56 participants (48.70%) were identified as sub-B2 learners. The mean facility value observed for this group was 0.49 (13.7 out of a maximum score of 28).. 107.

(6) Monográfico II. noviembre 2017. 2.4.2. The Reading Strategies Table 1 presents the results pertaining to the use of reading strategies. Table 1. The test performance and the use of reading strategies by all/strong/weak students. Mean facility value All participants (N=115, 100%) B2 participants (N=59, 51.30%) Sub-B2 participants (N=56, 48.70%). 0.69. 0.88. 0.49. All strategies. Global strategies. Problem solving strategies. Support strategies. Moderate. Moderate. High. Moderate. (3.29). (3.35). (3.74). (2.80). Moderate. Moderate. High. Moderate. (3.29). (3.39). (3.83). (2.65). Moderate. Moderate. High. Moderate. (3.30). (3.31). (3.66). (2.95). The participants in our study perceive themselves to be moderate users of reading strategies, although it is noteworthy that the overall mean of 3.29 on the 1-to-5 Likert scale is close to the mean of 3.50, which is interpreted by Oxford and Burry-Stock (1995) as indicative of the beginning of the high range. The most frequently used reading strategies are problem solving ones (falling into the high range, with the mean result of 3.74), followed by global strategies and support strategies in the moderate range (the means of 3.35 and 2.80, respectively). The respondents claim that, when reading, they rely on having a purpose in mind, guessing the content, and thinking about what they already know about the topic (top three global strategies, with the mean scores in-between 3.99 and 4.10). The readers also commonly use the strategies of re-reading, paying closer attention when the text becomes difficult, and adjusting the speed of reading (the top three problem solving strategies, with the mean scores between 3.96 and 4.03). The most popular support strategies are: thinking about the text in both English and the mother-tongue, reading aloud, and going back and forth in the text, to identify relationships between ideas (their means are between 2.91 and 3.57). Among the least-used reading strategies are: taking notes, using reference materials, and paraphrasing (the mean scores of these support strategies are between 1.95 and 2.64). Rather rarely reported are also the global strategies that involve critical analysis and information evaluation, checking to see whether the guesses about the text are correct or not, and examining the length and the structure of the text (their mean scores are between 2.62 and 2.87). Problem solving strategies are more commonly used—the one reported as the least frequently used has the mean of 3.1—and it involves stopping from time to time, to think about what is being read.. 108.

(7) Andrej Stopar and Gašper Ilc. Reading for a Test: The Effect of High-Stakes Exams on Reading.... 2.4.3. Strong and Weak Performers A comparison of strong and weak performers reveals very similar mean scores for the overall use of reading strategies (3.66 for the B2 group and 3.46 for the sub-B2 group, both in the high range). The t-test analysis shows that overall the difference in means is not statistically significant (p<0.05); the same finding applies if the t-test analysis is conducted separately for each of the three subgroups of strategies. Moreover, the ranges for different types of strategies are identical: high for problem solving strategies, and moderate for global and support strategies. An analysis of the mean numbers pertaining to individual statements supports the observed similarities. Taking into account the Likert-scale ranges proposed by Oxford and Burry-Stock (1995), the results for the strong and the weak students match almost perfectly. Both groups exhibit the same thirteen statements in the high range, twelve statements in the moderate range, and one in the low range. However, there are also some noticeable differences between the two groups. The strong and weak performers differ most significantly in their use of support and problem solving strategies, whereas the distance between the mean scores of B2 and sub-B2 readers for the global strategies is not so pronounced. Also, support strategies are the only group of strategies that is selected more frequently by the sub-B2 readers than the B2 readers. With regard to individual statements, only in four cases can some minor differences in ranges be detected. The differences show that the strong performers are more likely to visualise (a problem solving strategy) and critically analyse or evaluate the text (a global strategy), while the weak group may be more likely to resort to translation into their native language and note-taking (support strategies). Figure 1 presents the differences between the strong and the weak performers, with regard to each statement and the three main groups of strategies. Figure 1. The differences between B2 readers and sub-B2 readers by individual statements and types of strategies.. 109.

(8) Monográfico II. noviembre 2017. 3. Discussion The analysis of our sample has shown that the participants in this study are weak to moderate users of the reading strategies listed in the SORS questionnaire. The extent of their reliance on global reading strategies suggests a holistic, broad approach to reading, whereby the readers focus on the purpose of their reading by guessing the content, relying on their own knowledge, and considering the context. At the same time, the readers also exhibit a strong awareness of the fact that the (exam-like) tasks require careful reading: they rely on the problem solving strategies of re-reading, paying attention to demanding sections, adjusting the speed of reading, and staying focused. Especially significant is the finding on re-reading, since it ranks as the most frequently used problem solving strategy, and the second mostused reading strategy overall, which contradicts the established view that students consider re-reading to be overly time-consuming (Garner and Alexander, as reported in Mokhtari and Reichard, 2002: 255). Furthermore, the statements pertaining to support strategies show that skills such as note-taking, paraphrasing, using reference materials, asking yourself questions about the text, translating, and underlining are not employed very frequently. The participants in our study demonstrated different levels of L2 reading comprehension proficiency. Taking advantage of reading tasks that had been previously aligned with CEFR B2, we were able to divide them into groups of B2 and sub-B2 readers, and observe what reading strategies they use when reading. In line with Sheorey and Mokhtari’s (2001) claims about a clear link between reading ability and metacognitive awareness/reading strategies, we predicted that proficient readers would report using a different array of reading strategies than their less proficient peers. However, this prediction was not borne out: both groups of readers reported using virtually identical reading strategies. We interpret this to be a likely consequence of the washback effect, and of the constraints pertaining to the standard task formats found in reading examinations. This is not a mere speculation since a survey conducted by the Slovenian National Examination Centre among primary school teachers shows that because of the external high-stakes examination, 35% of teachers modified their teaching methods, 40% introduced the same task-types in their tests, 46% provided exam-focused practice, and as many as 82% integrated the past test papers into their teaching practice (National Examinations Centre, 2012). No such data is available for secondary schools but we strongly believe that the situation in secondary schools is comparable, and similar results have also been reported from other countries (e.g., Ahmad and Rao, 2012). Our claim is also supported by the fact the format of the GM is extremely rigid and predictable, which makes it possible for the exam-takers and the educators to practice and drill a limited number of reading comprehensions tasks based on a fixed array of text-types rather than to focus on a variety of different task- and text-types. Such a conclusion also suggests that Alderson’s (2000) claim about tests not being good measurements for reading strategies should be extended to include any reading exercise, the format of which, be it the task-type or the text-type, is identical or similar to the ones commonly used in high-stake examinations. The situation is further aggravated by the fact that the textbooks intermittently selected by the educators abound in reading comprehension tasks that are very similar to those found in high-stakes examinations. Hence, the question arises as to how effective the teaching of reading is when it is based on tasks that require any reader—without regard to their reading proficiency—to complete repetitive and predictable reading comprehension tasks by resorting to a uniform set of reading strategies.. 110.

(9) Andrej Stopar and Gašper Ilc. Reading for a Test: The Effect of High-Stakes Exams on Reading.... Despite these overall findings, which demonstrate a very uniform approach to completing reading tasks, some subtle variation between the strong and the weak performers can still be observed. Generally, the support strategies were more frequently chosen by the weaker group, most noticeably with respect to taking notes, underlining, and translating into the reader’s native tongue. If we combine this finding with the preference of the B2 group to visualise and critically analyse or evaluate the text—a problem solving strategy and a global strategy, respectively—we can conclude that some tendencies nevertheless exist among the readers, to use a set of strategies that best fits their proficiency level. We firmly believe that the results of our study have direct practical implications for EFL educators. We maintain that the role of teachers is central for the successful development of various reading strategies as suggested by Sheorey and Mokhtari (2001). To avoid negative consequences of resorting to the same standardised reading comprehension tasks resembling those found in the high-stakes examinations—which in our opinion leads to the exam’s potential negative washback effect—EFL educators should promote the use of a rich and varied array of reading comprehension task-types. First, starting with the reading skill only, we propose that EFL educators should incorporate into their teaching practice a larger number of open-ended reading comprehension tasks that stimulate critical, interpretative, extensive, more process-oriented and less data-/ detail-driven reading. In addition, they should stimulate extensive reading and reading longer texts (e.g., unabridged/authentic literary texts and semi-academic texts) involving a longer period of time, which would enable students to develop tasks that focus on a particular reading strategy (e.g., finding a connection between different parts of the text, note taking, using reference material and relying on visual support for better comprehension). All of these tasks require the reader to understand the text beyond the sentence/paragraph level, and make intra- and inter-textual connections (e.g., identifying relevant information, forming an overall view of the text). Coincidently, such tasks are also considered taxonomically and cognitively more complex (Khalifa and Weir, 2009). Second, the application of different reading strategies can be developed by introducing open-ended integrated tasks involving at least two different skills, for example, reading-writing and reading-speaking combinations, which reportedly have a positive overall effect on the general reading comprehension skill. Examples of such tasks include written or spoken text summaries, oral or written text simplifications/reductions, L2-L1 translations, and in-depth post-reading discussions/debates. Apart from developing reading ability, all of these involve the application of other skills, including the use of reference material and secondary literature (dictionaries, thesauri, on-line sources), register awareness (spoken/written vs. formal/informal language), critical evaluation of different authors/sources, language production skills (paraphrasing, reformulating, expression of opinion), presentation skills, etc. All of these are present in our everyday real-life reading situations, so highlighting their importance in the EFL context will help students to understand the imperative behind developing the reading skill.. 4. Conclusion The participants in the study have been revealed as weak to moderate users of the SORS reading strategies. Although they demonstrated different levels of L2 reading comprehension proficiency, both the strong and the weak respondents reported using almost identical reading. 111.

(10) Monográfico II. noviembre 2017. strategies. We attribute this finding to the washback effect and the task formats frequently included in reading examinations. We find these results to be meaningful and relevant for the classroom in the 21 st century. Nevertheless, we also believe that any future studies of reading strategies should be extended to cover types of strategies not (yet) included in the SORS questionnaire, e.g. technologybased reading strategies. Any future research should also examine students at other levels of proficiency and their reading strategies in non-institutional contexts. Finally, the results lead us to believe that the focus of the EFL classroom activities should not be solely on preparing the students to successfully pass a reading test, but rather on developing the cognitive abilities and skills that lead them to become proficient readers. It should not go unnoticed that the ability to read proficiently, by applying different reading strategies, will stay with the reader well beyond passing the examination, however high the stakes may be.. 5. References Abbott, M. L. 2006. “ESL reading strategies: Differences in Arabic and Mandarin speaker test performance”, in Language Learning 56, 4: 633-70. DOI:10.1111/j.1467-9922.2006.00391.x. Ahmad, S., & Rao, C. 2012. “Examination Washback Effect: Syllabus, Teaching Methodology and the Learners’ Communicative Competence”, in Journal of Education and Practice 3, 15: 173-83. Alderson, J. C. 2000. Assessing Reading. Cambridge: Cambridge University Press. Anderson, N. J. 1999. Exploring Second Language Reading: Issues and Strategies. Boston: Heinle & Heinle. Buck, G. 1988. “Testing listening comprehension in Japanese university entrance examinations”, in JALT Journal 10, 1/2: 15-42, available from: jj-10.1-art1.pdf, accessed 10 April, 2016. Carrell, P. L. 1983. “Some issues in Studying the Role of Schemata, or Background Knowledge in Second Language Comprehension”, in Reading in a Foreign Language 1/2: 81-92. Carrell, P. L., Devine J., & Eskey, D. E. 1998. Interactive Approaches to Second Language Reading. Cambridge: Cambridge University Press. Chevalier, T. M., Parrila, R., & Ritchie, K. C. 2017. “The Role of Metacognitive Reading Strategies, Metacognitive Study and Learning Strategies, and Behavioural Study and Learning Strategies in Predicting Academic Success in Students With and Without a History of Reading Difficulties”, Journal of Learning Disabilities 50, 1: 34-48. DOI: 10.1177/0022219415588850. Cohen, A. D. 1994. Assessing Language Ability in the Classroom. Boston: Heinle & Heinle. Council of Europe. 2001. Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press. Available from: http://, accessed 3 April, 2016. Hong-Nam, K., & Page, L. 2014. “Investigating metacognitive awareness and reading strategy use of EFL Korean university students”, in Reading Psychology 35, 3: 195-220. DOI:10 .1080/02702711.2012.675418. Freedle, R., & Kostin, I. 1999. “Does the text matter in a multiple-choice test of comprehension? The case for the construct validity of TOEFL’s minitalks”, in Language Testing 16, 2: 2-32. DOI:10.1177/026553229901600102.. 112.

(11) Andrej Stopar and Gašper Ilc. Reading for a Test: The Effect of High-Stakes Exams on Reading.... Ilc, G., & Stopar, A. 2015. “Validating the Slovenian national alignment to CEFR: The case of the B2 reading comprehension examination in English”, in Language Testing 32, 4: 44362. DOI:10.1177/0265532214562098. Khalifa, H., & Weir, C. 2009. Examining reading: Research and practice in assessing second language learning. Cambridge: Cambridge University Press. Koda, K. 2005. Insights into second language reading: A cross-linguistics approach. Cambridge: Cambridge University Press. Mijušković, M., & Simović, S. 2016. “The 21st Century English Language Reading Classroom in Montenegro: the Influence of Metacognitive Strategies on University Students’ Attitudes Regarding the Process of Reading in English”, in Porta Linguarum 26, 23-36, available from: pdf, accessed 13 February, 2017. Mokhtari, K., & Reichard, C. A. 2002. “Assessing Students’ Metacognitive Awareness of Reading Strategies”, in Journal of Educational Psychology 94, 2: 249-59. DOI:10.1037//00220663.94.2.249. Mokhtari, K., & Sheorey, R. 2002. “Measuring ESL students’ awareness of reading strategies”, in Journal of Developmental Education 25, 3: 2-10. National Examinations Centre. 2012. Vpliv nacionalnega preverjanja znanja na delo v razredu. Analiza vprašalnika za učitelje angleščine na osnovnih šolah [The effect of national assessment on classroom practices. Analysis of questionnaire for primary school English teachers.]. Ljubljana: Državni izpitni center. Available from: 2012%20analiza%20ang/2012111310480252/, accessed 10 April, 2016. National Examinations Centre. 2015. Letno poročilo—splošna matura 2015 [Annual Report— General Matura 2015]. Ljubljana: Državni izpitni center. Available from: http://www.ric. si/mma/Letno%20porocilo%20SM%202015/2015113014194261/, accessed 22 April, 2016. Oxford, R. L., & Burry-Stock, J. A. 1995. “Assessing the use of language learning strategies worldwide with the ESL/EFL version of the strategy inventory for language learning (SILL)”, in System 23, 1: 1-23. DOI:10.1016/0346-251X(94)00047-A. Purpura, J. E. 1997. “An Analysis of the Relationships between Test Takers’ Cognitive and Metacognitive Strategy Use and Second Language Test Performance”, in Language Learning 47, 2: 289-325. DOI:10.1111/0023-8333.91997009. Sheorey, R., & Mohktari, K. 2001. “Differences in the metacognitive awareness of reading strategies among native and non-native readers”, in System 29: 431-49. DOI:10.1016/ S0346-251X(01)00039-2. Stopar, A., & Ilc, G. 2016. “Item and task difficulty in a B2 reading examination: perceptions of test-takers and CEFR alignment experts compared with psychometric measurements.”, in Círculo de Linguística Aplicada a la Comunicacioń 67: 318-342. DOI:10.5209/CLAC.53487. Tavakoli, H., & Koosha, M. 2016. “The Effect of Explicit Metacognitive Strategy Instruction on Reading Comprehension and Self-Efficacy Beliefs: The Case of Iranian University EFL Students.”, in Porta Linguarum 25, 119-133, available from: articulos/PL_numero25/9%20Hossein.pdf, accessed 10 July, 2017.. 113.

(12) Monográfico II. 1. Appendix 1: Mokhtari’s. noviembre 2017 and. Sheorey’s (2002) Strategies. Abbreviations: S#: statement number; G: a global reading strategy; S: a support strategy; P: a problem solving strategy. S1-(G): I have a purpose in mind when I read. S2-(S): I take notes while reading to help me understand what I read. S3-(G): I think about what I know to help me understand what I read. S4-(G): I take an overall view of the text to see what it is about before reading it. S5-(S): When the text becomes difficult, I read aloud to help me understand what I read. S6-(G): I think about whether the content of the text fits my reading purpose. S7-(P): I read slowly and carefully to make sure I understand what I am reading. S8-(G): I review the text first by noting its characteristics like length and organization. S9-(P): I try to get back on track when I lose concentration. S10-(S): I underline or circle information in the text to help me remember it. S11-(P): I adjust my reading speed according to what I am reading. S12-(G): When reading, I decide what to read closely and what to ignore. S13-(S): I use reference materials (e.g. a dictionary) to help me understand what I read. S14-(P): When text become difficult, I pay closer attention to what I am reading. S15-(G): I use tables, figures, and pictures in text to increase my understanding. S16-(P): I stop from time to time and think about what I am reading. S17-(G): I use context clues to help me better understand what I am reading. S18-(S): I paraphrase (restate ideas in my own words) to better understand what I read. S19-(P): I try to picture or visualize information to help remember what I read. S20-(G): I use typographical features like bold face and italics to identify key information. S21-(G): I critically analyse and evaluate the information presented in the text. S22-(S): I go back and forth in the text to find relationships among ideas in it. S23-(G): I check my understanding when I come across new information. S24-(G): I try to guess what the content of the text is about when I read. S25-(P): When text becomes difficult, I re-read it to increase my understanding. S26-(S): I ask myself questions I like to have answered in the text. S27-(G): I check to see if my guesses about the text are right or wrong. S28-(P): When I read, I guess the meaning of unknown words or phrases. S29-(S): When reading, I translate from English into my native language. S30-(S): When reading, I think about information in both English and my mother tongue.. 114.

(13) Andrej Stopar and Gašper Ilc. 2. Appendix 2: Examples. Reading for a Test: The Effect of High-Stakes Exams on Reading... of the. Reading Subtest Tasks. Example 1 (Short Answers): How does she communicate? Answer (open ended): Using gestures. Example 2 (Gapped Text): … Since I was born and grew up in Venice, (1 ____). I have seen … Answer (one of 12 options): G) I feel very much part of the place Example 3 (Multiple Choice Questions): The author managed the escalator with the help of: A a saintly woman. B a pair of crutches C his new flying ability D his friendly nephew. Answer (one option): B. 115.



Table  1  presents  the  results  pertaining  to  the  use  of  reading  strategies.

Table 1

presents the results pertaining to the use of reading strategies. p.6
Figure  1  presents  the  differences  between  the  strong  and  the  weak  performers,  with  regard  to  each  statement  and  the  three  main  groups  of  strategies.

Figure 1

presents the differences between the strong and the weak performers, with regard to each statement and the three main groups of strategies. p.7