Entrevistas a expertas

• a. check to make sure that I understand what I am reading. • b. mentally repeat the sentence to myself.

• c. underline the sentence.

• d. try to focus on only the current sentence. After I read, it is a good idea to:

• a. try to figure out which sections I didn’t understand. • b. count how many words I didn’t understand in the chapter. • c. try to remember the table of contents.

• d. memorize the chapter review questions.

Table 5.3 Examples of Statements in the Reading Strategy Checklist

• I skip sections that are too difficult for me. • I have a hard time concentrating while I read.

• I often don’t read the textbook because I don’t have time.

• I think about how the pictures relate to the information in the chapter. • I check to see how many pages are in the chapter that I am going to read. • I read the chapter one time, and I don’t usually need to read it again. • I think about how ideas relate to each other in the chapter.

• I stop to review the main points to see if I understand what I’ve read so far.

* The results of this study were reported in O’Reilly and McNamara (2002, 2007). However, the majority of the results that we report here were not included in the final versions of those papers.

The correlation between the checklist measure and performance on the Gates Reading Comprehension Test and a measure of prior science knowledge were also very low (i.e., −0.06 and −0.03, respectively). In contrast, the MSI showed correlations of 0.30 with reading skill and 0.28 with prior science knowledge.

Table 5.4 further shows the correlations of reading skill (as measured by the Gates), prior knowledge of science, MSI, and the checklist with measures of student performance, including course grades, performance on the Virginia Standards of Learning test for biol- ogy, and an experimenter-created measure of text comprehension that included open- ended and multiple-choice questions. The results indicated that the Reading Strategy Checklist did not correlate with any measures of student performance. The MSI showed relatively low correlations, particularly as compared with the measures of reading skill and prior knowledge. Thus, the results for both the MSI and the checklist, in terms of their ability to predict student performance in science, were largely disappointing. In addition, while reliability was acceptable for the Reading Strategy Checklist, its construct validity was very low. As a consequence, we abandoned the checklist, and did not use it in any further studies.

Subsequent results in further studies that included the MSI have been largely disappoint- ing. For example, another study with 1025 high-school students from two schools in Virginia and one school in Kentucky, yielded similar results to the previous one. The MSI correlated 0.37 and 0.32 with reading skill and prior knowledge and correlated 0.20 with reading comprehension tests. In most studies where the MSI was somewhat predictive of the effectiveness of training, the Gates reading comprehension test was usually a better predictor and usually showed higher correlations with comprehension performance measures.

In his thesis project, Perry (2008) provided additional evidence that the MSI is poorly correlated with comprehension and strategy use. He assessed the extent to which aspects of the reader and reading strategies were correlated with comprehension and the extent to which readers’ attributes were correlated with the use of strategies. He measured working memory capacity, metacognitive awareness (with the MSI), need for cognition, vocabulary knowledge, and exposure to texts, in a sample of college students. Participants read a series of texts on the computer. Perry assessed the extent to which participants para- phrased, bridged, and elaborated while typing verbal protocols under think aloud instruc- tions. Comprehension for these texts was assessed via online short-answer comprehension questions. These questions were primarily why-questions regarding the sentence that was just read. Perry conducted a series of regression analyses to assess the relationship between the attributes of the reader and reading strategies with comprehension. With respect to readers’ attributes, only vocabulary knowledge and working memory span were significant Table 5.4 Correlations of Reading Skill (as Measured by the Gates), Prior Science Knowledge, MSI,

and Reading Strategy

Individual measure Course grade VA Standards of Learning test Open-ended comprehension Multiple-choice comprehension Reading skill 0.24 0.58 0.64 0.53 Science knowledge 0.25 0.59 0.55 0.51 MSI 0.20 0.15 0.26 0.24

Reading strategy checklist* −0.13 −0.07 −0.05 −0.03

* The reading strategy checklist consisted of measures of student performance, including course grades, perform- ance on the Virginia Standards of Learning test for biology, and an experimenter-created measure of text comprehension that included open-ended and multiple-choice questions

positive predictors of comprehension performance, whereas the extent to which readers paraphrased, bridged, and elaborated were all significant and positive predictors of com- prehension. Moreover, reading strategies accounted for 22% of the unique variance in comprehension performance, whereas readers’ attributes accounted for only 9%. Appar- ently, the successful use of reading strategies is more predictive of comprehension than the attributes that the reader brings into the reading situation. Moreover, these findings are consistent with the research discussed earlier showing that the MSI is not strongly correl- ated with comprehension performance. Indeed, Perry (2008) found that MSI was not significantly correlated with any of the strategies. Although the Need for Cognition scale was positively and significantly correlated (r = 0.18) with the extent that the participants produced bridging inferences. The only measure that was consistently correlated of strat- egy use was vocabulary knowledge (r = 0.47 for paraphrasing, r = 0.24 for bridging, and r = 0.27 for elaboration).

There has been only one exception to the trend that the MSI is poorly correlated with strategy use, comprehension, and response to self-explanation training. The study con- ducted by McNamara et al. (2006) with middle-school students discussed earlier also investigated the effectiveness of iSTART (i.e., the automated version of SERT). The results showed that the effectiveness of training on their ability to comprehend science text depended on both their strategy knowledge (as indicated by the MSI), as well as the type of comprehension question. Among students who performed less well on the MSI (low strat- egy knowledge), those given iSTART training performed substantially better than the control participants, but only on the text-based questions (Cohen’s d = 1.00). In contrast, among the high-strategy knowledge students, those given iSTART training performed better on the bridging-inference questions (Cohen’s d = 1.04). As such, this study indicated that the MSI was not indicative of who would benefit from training, but rather how they would benefit from training: The low-strategy knowledge students improved in terms of text-based comprehension (for that text), whereas the high-strategy knowledge students improved at the level of the situation model. These results have been replicated by Magliano et al. (2005), but using a measure of reading skill (the Nelson–Denny Reading Test), rather than with a measure of strategy knowledge, or metacognition.

By and large, the results of the majority of the studies we have conducted with the MSI have indicated that it correlates moderately with comprehension and may be predictive of training gains with younger students. However, for the most part, the results have been largely disappointing. Likewise, we have had disappointing results using the the Metacog- nitive Awareness of Reading Strategies Inventory (MARSI; Mokhtari & Reichard, 2002).

Despite our limited success using standardized measures of metacognitive awareness to assess the relationship between reading strategies and comprehension, we believe that metacognitive awareness is critical for the successful use of comprehension strategies. One basis of this belief stems from the dynamic nature of self-explanation and thinking aloud. As we will describe, readers alter their use of strategies based on the demands of the texts, their goals, and their comprehension failures and success. Although we do not have strong evidence yet, we believe that a sense of metacognitive awareness enables readers to modu- late their strategy use. Our goal in the following section is to present evidence that this relationship warrants further investigation.

The Dynamic Nature of Self-Explanation and Reading Strategies

Self-explanation, by its very nature, calls on the reader to assess comprehension, as it relates to a host of factors, including what the reader knows about the information in the text, the difficulty of the text, the importance of the text, the reader’s comprehension goals,

and the reader’s awareness of the strategies that can be used to meet those goals. Thus, self- explanation involves dynamically changing processes by which readers engage in whatever strategies are possible for that reader and deemed important for constructing meaning at any given point in a text. For example, explanations of narrative text are most appropriate when the text describes a causal process and the reader is asked to understand an event that is part of that process. However, not all texts or sentences describe events that exist on a causal chain. For example, when readers generate a self-explanation for a topic sentence, it may be most appropriate to use the content of the sentence to anticipate future text content. Additionally, generating explanations may not be feasible or appropriate given a reader’s level of comprehension. If, for example, a sentence contains unfamiliar words, the most appropriate strategy may be to rephrase it using more familiar words (i.e., paraphrasing) as opposed to generating knowledge-based inferences.

Hence, in our research, we have typically described self-explanations in terms of their relationship and appropriateness to the target text, individual sentences within the text, and the reader’s level of comprehension, rather than their relationship to higher-level processes (e.g., explanation, reasoning, problem solving). In turn, the relation between the text and the protocol typically corresponds to metacognitive processes or reading strategies that are likely being engaged by the reader. The nature of this process will be heavily influenced by the extent that the reader is aware of and applies strategies that are appropriate to the texts, individual sentences, and the reader’s comprehension goals.

Our ultimate goal is to better understand comprehension processes by examining the traces of these processes in protocols that are generated in the context of tasks such as self-explanation and think aloud. For example, “I don’t understand this” expresses the reader’s lack of understanding, but also indicates that the reader is engaging in com- prehension monitoring. The third example in Table 5.1 focuses on each of the two sen- tences in the target text separately and rephrases each of the sentences into the reader’s own words. From this, we can characterize the protocol as consisting of paraphrases; and in turn, we can infer that the reader is focusing on the text and is not generating inferences concerning the relations between concepts expressed in the text or relations between the text and what the reader already knows. Hence, the reader is primarily engaging in text- based processing, in contrast to building a deeper understanding of the text (Coté & Goldman, 1999). This contrasts with Examples 5 and 6, where the reader expresses the relationship between the two sentences, which requires making inferences that bridge or link the sentences.

Analyses of self-explanations and think-aloud protocols provide some indication of what strategies readers tend to use while reading, and which strategies are used in combin- ation, and which are not. For example, McNamara (2004) examined the frequency of strategies used by adult readers who did and did not receive prior training to self-explain while self-explaining science text. McNamara (2004) found that the most predominant strategy used by participants was paraphrasing. Accurate paraphrases were evident in 76% of the explanations and incorrect paraphrases were present in 13% of the explan- ations. The next most likely strategy was making bridging inferences linking the target sentence to a prior sentence: Correct bridging inferences were found in 38% and incorrect bridging inferences were present in 11% of the explanations. The least likely strategy to occur was prediction. This result replicated findings reported by Magliano et al. (1993) who found evidence using a lexical decision task that readers generate few predictions during the normal course of reading. Likewise, Magliano, Trabasso, et al. (1999) found that when readers used think aloud with narrative text, predictions were unlikely unless the reader was instructed to make predictions.

Magliano and his colleagues have found that college students produce knowledge-based explanations more frequently than any other strategy when reading simple narrative texts (Magliano, Trabasso, et al., 1999; Trabasso & Magliano, 1996a, 1996b). These results contrast with those reported by McNamara (2004), presumably because some kinds of elaboration may be more prevalent for narrative texts than for science texts. Differences as a function of genres such as narrative and science likely occur because of differences in the amount of relevant background knowledge possessed by the participants. In our studies with college students reading science texts, they typically do not possess extensive back- ground knowledge associated with the topics described in the text, whereas these students have greater knowledge of the types of social situations that arise in the stereotypical narratives that are used in our research. Thus, strategy use is likely to depend on the reader’s abilities as well as the type of text.

We have also found systematic relationships between the co-occurrence of different strategies (McNamara, 2004; Todaro, Magliano, Millis, Kurby, & McNamara, 2009). McNamara (2004) found that paraphrases and bridging inferences were likely to co-occur. Indeed, 84% of the self-explanations containing bridging inferences also contained para- phrases. In contrast, strategies that went beyond the text-based information, such as elab- oration and prediction, were less likely to co-occur with paraphrasing. Similarly, Perry (2008) found moderate but significant correlations between the extent that college stu- dents generated paraphrases and bridging inferences (r = 0.32) when producing verbal protocols under think-aloud instructions. However, paraphrasing was not correlated with elaboration. Both Perry and McNamara also found that there was no significant correlation between readers’ generation of bridging inferences and elaborations.

Paraphrasing may be a highly prevalent strategy and co-occur with bridging inferences because it serves as an “anchor” for these inferences when producing verbal protocols. That is, when reading to explain how the current sentence is related to the prior discourse, readers tend to paraphrase the current sentence to form a foundation. As will be discussed below, the lack of a strong relationship between bridging and elaboration may be because the features of the text that afford these inferences tend not to co-occur (Magliano, Zwaan, & Graesser, 1999; Magliano, Trabasso, et al., 1999).

Magliano, Trabasso, et al. (1999) provided evidence for a dynamic perspective of read- ing strategies in the context of a think-aloud task. They asked college students to think aloud after every sentence while reading simple narrative texts. Discourse analyses of the texts were conducted to identify factors that were likely to influence the probability that readers generated explanations based on the prior texts or world knowledge. They identi- fied two factors that were related to the extent to which readers generated explanations based on prior text content or world knowledge. One important factor was the presence or absence of causal antecedents in the text. The importance of causal antecedents in texts follows from research showing that relationships inferred by a reader are constrained by what is afforded by the text (Trabasso et al., 1989; van den Broek, 1990). Magliano and colleagues identified causal relationships between story elements in the text on the basis of causal network analysis (Trabasso et al., 1989). This analysis provided an estimate of the number of causal antecedents and causal consequences in each text. They also identified whether a sentence introduced new entities (i.e., new characters or objects), and hypoth- esized that the presence of these new entities would activate new and potentially relevant world knowledge that could be used while thinking aloud.

Magliano, Trabasso, et al. (1999) found that the presence of causal antecedents in prior text was positively correlated with the number of explanations based on prior text content. In contrast, the lack of causal antecedents was negatively correlated with more knowledge- based (elaborative) inferences. Conversely, Magliano et al. found a positive correlation

between the introduction of new entities and the production of knowledge-based explan- ations, whereas this variable was negatively correlated with the production of text-based explanations. Thus, for relatively familiar and easy text, skilled (adult) readers are more likely to make inferences that link back to prior text when the text affords those inferences. When it does not, and when the text introduces new elements, the reader is more likely to reach outside the text, making knowledge-based inferences.

Todaro et al. (2009) conducted a similar study, but in the context of science texts. They assessed factors that influenced the availability of information from the prior text and world knowledge that could be used for inference and strategic processes. They had parti- cipants type their thoughts (via think-aloud instructions) after each sentence. They classi- fied the content words (i.e., nouns, pronouns, verbs, adverbs, and adjectives) in terms of whether they came from the current sentence, prior text context, or world knowledge. Like Magliano, Trabasso, et al. (1999), they conducted discourse analyses of the text (i.e., a causal network and argument overlap analyses) to assess the presence of text-based factors that should influence the availability of information from the prior discourse and world knowledge. However, in addition to these analyses, they computed a measure of the extent to which a reader’s relevant background knowledge with the topic of a text (e.g., the development of thunderstorms, heart disease) overlapped with the sentences contained in the text. Specifically, they had participants write down as much as they knew about the topics discussed in their text. They then used Latent Semantic Analysis (Landauer & Dumais, 1997; Landauer, McNamara, Dennis, & Kintsch, 2007) to compute a knowledge- resonance score, which was the cosine between each sentence of a text and the prior knowledge protocol associated with that text. The knowledge-resonance score measured the extent to which a reader’s world knowledge overlapped with each sentence of the text. Todaro et al. (2009) conducted a series of multiple regression analyses to determine factors that influence the use of content words from the current sentence, prior text, and world knowledge. Consistent with Magliano, Trabasso, et al. (1999), they found that factors that were theoretically associated with establishing relations between sentences (e.g., number of causal antecedents) were positively correlated with the inclusion of words from the prior discourse context, but negatively correlated with the use of words from world knowledge. Conversely, factors that were theoretically associated with elaboration (introduction of new argument nouns and knowledge overlap scores) were positively cor- related with the inclusion of world knowledge words, but negatively correlated with the inclusion of prior text words.

These studies paint a consistent picture that is relevant to our perspective of self- explanation as a metacognitive process. Specifically, text-based and reader variables are differentially associated with the strategies that we conceptualize as composing self- explanation. The presence of these factors varies across sentences in a text, and therefore, the appropriate strategies for self-explanation vary as well. Readers will be more or less effective when self-explaining to the extent that they modulate the strategies they use across sentences.

Conclusions

We began this chapter with the claim that metacognitive processes are critical for success- ful self-explanation and for strategic reading in general. However, we have described several studies that have failed to establish that metacognition has a strong relationship