Semiotic mediation and transmediation shaped discussion and literary interpretation by priming students’ interpretive thinking and promoting students’ participation in whole class discussions in the multimodal condition. Figure 5 displays two episodes that came out of the multimodal focal group’s small group discussion.
Small Group Episode 1
Louise: Should they be around like a big cauldron? Leonard: No, they’re not in the first one they weren’t. Louise: What were they doing for the first one? Leonard: They just meet period.
Louise: They’re just meeting?
Leonard: It says like they meet in a place where (?) […] Louise: Where were they meeting in the first part? Nick: In a creepy.
Leonard: No, straight, like plain, nothing, a tundra place. Nick: But it was like foggy and like=
Leonard: =There was storms comin’ and stuff or whatever and there was three people and then him. Nick: =Yeah. And it’s all like cloudy and eerie.
Louise: Weren’t they on horses? Leonard: Nope.
Small Group Episode 2
Leonard: Mr. Smith? Okay, I put “The witches know the prophecies are fake but make Macbeth believe him because...” what would be a word for like when you try to make you believe something that isn’t true but the way I say it makes you believe it? Like calling him into it (extends left arm out and retracts left
arm back toward his body), but a different word?
Mr. Smith: Well, remember that specific word we talked about that goes with the play? Like… Leonard: Is it in the fifth act?
Mr. Smith: It’s used in the fifth act. Louise: It starts with an "e"? Mr. Smith: Yeah.
Louise: I can't remember what it is, but I know it starts with an "e" though. Leonard: Edgerdiction or something like that?
Mr. Smith: The word, what's the specific thing that they do, the way they speak, the words she speaks (??) Louise: Oh my Jesus.
Leonard: Why do I keep wantin’ to say (dictation?) Nick: What are some of the things they do? E? Enchant. Mr. Smith: (with text in hand) Act 5, scene
Leonard: One
Mr. Smith: More like scene Leonard: four.
Mr. Smith: seven.
Leonard: How many scenes are there in this one? Damn. (leafing through Mr. Smith's text) It’s before this.
Mr. Smith: Yeah, it’s before that. Leonard: Equivocation.
Louise: That's the word! Equivocation.
Leonard: “Because of their equivocation.” Can I just put that? Mr. Smith: Yep.
Nick: That's a big word. I don't even know what it means.
Leonard: (without looking in any text) The ability to make someone believe something false—a lie is another way to say it. Alright this is what I have. I'll slim it down, but this is just like the gist of it. If you want to add anything, go ahead.
Louise: No, that’s alright. I'm not really tryin’ too hard to make this like some masterpiece. Nick: You coulda used deception there, too.
Figure 5. Cycle 1 Multimodal Focal Group Talk Prior to Whole Class Discussion
Students responded during the whole class discussion to the question, “Why does the success of Macbeth and Lady Macbeth—prophesied by the witches, promising the couple power and riches and ‘peace to all their nights and days to come’—why does their success of becoming king and queen turn so quickly to ashes, destroying their relationship, their world, and, finally, both of them?” Leonard’s contributions below can be traced to aspects of the small group conversation that were excerpted in Figure 5:
1. Ian: ‘Cause they did it the wrong way, not the way it was supposed to be. It altered the path, like it altered everything that happened. So…
Leonard: What the witches told him was true. All they did was tell him exactly what he wanted to hear. And the fact that they’re witches and have the powers that they’re believed to have was enough to encourage him to do it.
Ian: The witches also said what would happen when he died, so how could it be true?
Louise: They didn’t lie to him=
5. Leonard: =They never said anything about him dying though, did they? Louise: They didn’t lie to him, they just manipulated what they were
saying. They just manipulated him by saying things Malcolm: (???)
Louise: Yeah, like by saying things in a very vague way. Leonard: Cryptic messages. Like they made him believe a lie. 10. Louise: They weren’t exactly lying to him.
Leonard: That’s kinda how it is, yeah.
Louise: What they said was going to happen, they just changed it around for their own benefit. All of the things that happened were going to happen, they just…
the truth or anything. I just think they manipulated what they were saying to use him, but I don’t think they actually lied.
15. Malcolm: (half sarcastically?) They exaggerated the truth. Louise: Exaggerated would be a good word to use.
Leonard: Didn’t the head say, “What have you done?” or whatever? Or “Why would you have done this?” Isn’t that kinda like saying it was allowed by the head like saying that, like why did you go and do this?
Mr. Smith: Well, this is something that I’m wondering right now. We know the second set of prophecies about “not of woman born and the forest moving”—we know that those were definitely meant to mislead him.
Leonard: And plus the apparitions, is that what you’re talking about? 20. Mr. Smith: Right.
Leonard: Well those were like spells and things like that, because they had the cauldron and they were putting stuff in it. Actually like creating the spells or having the apparitions come out and say that to him. And before the first time they’d met it was just the three witches and him and just like a tundra (gestures to show a showering down
of sorts).
Leonard’s interpretation in turn 2 (“And the fact that they’re witches and have the powers that they’re believed to have was enough to encourage him to do it”) was mediated by his talk up until that point. That is, this new idea came about through his speaking. This interpretation
generated Ian’s challenge to Leonard’s claim that the witches told Macbeth what he wanted to hear because they foretold what would happen when Macbeth died. This utterance led both Louise and Leonard back to the text, specifically the language of the text and what a “lie” might actually mean as well as whether or not the witches even mentioned Macbeth’s death.
Leonard’s contribution, “Like they made him believe a lie” (turn 9) came from the previous day’s small group discussion, when drafting the caption for the painting, Leonard sought to use the “e” word that Mr. Smith had mentioned during a previous class (see Figure 5). He and Mr. Smith returned to the text together, found the word “equivocation,” and then, Leonard, said the word out loud. When Nick heard the word, he reacted “I don’t even know what that means.” Immediately, Leonard retorted, “It’s like they made him believe a lie.” In the passage from the whole class discussion above, Leonard’s contribution supported Louise’s claim that manipulation and lying were different things, that had different connotations attached to them, and thus, built a classroom culture in which students were accountable for their talk in being accountable to the discourse conventions of the discipline (Michaels, O’Connor, & Resnick, 2008).
With less than a minute left in the whole class discussion of Macbeth, Leonard responded to the final discussion question (Why do people do evil knowing that it’s evil?) by drawing on the product of the previous day’s collaborative multimodal project. That is, Leonard’s response seemed to recast his understanding of the visual composition he co-created into the linguistic mode of discussion, functioning, for the lack of a better word, as a re-transmediation in which he brings back to the linguistic mode what had been previously transmediated from the linguistic mode into the visual mode (see Figure 1):
I think people are blind to the evil in the sense that anyone who’s about to do an evil deed, the person who has come to them to talk to them about it has influenced them enough for them to only see the good in the outcome and not the bad. They know the bad things that will happen, but when they are confronted with the evil deed that they must do, they are confronted with the fact that if you do this, this will come out of it, this will come out of it, this…And [Macbeth] looked past the fact that he was going to kill the king, and he was going to have to kill people to become king. He just said, “I’m gonna think about the good of it and that’s it.”
Leonard’s reference to “the person who has come to them to talk to them about it has influenced them enough for them to only see the good in the outcome and not the bad” constituted the verbal expression of the visual image depicted in Figure 1. The collective “person” who came to talk to Macbeth was the group of three witches in the left side of the painting. The witches influenced Macbeth to “only see the good” (the crown of the king) in the top right side of the painting and “not the bad” (the former king’s decapitated head and the dead bodies of those Macbeth murdered) at the base of the image. In essence, Leonard’s response translated the image of the painting from the visual sign system into the linguistic sign system of the whole class discussion.
In sum, students’ participation in collaborative multimodal project work afforded a number of learning opportunities. First, students extended their responses during whole class discussions without prompting by the instructor. This was evidenced by the average number of words that students used per turn when compared to baseline, unimodal, and teacher averages in the same category (Table 17). Students in the multimodal condition provided additional perspectives on the text as evidenced by the number of interpretive discourse turns coded as
extending/elaborating, perspective taking, challenging, uptaking, intertextually responding, and argument conceding (Table 15). The use of inferences to evidence claims made about the text as well as interpretive, text-based rather than personal, non-text-based reasoning by examining characters and events in the text characterized students’ talk in the multimodal condition (Table 15).
Discourse analyses revealed how focal group students pushed each other to think deeply about the text by connecting metatalk about the text with ways in which to facilitate the completion of the task and the achievement of the group’s goals for the task (Table 7). Students in the focal group actively co-constructed meaning during their small group conversations in which every group member provided the same approximate ratio of codes for semiotic mediation (Table 11). These students also primed their own interpretive thinking by using ideas generated during small group discussions to respond to others’ ideas during whole class discussions (Table 18). Finally, one focal student used the multimodal product as a thinking device as he re-shaped his response during a whole class discussions of literature by leaning heavily on the product of the multimodal task to respond to an interpretive question during the whole class discussion. Although these findings do not prove that multimodality accounted for the affordances described above, these findings do point toward the positive trend that existed in the multimodal condition, especially for members of the focal group who leveraged their multimodal literacy practices and effectively mediated their thinking during the small group discussion that supported their participation during the whole class discussion.