While rating drafts with the rubric provides an overall picture of improvement (or lack thereof), gains in score cannot fully explain what peer responders decide to do with comments after their feedback sessions. For example, Ivana, whose first and second drafts were used as an example for calculating score gains in the last section of this chapter, had only a 1.5 point average gain in score for the second assignment. It may appear from this number alone that she
did not make substantial changes on the second draft. However, classifying the types of her revisions in the way described in the rest of this section shows that she actually made multiple revisions throughout: Ivana was substantially engaged in making changes to her draft.
Classifying the types of revisions students make (the focus of this section), as well as tracing reviewer comments for uptake in the second draft (described in the next section), provide a more nuanced account of what happens after peer response.
Faigley and Witte’s (1981) taxonomy of revisions, which analyzes the effects of revision on text meaning, is used in this study to investigate the kinds of changes that writers decide to make after peer response sessions. In this taxonomy, there are two different kinds of revisions: those that affect meaning (text-based changes), and those that do not (surface changes). Meaning is construed as “concepts in the extant text, as well as those concepts that can be reasonably inferred from it” (p. 404). If new information is either brought to or removed from the text in such a way that it cannot be recovered through drawing inferences, a text-based change has occurred. Text-based changes can be either macrostructure changes, which are major revision changes that would alter the summary of a text, or microstructure ones that would not alter the summary of the text. Surface changes, on the other hand, do not bring new information to the text, or remove old information: these changes include formal changes like edits in spelling, tense, and punctuation; and meaning-preserving changes, or paraphrasing where meaning is not altered. Within each category, revisions can be further classified according how the text is altered (eg., additions, deletions). Figure 5-3 shows a visual representation of the revision taxonomy.
Figure 5.3 Faigley and Witte's revision taxonomy (1981)
The following three sections will explain how I marked revisions made in the second draft, classified them according to the taxonomy provided in Figure 4.2, and trained research assistants to confirm or amend these classifications.
On clean copies of the drafts, I first placed first and second drafts side by side and underlined the revisions present in the second text. If words were removed from the first text,
() was written on the second draft where the original words had been. Figure 5-4 provides an
example of how second drafts were marked up in this way. In this figure, the beginning of Ivana’s first and second drafts for assignment two are displayed. Revised sections of the second draft are presented in bold and underlined font:
First Draft
The article The Immunzation-Autism
Myth Debunked (Recame, 2012) was
written by Ph.D student Michelle Recame, BS in response to British medical
researcher Wakefield’s study, which
claimed that autism is an outcome of MMR (measles, mumps, rubella) vaccine.
Wakefield’s study was published in 1998 and caused big concern among the public. The researcher argued that MMR vaccine lead to gastrointestinal disease, Crohn’s disease and autism. Michelle Recame (2012) completely disagree that there is any connection between immunization and these diseases. In the article the author asserts that not enough evidence was provided to prove this connection, and the Wakefield’s study was based on
insignificant amount of examined children. The author accused Wakefield and his co- researchers in creating fear among the parents, who became frightened to take MMR vaccine, which rating was
essentially declined in Britain. According to the article, the number of unvaccinated children increased, therefore there is a big risk of disease outbreak. The author
explains the risk of rejecting immunization, showing that 98% of infected children were not previously vaccinated.
Second Draft
The article The Immunzation-Autism
Myth Debunked (Recame, 2012) was
written by Ph.D student Michelle Recame
in response to British medical researcher
Andrew Wakefield. The study claimed that autism is an outcome of MMR (measles, mumps, rubella) vaccine. Wakefield’s study was published in 1998 and caused big concern among the public. The researcher argued that MMR vaccine lead to gastrointestinal disease, Crohn’s disease and autism. Michelle Recame
(2012) completely disagrees that there is
any connection between immunization and
these diseases. The author asserts that
not enough evidence was provided to prove this connection, and the Wakefield’s study was based on insignificant amount of
examined children. Also, Recame accused
Wakefield in creating fear among the parents, who became frightened to take
MMR vaccine. Consequently, the rating
of the vaccine has essentially declined in Britain and all over the world in the aftermath. According to the article, the number of unvaccinated children increased, therefore there is a big risk of disease outbreak. The author explains the risk of rejecting immunization, showing that 98% of infected children were not previously
vaccinated. Recame claimed that
concerned parents amp to trust more the mass media, Internet, family and friends instead of doctors. The lack of dialogue between parents and their pediatricians is the chief problem, which must be resolved, according to Recame (2012). Figure 5.4 Ivana's first and second drafts (revisions highlighted and in bold text) As the right column (Second Draft) in Figure 4.4 shows, Ivana added new text in the second draft of her paper in five different places. These additions are shown in bold and
underlined text. Among these additions are sentences (e.g., “The study claimed that autism is an outcome of MMR (measles, mumps, rubella) vaccine”), phrases (e.g., “Also, Recame accused Wakefield …”), and words (e.g., “disagrees). The right column also shows that there were two places where Ivana omitted words or phrases from the second draft that had been included in the
first one; these omissions are marked with the symbol “” in the second draft. For example, in
the first draft Ivana referred to the author of the article she summarizes as “Michelle Recame, BS”, but in the second draft, she removes the bachelor’s degree abbreviation, so that the second
draft reads simply “Michelle Recame”. This change is represented as “Michelle Recame ” in
the right column of Figure 4.4.
After changes were marked on all twenty-six second drafts, I completed a revision record for each draft, which listed each change that occurred in the second draft, and assigned it a revision code according to Faigley and Witte’s taxonomy. Figure 5-5 provides the revision record for the same paragraph of Ivana’s writing that is presented above.
Draft One Draft Two Revision Code
(1) BS S-Del
(2) Study, which claimed Wakefield. The study
claimed.
S-Dis
(3) disagree disagrees T
(4) In the article S-Del
(5) The author accused Wakefiled
Also, Recame accused Wakefield
S-Sub
(6) Consequently, the rating of
the vaccine …
Mic-Ad
(7) Recame claimed … Mic-Ad
(8) The lack of dialogue … Mic-Ad
Figure 5.5 Revision worksheet for Ivana's second draft
As Figure 4.5 shows, if a writer adds new sentences to the second draft, each new sentence is assigned a code (see entries six through eight). On the revision worksheet, entire sentences are not reproduced, but rather the first few words are written, followed by an ellipsis. Because raters are looking at first and second drafts while they complete the worksheet, they can locate the entire sentence on the draft.
After I completed a worksheet like the one shown in Figure 4.5 for each of the twenty-six second drafts, the same group of raters who scored drafts with the rubric were trained to confirm or replace the revision codes. In a training session, I described the coding scheme and provided examples of each type of revision from Faigley and Witte’s (1981) article. The coders and I then worked together to practice assigning revision codes to student writing not used in the study. During the second coding that followed, research assistants used the worksheets I had already
completed (see Figure 4.5 above for an example), and noted whether they agreed with the code, or thought a new one should be assigned. Inter-coder reliability was calculated at ninety-eight percent in this phase of data analysis. The few discrepancies were resolved by discussion
between me and the second coder. In these discussions, we referred to the description of Faigley and Witte’s taxonomy, sometimes referencing Faigley and Witte’s original article in addition to the summary contained in the rater-training packet. After these disputes were resolved, one hundred percent inter-coder reliability was reached.