3. MATERIAL Y MÉTODOS 1 Introducción
3.4. Secuencia de actividades
An Exploration of a Multi-Component Literacy
Intervention for Teenage Mothers Attending a Teen
Parent Unit
4.1 Introduction
Teenage mothers demonstrate a greater risk for low achievement in literacy (Bennett et al., 2013; Oxford & Spieker, 2006), which may have a negative impact on their own children’s language and literacy development. Despite these difficulties, literacy intervention studies for this group are sparse. Chapter 3 presented data on the language and literacy skills of 41 teenage mothers attending a Teen Parent Unit in a large metropolitan city within New Zealand. The results of the comprehensive assessment battery indicated 60% of the cohort performed below average on standardised tests of their vocabulary and passage
comprehension skills. This may have been in part due to their early disengagement with education and consequent lack of learning opportunities. As explored in Chapter 1, the implications for this educational underachievement on the young children of teenage mothers are well-reported (Burgess, 2005; Keown et al., 2001; Wadsworth, Taylor, Osborn & Butler, 1984). Additionally, difficulties with literacy limit the future potential for this population to engage with higher education, to increase their job prospects and improve their economic status. Further, they may limit the effectiveness of interventions designed to support the learning outcomes of their children. Intervening with this vulnerable group thus provides an opportunity to halt negative intergenerational literacy effects.
The literature review in Chapter 1 highlighted the limited information available on interventions for adolescents not identified with any learning or other difficulties, but who are struggling with literacy (such as the teenage mothers in this study). Other literature exploring adolescent reading interventions has recognised motivation and engagement as an important consideration in the development of effective interventions for adolescents, and this could be viewed as even more relevant when working with a previously disengaged population. Guthrie and Wigfield (2000) reported that student engagement, over and above the type of instruction, is the primary factor that influences student outcomes. When intervening with a group of students presenting with largely negative experiences around literacy achievement and a history of education disengagement, a motivating and engaging intervention seems essential to affect any change in their literacy skills. Film is increasingly being used as a method of motivating reluctant learners to engage with classroom content (Champoux, 1999; Koskinen, Wilson, Gambrell & Neuman, 1993; Parkhill et al., 2011; Voller & Widdows, 1993). The intervention explored in this study was chosen for its engaging nature, which integrates popular and familiar movies, with a variety of literacy-based activities. The intervention had previously been shown to engage a group of disengaged students with similar characteristics to teenage mothers (Parkhill & Davey, 2012) and was suitable for implementation in the classroom environment. Its multi-component nature also targets a range of literacy skills, a factor identified as successful by Edmonds et al. (2009). This is important as the assessment data presented for teenage mothers with reading comprehension difficulties in Chapter 2 indicated they needed support across vocabulary, comprehension and language.
This study examined the effectiveness of an intervention that integrated the engaging nature of popular movies with same-language subtitles, and multi-component literacy-based activities, to increase literacy skills in a group of teenage mothers with reading
comprehension difficulties. To assess the effectiveness of the intervention, the following research questions were addressed:
1. What is the impact of a multi-component literacy intervention on teenage mothers’ literacy skills? Specifically, passage comprehension, oral reading fluency, vocabulary, spelling and morphology.
2. Does a multi-component literacy intervention improve the targeted literacy skills of a research cohort to a greater extent than those of a general comparison cohort?
4.2 Method
Study Design
A pre-test/post-test research design with a general comparison group was employed to evaluate the effectiveness of a literacy intervention for 10 teenage mothers with reading comprehension difficulties.
Context
All participants were drawn from three Teen Parent Units (TPUs) across two
metropolitan cities of New Zealand. Chapter 1 provides further detail on the context of TPUs in New Zealand.
Participant Recruitment
Forty-one young mothers were participants in the initial eligibility assessment stage, which was described in detail in Chapter 3. Following ethical approval by the University of Canterbury ERHEC, the students’ non-verbal intelligence, language and vocabulary
Verbal Intelligence Fourth Edition (TONI-4; Brown et al., 2010), the Peabody Picture
Vocabulary Test Fourth Edition (PPVT-4; Dunn & Dunn, 2007), and the Receptive Language subtest cluster of the Clinical Evaluation of Language Fundamentals Fourth Edition
Australian Standardized Edition (CELF-4 Australian; Semel et al., 2006). Word
Identification, Word Attack, Word Comprehension, Passage Comprehension, and Oral Reading Fluency subtests of the Woodcock-Johnson Reading Mastery Test Third Edition
(WRMT-3; Woodcock, 2011) were administered to establish the students’ reading abilities. Background information such as number of schools attended prior to the TPU, length of time at the TPU and additional information on their personal backgrounds and child/ren was also collected via an informal interview with each student. Demographic information was gathered from school records.
Students who scored more than one standard deviation below the mean on the
Passage Comprehension subtest of the WRMT-3 were invited to participate in the
intervention (n=25). Of the 25 students who met this criterion, six left school between testing and the implementation of the intervention programme; eight students were excluded due to clashes between the intervention sessions and their regular academic curriculum; and one student was on maternity leave during the intervention. The remaining 10 students comprise the research cohort. An additional four students from a third TPU (school C), along with nine of the students from schools A and B who qualified but did not complete the intervention, acted as a general comparison group. Research and comparison cohort characteristics are presented in Table 4.1.
Table 4.1: Characteristics of Participants
Research Cohort (n=10) Comparison Cohort (n=13)
Mean SD Mean SD Age 18;10 1;2 18;6 1;2 # schools 2 - 2 - # children 1 - 1 - TONI-4 SS 90.5 4.8 91.5 5.9 PPVT-4 SS 73.5 12.8 78.8 10.0 Ethnicity 60% NZ Euro 30% Maori 10% Other 46% NZ Euro 42% Maori 12% Other
Note. Age, (years;months). # schools = number of schools attended prior to TPU.# children =
number of children students have. TONI-4, Test of Non-Verbal Intelligence Fourth Edition (Brown et al., 2010). PPVT-4, Peabody Picture Vocabulary Test Fourth Edition (Dunn & Dunn, 2007). SS = standard score. NZ Euro = New Zealand European.
4.2.3.1 Research Cohort
The research cohort consisted of 10 young mothers (school A: n=5, school B: n=5) who were working at NCEA Level 2 or lower in the New Zealand curriculum. Students ranged in age from 15;6-19;5 years (M=18;10 years).
4.2.3.2 General Comparison Cohort
A general comparison cohort of 13 participants (school A: n=2, school B: n=7, school C: n=4) was established from students from schools A and B who were unable to or opted not to complete the intervention programme, and students from a third TPU (school C) whose data was collected strictly as a comparison measure. Due to ethical and feasibility reasons,
students in the general comparison cohort were not able to be matched or randomised to the research cohort. No significant differences were found between cohorts on pre-assessment measures of age (p=.351), passage comprehension score on the WRMT-3 (p=.183), vocabulary score on the PPVT-4 (p=.418), or non-verbal intelligence score on the TONI-4 (p=.663). General comparison cohort students completed one time point of the assessment battery, results of which were compared to the post-intervention scores of the research cohort. Due to time limitations placed on the study by the participating schools, the comparison cohort did not receive the intervention programme, however, general advice was provided to the schools on how to support these students’ literacy and language needs. At the time of assessment, students ranged in age from 16;4 to 20;0 (M=18;6, SD=1;2).
Assessment Procedure
Eligible research students completed all components of the assessment battery at pre- and post-intervention to monitor the effect of the programme. Standardised measures were completed as part of the assessment of the wider cohort of teenage parents described in Chapter 3; criterion-referenced measures were gathered in an initial assessment session at the beginning of the intervention programme. Post-assessment of all measures occurred in the two weeks following the end of the intervention programme. General comparison students completed one episode of the same assessment battery. All assessments were administered by the primary researcher, a qualified Speech-Language Pathologist, in accordance with
standardised test administration procedures. For individual test items, students were assessed in a quiet room within the school environment; the group items were administered in the classroom setting. Ninety percent of scoring was completed in real time, with the exception of the Oral Reading Fluency subtest, which was scored from the audio recording at a later date. All assessment tasks were audio recorded for reliability purposes.
Pre- and Post-Intervention Assessment Measures
4.2.5.1 Standardised Measures
Passage Comprehension and Oral Reading Fluency subtests from the WRMT-3 were
administered to track changes in reading comprehension and fluency. Different form versions were used for pre- and post-intervention assessment to eliminate any potential learning of items. Standard scores were analysed for each assessment to measure progress.
4.2.5.2 Criterion-Referenced Measures
Criterion-referenced tasks were developed to provide a sensitive measure of change in spelling, vocabulary and morphological awareness skills in response to the intervention.
4.2.5.2.1 Spelling
The spelling assessment consisted of a 15-item verbal spelling test that included proximal, medial and distal items (Hogan, 2012). Proximal words (n=5) were taken from activities within the intervention that directly targeted spelling; medial words (n=5) were morphological derivations of the proximal words (e.g., the medial item ‘continuously’ was selected based on its relationship to the proximal item ‘continuous’); and distal words (n=5) were words taken from the ceiling of an Australian age-equivalent spelling test (South Australian Spelling Test Form A) from Westwood (2005).
4.2.5.2.2 Vocabulary
The vocabulary task consisted of 14 cloze sentence items, which required the student to choose from a given list of words to complete sentences. Words from vocabulary-specific activities within the research intervention were selected as the proximal measures (n=5); words exposed to students during the intervention but not directly targeted for their meaning
were selected as medial targets (n=5); and words selected from the ceiling end of a New Zealand standardised vocabulary measure (NZCER, 2008) acted as the distal measures (n=4).
4.2.5.2.3 Morphology
The 18-item morphology task was adapted from Carlisle (2000) and Kirk and Gillon (2007). In this activity, students were read a prompt word and then a short sentence that they had to complete using a word related to the initial prompt. For example, “Permit. Father refused to give _____ (permission)”.
4.2.5.3 Attendance
Students’ attendance to the intervention was measured by the researcher as present or absent for the intervention session. Attendance ranged from 9 to 23 sessions (mean=14.7, SD=4.9).
Intervention
The intervention was based on the AVAILLL (Audio-Visual Achievement in Language Literacy and Learning) programme, a classroom-based literacy and language intervention using same-language subtitling of popular movies, and associated books
(Parkhill & Johnson, 2009; see www.availll.com/research.html for programme details). Based on the concept of using same-language subtitled movies to improve reading comprehension and vocabulary, the programme activities are designed to engage and motivate students to read; target reading comprehension and vocabulary; and use movies as a way to interest students in the associated novel. The programme utilised five movies and one BBC documentary series as the film component of the intervention. These were: The Freedom Writers, Boy in the Striped Pyjamas, The Blind Side, October Sky, The Golden Compass,
and the documentary series Planet Earth. The associated books used in the programme were: The Freedom Writers, Boy in the Striped Pyjamas, and The Blind Side.
The research cohort completed 26 one-hour sessions of the literacy intervention. Sessions occurred three times per week for a period of nine weeks. Intervention sessions required systematic completion of a range of literacy-focused activities, integrated with watching of the associated movie and reading from the related book. Table 4.2 provides a more detailed description of the activities. Further information on the programme details can be accessed via the programme’s website (www.availll.com/research/html).
Table 4.2: AVAILLL Activity Descriptions
Activity name Key task focus Activity description # of times used in intervention Word Bank Vocabulary,
Morphological Awareness
Students search for words within subtitles that are unfamiliar or interesting to them. At the end of a video clip, words are explored for definition and morphological characteristics.
18
Take a Dictionary to the Movies
Vocabulary The movie is paused on the subtitle containing a challenging word and students find the word in a dictionary and record the
meaning.
5
Sound Effects Reading Fluency Students search for and write down all sound effect subtitles (identified by italics) in a video clip.
3
Surprise Subtitles Reading Fluency The film is stopped unexpectedly eight times, and students write down the last subtitle they read on screen.
2
Next Word Hunt Reading Fluency As the movie plays continuously, students write down the word that immediately follows a common word (e.g. ‘in’) in the subtitles.
2
Film’s End but Book’s Beginning Reading Fluency, Reading Comprehension, Oral Reading Fluency, Imagery
First half of the session was the ‘Surprise Subtitles’ activity, second half of the session was the ‘Read It, See It’ activity.
Spelling Challenge Spelling The film is stopped eight times on challenging words, which students study and record, with a focus on spelling.
2
The Last Word Reading Fluency Students record the last word in the subtitle on screen after the film is stopped.
2
Synonym Search Vocabulary Students are provided with a list of eight words, which they have to locate the synonym for when the film stopped.
2
Match the Meaning Vocabulary Students are provided with eight definitions, which they have to match to a word in the subtitle when the film is stopped.
2
Spellbound Spelling, Reading Fluency
During continuous watching of a film, students locate in the subtitles and correctly record words of ten or more letters.
2
What’s the Word Vocabulary, Spelling
A combination of two activities that focus on word games such as making words from a letter string, and matching letter strings to a word from the subtitle.
1
Read it See It Imagery Students retell a scene from a piece of writing or a film clip using drawings.
1
Word Before Reading Fluency Students search for key words in subtitles and write down the word that occurs before it.
Sessions began with an introduction of the activity and goals of the day, a brief overview of the targets covered from last session and a synopsis of where the movie was stopped (where necessary). The remainder of the session consisted of a selection of three literacy-based activities. Depending on the focus of the task, the activities would occur while the movie was being watched, during pauses to the film, or after the film segment had been completed for the session. In addition to the literacy activities, sessions included viewing at least 20 minutes of same-language subtitled film footage, with the exception of one activity that focused solely on written text from an associated book.
The original structure and timing of the programme was modified to be more suitable to the timetable of the participating schools. Some activities from the original programme were deemed unsuitable or unnecessary for this research group due to the group’s age and interests and were removed. Two additional activities were written by the researcher to cater for the students’ needs. One adaptation allowed for the completion of a movie that was watched only in-part during the original programme; the second was a targeted
vocabulary/morphology activity to focus on an identified need highlighted by pre-assessment data in the activities that did not already include this as a focus (‘Word Bank’). During this activity, targeted vocabulary and other words of interest from the films’ subtitles were discussed and pulled apart to investigate their roots, morphological structure and other characteristics of interest to the students. Along with dictionary usage, students became comfortable with terms like prefix, suffix, tense and root words, and grew their skills around using the morphology of a word to help interpret what it may mean. Furthermore, discussion occurred in response to the students’ interest of subjects and events raised from the content of the films or books.
Sessions were completed face-to-face in a classroom setting. All activities were facilitated by the primary researcher and completed by the research students.
Intervention Fidelity
Eleven randomly selected intervention sessions (just over 20% of the sessions) conducted by the lead researcher were evaluated by a trained independent reviewer familiar with the intervention activities and protocol. The reviewer listened to audio recordings of the selected sessions and recorded the presence and where appropriate, frequency of the
following elements: an introduction of the day’s session, a review of the previous session’s content; a vocabulary activity, a word study activity, and at least one additional literacy activity. In addition, frequency of behaviour management techniques required were also recorded.
Analysis of the sampled sessions showed 100% of sessions adhered to the treatment fidelity protocol described for introduction, review, vocabulary activity and literacy activity. Adherence to fidelity protocol for the word study activity was seen in 81.8% (9 out of 11) of sessions. Ninety percent of sessions (10 out of 11) required a range of behaviour management strategies to keep students on task.
Reliability
Inter-rater reliability of assessment measures was completed for 20% of research participants in passage comprehension, morphology and oral reading fluency subtests. To establish reliability, the independent rater listened to audio recordings of the four verbally presented subtests and scored as per her interpretation. The rater’s scores were then aligned with the original scores, and any disagreements were discussed until a resolution was met. The rater was a qualified Speech-Language Pathologist, independent to the research project, and trained in the appropriate procedures for the assessments used. Marking accuracy was checked on remaining subtests (spelling and vocabulary). Reliability was 100% for these measures.
4.3 Results
Research Cohort
Descriptive statistics of pre- and post-intervention assessment scores for the
intervention cohort in passage comprehension, oral reading fluency, vocabulary, spelling and morphology subtests, p values and effect sizes are presented in Table 4.3. A non-parametric Wilcoxon Signed Rank test was employed to compare changes in standard scores for standardised measures and raw scores for criterion-referenced measures. No significant differences were found for passage comprehension (p=.138), oral reading fluency (p=.258), vocabulary (p=.215), or spelling (p=.206) measures. A significant improvement was observed for morphological awareness from pre- to post-intervention assessment of the research group (z=-2.501, p=0.012), with a moderate effect size (r=.56).
Effect size was calculated by dividing the z value by the square root of N (where N= the number of observations over the time points) (Pallant, 2013). Effect size is useful for comparing the magnitude of difference between two groups, and is particularly pertinent in small group studies, where the impact of statistical significance can be masked by small sample sizes. An effect size of .25, for example, indicates that the research cohort
outperformed the control cohort by one quarter of a standard deviation, and may be viewed as