Quantitative research instruments were utilized by the researcher to analyse the effectiveness of the implemented phonological awareness programme; however, as educational design research also aims to examine ‘how’ and ‘why’ intervention
determine ‘how’ and ‘why’ the programme worked, if indeed it did, for those
implementing it. To enable the researcher to gain a more in-depth understanding of the phenomenon under investigation, qualitative research makes use of a variety of research instruments and data collection strategies. For the purposes of this dissertation, teacher logs, monthly progress reports, and focus-group interview were employed. Reflecting the use of a mixed-methods approach, the use of different research instruments afforded the researcher the possibility to build on the strengths of each type of data collected and minimised the weaknesses of a single approach. The research instruments adopted in this dissertation are described, in more detail, in the next sections.
3.7.1 Quantitative Research Instrument: iPad-based phonemic awareness
assessment tool. Phonemic awareness assessments are an important part of any research
that aims to investigate children’s development of phonemic awareness skills. In order to empirically test the efficacy of the phonological awareness programme, and to address the shortcomings of more traditional paper-based phonological awareness assessments (see Chapter Two, section 2.7), an iPad-based phonemic awareness assessment tool was created by the researcher and used to assess the children’s phonemic awareness skills in both Study One and Two. This assessment tool was based upon and adapted from Carson, Gillon, and Boustead’s (2011) computer-based phonological awareness screening and monitoring assessment tool (see Chapter Four for more detail).
In Study One, two phonemic awareness skills were measured: initial phoneme identity and final phoneme identity skills. The junior infant children were assessed in the third week of September. The assessment was self-administered by the children and the results were sent to a database where scores were recorded and stored. Across the two tasks, there were 20 test items. One point was allocated to each test item with a correct response and incorrect answers were awarded zero.
In Study Two, the iPad-based phonemic awareness tool measured four phonemic awareness skills: initial phoneme identity, final phoneme identity, phoneme blending and phoneme segmentation. The junior infant children were assessed over three time intervals during their first year of school: September, January and June (see section 3.7.1.1 for more detail on the assessment phases of Study Two). During this round of assessments, the four phonemic awareness skills assessed in January were reassessed. These assessments were also self-administered by the children and results sent to a database, where scores were recorded and stored. Across the four tasks, there were 42 test items. Chapter Four of this dissertation explores, in detail, the creation of the iPad-based phonemic awareness
assessment tool implemented in this research.
3.7.1.1 Assessment phases. During Study Two, the participating children’s phonemic awareness skills were assessed at three points during their first year in school using an iPad-based phonemic awareness assessment app. The time schedule for these assessments can be seen in Figure 3.6.
Figure 3.6 Assessment phases in Study Two
The first round of assessments in September acted as a baseline for the study. During this assessment phase, two phonemic awareness tasks were assessed (initial phoneme identity and final phoneme identity), as the researcher felt that the phoneme blending and segmentation tasks would have been too challenging for junior infant children in their first month of school. This decision was made on the basis of research
findings which have demonstrated that these particular tasks are very difficult for young children to complete (Moats, 2003). This decision was further supported by Carson, Boustead, and Gillon’s (2015) investigation into the validity of her computer-based assessment tool (Com-PASMA), where they found that the initial and final phoneme identity tasks were the most appropriate to assess at the beginning of the school year. The assessment phase in January occured directly after the completion of the implemented programme and consisted of four phonemic awareness skills: initial phoneme identity, final phoneme identity, phoneme blending and phoneme segmentation (at this stage, the children had received instruction in phoneme blending and segmentation). The third assessment phase, in June, was administered to ascertain whether the children’s phonemic awareness skills had been maintained, had improved or had declined in the months subsequent to the implementation of the programme. It should be noted that, while the administration of the assessment in September took approximately 10 minutes, the assessments in January and June took approximately 20 minutes, as the children were assessed on four phonemic awareness skills on these occasions.
3.7.2 Qualitative Research instruments. As the teachers were integrally involved
in co-constructing and evaluating the implemented phonological awareness programme, it was important to ensure the teachers’ ‘voices’ was heard both during and after the
implementation of the programme. Two qualitative research instruments were employed to achieve this: the use of documentary sources and a focus-group interview.
3.7.2.1 Teacher logs. The use of documentary sources refers to the analysis of documents that contain information about the phenomenon under investigation (Bailey 1994). Two documentary sources were utilized, collected and analysed for the purposes of this research: teacher logs and teachers’ monthly progress reports. The advantages of using documentary sources are that they are unobtrusive, they provide a good source of background information, and they can highlight issues that are not noted by other means.
Each teacher involved in implementing the phonological awareness programme was asked to complete a weekly log of the phonological awareness skills and activites taught during each week. These logs were implemented to enable the teachers to reflect upon and evaluate the phonological awareness programme, while providing the researcher with feedback on the activities and resources included in the programme. In this log, the teachers documented the tasks that were completed (eg. initial phoneme identity) and outlined the activities they engaged the children in during that week. They were asked to reflect upon and comment on aspects of the instruction that went well or did not go well when implementing the activity (examples of completed logs can be found in Appendix B and B). The teachers discussed their input into these logs at their weekly planning
meetings and, when issues arose, the teachers contacted the researcher to discuss possible amendments to the programme.
3.7.2.2 Monthly Progess Reports. Monthly progress reports are legal documents that individual teachers in Ireland must complete at the end of every month’s teaching. These reports document what has been taught to the class during that month. At the end of each school year, the teachers’ monthly progress reports (both control and experimental teachers), relating to the teaching of English, were collected by the researcher. These reports were collected for two reasons. Firstly, they allowed the researcher to establish the ‘usual’ English curriculum that was delivered to the control groups. Secondly, using these reports, the researcher was able to compare and contrast how much and/or when instruction in phonological awareness took place during the school year for both the control and the experimental groups. An example of a completed English monthly progress report can be found in Appendix C.
3.7.2.3 Focus-group interview. A focus-group interview is a group discussion on a particular topic organised for research purposes, with the discussion guided and recorded by a researcher. Focus groups are used for generating information on collective views, and
are useful in generating a rich understanding of participants’ experiences and beliefs. According to Morgan (1988), the advantage of holding a focus-group interview is that the reliance is on the interaction within and between the group who discuss a topic supplied by the researcher. It is from this interaction, that rich descriptive data emerges. In preparing the interview schedule for the focus-group interview, the researcher drew on Stewart and Shamdasani’s (1990) two guiding principles:
1. Questions should move from general to more specific questions 2. Question order should reflect the importance of issues in the research
agenda
The questions posed to participants during the focus-group interview can be found in Appendix D. During focus-group interviews, discussions often take on a life of their own, as the researcher probes and expands on issues as they arise during the discussion.
However, it is important that the researcher can keep the discussion focused without leading it. Researchers conducting focus groups should guide a discussion rather than join in with it, as expressing one’s own views may give participants cues as to what to say and this can introduce bias. Researchers must also be prepared that some views may be critical of aspects of the research that may be regarded as important to him/her.
The focus-group interview was conducted in a classroom within the school where the research took place. All four teachers involved in implementing the phonological awareness programme took part in the focus-group interview that lasted no longer than one hour. The interview was conducted at the end of the school year, after the last round of assessments had been completed. The objective of the focus-group interview was to determine ‘how’ or ‘why’ aspects of the phonological awareness programme worked, or did not work for those implementing it. This interview provided rich insights into how the programme supported the teaching and learning of phonological awareness skills and highlighted aspects of the programme that needed attention for future implementations. As
the teachers were important stakeholders in the research, it was crucial to listen to their views as, ultimately, they were the gatekeepers to the future sustainability of the
programme. The focus group interview was audio recorded and, later, transcribed by the researcher.
The next section examines an emerging and evolving approach to educational research that embraces the need to address educational problems within real-world contexts and aims to reduce the existing, and much debated, research-practice divide. Referred to as educational design research, this methodological approach underpins the research that was conducted in this dissertation.