The purpose of this study will be to describe the current e-learning initiatives and projects for students in secondary schools in New Zealand. This chapter describes the design of the study, restates the research questions, describes the scope and sources of data as well as considerations for human subjects, and provides a description of the data gathering instrument, proposed analysis procedures and validation processes that were employed.
Design
The purpose of this descriptive case study was to explore, understand, and describe the current e-learning initiatives and projects for students in secondary schools in New Zealand. The study employed the case study method to help understand processes of online programs and to discover context characteristics that will shed light on e-
learning in secondary schools (Merriam, 1998). A qualitative approach was selected for this study, as it allowed the researcher to describe and interpret the perceptions and experiences of the Ministry of Education, school administrators, and secondary school teachers of secondary school students. Understanding the process of implementation of online learning throughout the secondary education system will most likely be of interest to leaders, stakeholders and developers who are starting and growing e-learning programs and schools throughout world.
Case study method. A review of the literature was conducted in preparation for this project, and that research indicates that this study may be the first to examine the full implementation of policy, operations, professional development and content development of online learning in New Zealand‟s secondary schools. To implement this plan the researcher conducted a case study, focusing on a single case: online learning in the country of New Zealand‟s secondary schools (Stake, 1995).
According to Merriam (1998),
A case study design is employed to gain an in-depth understanding of the situation and meaning for those involved. The interest is in process rather than outcomes, in context rather than a specific variable, in discovery rather than confirmation. Insights gleaned from case studies can directly influence policy, practice, and future research. Case studies are differentiated from other types of qualitative research in that they are intensive descriptions and analyses of a single unit or bounded system (Smith 1978) such as an individual, program, event, group, intervention, or community (p. 19).
The case study focused on the implementation of online learning in secondary schools in New Zealand. While it may be important to know and understand the
effectiveness of a program after it has been implemented, the focus of this study was to analyze the process of the implementation. “The lack of concern for implementation is currently the crucial impediment to improving complex operating programs, policy analysis, and experimentation in social policy areas.” (Patton, 2002, p.161). The research questions of this study allowed the researcher to gather detailed, descriptive information about what is going on in the program, how the program has developed over time, and perceptions of what worked and what did not work during the program‟s implementation (Patton, 2002).
This study attempts to tell a story about the unique online learning program in New Zealand by capturing the experiences of the education ministry, school
administration, and teachers‟ experiences through artifact collection and face-to-face and online interviews.
Scope
The scope of the study was limited to the over 2,000 primary and secondary schools in New Zealand and the Ministry of Education of New Zealand. All of the involved schools are state schools and state integrated schools, which are primarily funded by the central government (World Education Encyclopedia, 2001).
Sources of Data
Two types of data from multiple sources were used in this study, and they can be categorized as existing data or newly acquired interview data. Each is described in detail below.
Existing data. In the spring of 2006, the researcher worked as a consultant for the International Association for K-12 Online Learning (iNACOL). One of the projects she completed was a survey to collect data from over 15 countries. She asked leaders in each country‟s Ministry of Education (MOE) to provide information about the current state of K-12 online learning. The returned surveys resulted in a large amount of data in the form of detailed descriptions of how online learning is implemented and provided within each
of these countries. New Zealand‟s MOE provided a response to the survey, and the data from their completed survey are reported in this study.
Newly acquired interview data. Financial, geographic, and temporal constraints prevented the researcher from being able to conduct face-to-face interviews of a large sample of teachers, administrators, and MOE officials in New Zealand. Therefore, some electronic communications were used to conduct the interviews. For this study, the researcher solicited interview data via email and VoIP software from secondary school teachers, administrators, and MOE officials distributed across the state and state integrated secondary schools. The criteria for selection of these teachers and MOE
officials include the following: teaching, participating in, or implementing online learning with secondary students as well as being representative of a variety of different contexts. Such contexts would include: policy makers from various branches of the Ministry of Education and principals and teachers from a few of the schools who were first to
implement online and blended learning opportunities as well as participants from schools who implemented ICT later in the process. The researcher also obtained a list of email addresses from the Ministry of Education prior to the study. Tacit and non-verbal communications were absent from an email interview, but the use of email has been proven to be a viable source of qualitative data (Mann & Stewart, 2000; McAuliffe, 2003; Selwyn & Robson, 1998).
The guidance provided by Anderson and Kanuka (2003) and Mann et al. (2000) for email protocols were followed. These processes included establishing and
maintaining rapport and trust; reflecting in a journal how the Internet may be influencing the interview process; writing short, easy to read and understand, jargon free questions; and providing adequate instructions. Each participant was sent an introductory email, which provided a brief description of the study and a request for their voluntary
participation. This process provided some indication of the number of participants willing to respond, and provided the participants with an opportunity to seek clarification and understanding before committing to the study.
Creswell (1998) recommends that an actual interview use no more than five-open ended questions. Participants, however, were sent nine open-ended questions that focused on the central research questions. This process somewhat expanded the Creswell
recommendation. Some additional follow-up emails were sent for clarification. The interview protocol can be found in Appendix A.
In addition to the email communications, the researcher traveled to New Zealand for two weeks in January 2010. While in New Zealand, the researcher interviewed teachers and administrators involved in online learning from several secondary schools where online learning has/is being implemented. These interviews were recorded. The researcher was also able to visit the Ministry of Education to conduct interviews with the Ministry of Education‟s e-learning team and other officials within the Ministry. These interviews used open-ended questions that lead to discussions on online learning in secondary schools in New Zealand.
1. What has been your role in the implementation of e-Learning in New Zealand‟s secondary schools?
2. What is the nature and extent of e-learning in New Zealand‟s secondary schools?
3. What steps were taken for New Zealand‟s Ministry of Education to
implement a system-wide online learning environment across the country? 4. What areas of the implementation of online learning in New Zealand‟s
secondary schools have had the greatest impact?
5. What problems/obstacles did you encounter and ideas did you abandon during the implementation of online learning in secondary schools? 6. What are the unexpected benefits of implementing online learning in the
country of New Zealand?
7. What are your plans for the future of online learning in New Zealand‟s secondary schools?
8. Would the approach used here in New Zealand work well in other countries? Why or why not?
9. What advice would you give someone preparing an online initiative based on your experiences here in New Zealand?
Process for collection of data. The original recorded interviews were
downloaded into a software program for easy access and review. The researcher then had them transcribed by a company that specializes in this service. All of the transcriptions
were stripped of identifying information. Each interview was given a number to label the information from each participant in order to identify the source in the study‟s findings.
To ensure the accuracy of the interviews, the researcher returned the
transcriptions to each interviewee to make any edits and to correctly input information that was not transcribed due to the heavy accents of some of the participants. After the transcriptions of the interviews were reviewed, the researcher removed all information that was not directly related to answering one of the five research questions. The applicable content was then sorted based on the specific research question and then put into themes within the data.
Human Subjects Considerations
This study abided by all regulations governing the protection of human subjects from any and all harm, including those at the New Zealand Ministry of Education, and Pepperdine University. The study design and plan was submitted by the researcher, as required by Pepperdine University, for review and approval by the Institutional Review Board (IRB) through an exempt review application and was approved prior to the study being conducted. The research involved no more than minimal risk to the participants who were all consenting adults and in no way adversely affected their welfare and rights. The research study was submitted for Exempt Review under Category 7. The researcher worked with the Ministry of Education about IRB policy within the Ministry. The approval of Pepperdine University‟s IRB, will suffice for working with the Ministry of
Education (MOE) in New Zealand. A copy of the Pepperdine-approved IRB proposal appears in Appendix A.
All participants in this study signed a letter of informed consent at the time of the interview. The signed informed consent form verifies that they were adequately informed of the purpose of this study, the questions they were asked, how the information they provided was to be recorded and transcribed, their confidentiality and anonymity; and most importantly, the potential contributions to the field. They were reminded that participation in the study was totally voluntary and that they were able to terminate their participation at any time. Participants were also asked if they wanted to participate in peer-checking to review and confirm the data before the study was published.
Data Collection and Analysis Procedures
Data collection. The MOE was contacted for permission to conduct the study and agreed to participate in the study. They directed the researcher to the employees in the MOE, secondary school administrators, and secondary school teachers who participated in interviews. After the researcher completed the preliminary oral defense, she contacted the MOE to schedule interviews. An explanation of the study was presented and a consent letter provided for all participants, prior to the interviews. The researcher informed each participant that this was a voluntary study and they could opt out at any time. The researcher also explained that only the researcher will know the identity of the participants and their answers. As part of the verification of the study, the researcher
asked participants if they wanted to participate in peer-checking to review and confirm the data before the study was published.
After participants completed a consent form, the researcher contacted each person to schedule a time for the interview.
The interview protocol was tested on a pilot group for language, clarity, cultural differences, and time allotment. This pilot was conducted prior to the researcher‟s trip to New Zealand. An interview guide was used as the approach to the researcher‟s
interviews. This allowed the researcher to “remain free to build a conversation within a particular subject area, to word questions spontaneously, and to establish a conversational style but with the focus on a particular subject that has been predetermined” (Patton, 2002, p.343). By using an interview guide, the researcher was prepared to use the limited time she has with each participant, and it made interviewing such a diverse group of individuals more systematic.
While in New Zealand, the researcher recorded interviews, and then, upon her return to the United States, she employed the services of an outside company to transcribe the notes taken during the interview. The researcher conducted a “fireside chat” with leaders from the MOE‟s e-learning team to summarize the findings and looked for patterns and common themes to help the researcher create codes for those that were found. The researcher also compared the patterns to the notes from document reviews to determine if there was anything that deviated from the interviews. From these results, the researcher made conclusions about the study.
Validation of the data gathering instrument. The validation of the data
gathering instrument, the interview protocol, was completed prior to the interviews with participants. The researcher sent a letter to three experts working in the field of K-12 online learning at the doctoral level that described the study. The research questions and interview guide were shared in the letter. The experts were asked if they felt that the researcher would be able to answer the research questions based on participants‟ answers to the questions in the interview protocol. All three experts felt the answers could be reached with the interview protocol, and the data gathering instrument was used. A few comments were provided to improve the interview protocol and those comments were included in an updated protocol. The final interview protocol that was used in the interviews can be found in Appendix B.
Data analysis. “Two strategic ways that researchers reach new meanings about cases are through direct interpretation of the individual instance and through aggregation of instances until something can be said about them as a class” (Stake, 1995, 74). There were some parts of the interviews and document reviews that the researcher had to directly interpret. Some instances of data had to be pulled apart and put back together again in order to create a direct interpretation in order to understand the case (Stake, 1995). The researcher aggregated the data within pre-defined categories based on the research questions but was also able to define new ones as more was learned during the interviews. Based on the findings during the case more time was spent on one strategy over the other in order to better understand the case.
Finally, a narrative account was written based on the direct interpretations and aggregations of data from the researcher‟s interviews, observations, and document reviews. Stake (1995) states:
Our readers are often more familiar with the cases than we researchers are. They can add their own parts of the story. We should allow some of this input to analysis to help form reader generalizations. The reader will take both our
narrative descriptions and our assertions: narrative descriptions to form vicarious experience and naturalistic generalizations, assertions to work with existing propositional knowledge to modify existing generalizations. (p.86)
In the narrative, the researcher focused on a more personal note, describing her experiences but also provided an opportunity for the reader to make naturalistic generalizations based on a shared description of their own experiences.
Verification
Merriam (1998) discusses how internal validity deals with the question of how research findings match reality. She quotes Ratcliffe on assessing validity in every kind of research.
It should be remembered, he suggests, the (1) “data do not speak for themselves; there is always an interpreter, or a translator” (p. 149); (2) that “one cannot observe or measure a phenomenon/ event without changing it, even in physics where reality is no longer considered to be single-faceted”‟ and (3) that numbers, equations, and words “are all abstract, symbolic representations of reality, but not reality itself” (p.150). Validity, then, must be assessed in terms of something other than reality itself (which can never be grasped; p.202).
The term verification was used “instead of validity because verification
underscores qualitative research as a distinct approach, a legitimate mode of inquiry in its own right” (Creswell, 1998, p.201). Verification of this study was accounted for by
incorporating the following five strategies: (a) triangulation, (b) member-checking, (c) rich, thick description, (d) spending prolonged time in the field, and (e) peer de-briefing. The combined use of these five strategies helped the researcher check the accuracy of the findings to strengthen the verification of the qualitative research.
The first and strongest procedure, triangulation, was used by the researcher using multiple methods to collect data. “With multiple approaches within a single study, we are likely to illuminate or nullify some extraneous influences” (Stake, 1995, p. 114). The researcher conducted interviews with officials from the Ministry of Education and secondary school administrators and teachers. These interviews confirmed the
researcher‟s observations of online learning. In addition to interviews, the researcher also reviewed documents and online content, which also confirmed the findings.
Member checking involves the researcher taking the current data and
interpretations back to the people from whom they were derived and asking them if the results are plausible (Merriam, 1998). The researcher maintained communication with the interviewees to seek that their feedback was consistent with the researcher‟s
interpretations of the data. Each interviewee was first sent a copy of the transcription of their interview to ensure the translation was correct as some interviewees had a thick accent. After they had made any corrections, the researcher took the findings to write chapters four and five. These chapters were sent to the interviewees to ensure the researcher‟s interpretations were correct. Any questions and feedback the interviewees provided was edited in the final version of the dissertation.
The use of rich, thick description was used to convey the findings. This description transported the readers to New Zealand‟s secondary schools and gives the discussion an element of shared experiences (Creswell, 2003). Because the use of a rich, thick description was used, the researcher enables the readers to transfer information to other settings to determine if the findings can be transferred to other settings because of share characteristics (Creswell, 1998).
As stated, the researcher had the opportunity to travel to New Zealand for two weeks to conduct the interviews. Spending prolonged time in the field gave the researcher an in-depth understanding of online learning in New Zealand‟s secondary schools. The