As this study is concerned with the exploration of student motivation to learn in online contexts, it primarily seeks to use the contributions, experiences and interpretations of the participants themselves to inform understanding. Therefore, multiple data were collected using a variety of data collection methods. A summary of data collection methods used to address the research questions are shown in Table 3.3.
The data collection methods are described in the sections that follow. These procedures included: data generated after the completion of the assignment in each case study, namely questionnaires, interviews, and aggregated data; and data generated during the assignment period as part of normal online course administration processes (i.e. online asynchronous discussions and student usage statistics) but collected after all coursework was completed, graded and results submitted.
Table 3.3: Research questions and data collection methods
Research Question Data Collection Methods 1. What is the nature of motivation to learn of
pre-service teachers in online-distance learning environments?
Student questionnaires Student interviews Lecturer interviews Archived online data Aggregated data
2. How does the motivation to learn of pre- service teachers relate to their participation in online distance learning environments?
Student questionnaires Student interviews Archived online data Aggregated data
3. In what ways do social and contextual factors relate to pre-service teachers’ motivation to learn in online distance learning
environments?
Student questionnaires Student interviews Lecturer interviews Archived online data Course resources
A questionnaire was created for each case study. Prior to making the questionnaire available to participants, the questionnaire was independently tested to ensure the instructions were clear and the questionnaire worked as intended (see Appendix L). The
58 The questionnaire contained three sections. Part one consisted of fixed response
questions designed to collect demographic information, including name, email address, gender, age group and ethnicity, that helped to situate the respondents in relation to others. As achievement can be an important indicator of motivation (Schunk et al., 2008), respondents were also asked to supply their assignment mark.
Part two measured learner motivation using the self-report situational motivational scale (SIMS) developed by Guay, Vallerand, and Blanchard (2000) that operationalises the self-determination continuum (Ryan & Deci, 2000a) described in Chapter Two. It measures situational intrinsic motivation, extrinsic forms of motivation (identified regulation, external regulation) and amotivation using 16 seven-point Likert scales with four questions for each motivation subscale (see Appendix L). The questionnaire was administered after the completion of the assignment and associated activities.
Therefore, the SIMS data can be considered retrospective and cross-sectional in nature.
Part three consisted of nine open-ended questions, developed with reference to current motivation literature (Brophy, 2010; Reeve, 2002), exploring possible relationships between social and contextual influences and learners‟ motivation. These were designed to gain some initial understanding of the interplay between the environment and student motivation. Questions covered the structure of the assignment, support and feedback received, the online course environment, interactions with peers, and available learning resources. Questions asked participants to reflect on their experiences throughout the duration of the assignment. Therefore, this data can be considered semi-longitudinal in nature.
3.8.2 Interviews
Following the questionnaires, individual semi-structured interviews were undertaken to “gather descriptive data in the subjects‟ own words so that the researcher ... [could] develop insights on how subjects interpret some piece of the world” (Bogdan & Biklen, 2007, p. 103). Interviews with student participants provided an opportunity to gain insight into their experiences, how the context influenced their thinking, feeling and behaviour, and perspectives of learning in an online environment over the duration of the assignment. The interviews represented a central source of data because of the richness and depth of material (Gillham, 2000b). They also afforded the opportunity to
explore open-ended questionnaire responses in greater depth (Glesne, 2006), and check for consistencies or anomalies in responses. Interviews were also conducted with the lecturers responsible for online teaching and management within the two cases studies. The purpose of these interviews were to explore teaching approaches, methods and interactions with students, which are known to influence learner motivation (Brophy, 2010), and for data triangulation purposes (Glesne, 2006).
Student interviews
A semi-structured interview schedule was designed to investigate students‟ motivation to learn within the context of the online learning environment during the period students worked on the assignment and associated activities in each case study. It also included several questions that explored learner perceptions of the whole course (see Appendix N). Interview questions were developed after reviewing the literature on motivation (see Chapter Two) and were designed to tease out factors that influenced motivation as the assignment progressed. Topics covered included: assignment workload; areas of interest and enjoyment; clarity of understanding of assignment requirements; choices available (if any); challenges, problems and frustrations experienced (if any);
perceptions of the online learning environment; perceptions of what had been learnt; and impressions of the course as a whole.
Although student participants were located throughout New Zealand, face-to-face interviews were conducted where possible. Meeting participants in person was seen as an opportunity to further develop rapport, trust and build relationships (Bogdan & Biklen, 2007) initiated online. All student participants, with one exception, were located in the North Island of New Zealand. This enabled the researcher to conduct face-to-face interviews in all but one instance, which was conducted by telephone. Interviews took place in a neutral location to provide a comfortable, informal setting where both parties could feel free to „chat‟.
At the outset of each interview, the researcher encouraged participants to express their opinions freely as their identity would remain confidential in the reporting of any
60 where necessary, as well as to explore participant responses in more depth when
appropriate. Interviews were approximately 45 minutes to one hour in duration.
Lecturer interviews
A further semi-structured interview schedule was designed to explore the teaching approaches and methods of the lecturers involved in teaching the courses under
investigation (see Appendix O). Interview questions were developed after reviewing the literature on motivation (see Chapter Two) and covered topics including: overall
impressions of the student cohort taking the course; structure of the course curriculum and the specific assignment of interest, learning objectives and resources provided; expectations of student participation; methods used to develop learner interest; choices and feedback/support given to learners; challenges, problems and frustrations
experienced (if any); and impressions of teaching in an online learning environment.
Lecturer interviews were conducted in the lecturer‟s office on each occasion and were approximately of one hour duration.
3.8.3 Archived online data
Usage statistics and asynchronous online transcripts, collected throughout the course, represented an important source of data that served to supplement interview and questionnaire data which encompassed participants‟ perceptions collected after the courses were finished. These data were longitudinal in nature (i.e. covered the duration of the assignment) and were not affected by the presence of the researcher (Berg, 2004).
Online asynchronous discussion transcripts
By collecting online asynchronous discussion transcripts, perceptions of both lecturers and student participants could be confirmed, or anomalies highlighted. Furthermore, online discussion transcripts provided a source of data that enabled the quality of online participation to be explored.
Asynchronous discussion data was accessed and downloaded once all student course work had been completed, graded and results finalised. This was done using the compile
downloaded from the website as a text file. Downloaded transcripts included
contributions from all students in each course. This was done so that all messages could be read within the context in which they were originally posted. Transcripts were refined by removing any messages posted outside the assignment timeframe and only those messages posted by study participants were included for analysis purposes.
Online usage statistics
WebCT automatically maintains internal usage logs via the Track Student function. This keeps a record of the number of times a student accesses the various course features (WebCT Inc., 2003). Three measures of WebCT use were obtained as quantitative indicators of online participation:
1. WebCT hits: The number of times each student accessed the homepage (first page following sign on), any tool (from the options available), or a content module page. 2. Messages read: The number of messages each student opened in the discussions
tool. This included messages read across all discussion topics.
3. Messages posted: The number of messages each student posted in the discussions tool across all discussion topics (Johnson, 2005).
Automatic collection of WebCT usage statistics is cumulative. However, as part of normal course protocol, the course coordinator for Case Study One captured this data on a weekly basis. This meant that the researcher was able to access and analyse week by week usage statistics over the six-week duration of the assignment. This allowed participation rates of students and lecturers, over the period of the assignment, to be explored to determine whether any patterns existed.
Case Study Two student online usage statistics were collected over the duration of the whole course, therefore no week by week comparison could be undertaken. This meant that only a general impression of online attendance could be obtained for student participants over the whole course rather than statistics specific to the assignment of interest. Although crude, this was sufficient given the limitations of approaches that use
62 3.8.4 Course resources
Resources provided to students at the beginning of each course were also collected as part of the data generation process. This ensured that any references made by
participants to the course structure, objectives and resources, during questionnaires and interviews, could be cross-referenced and explored further.
Case Study One course resources included a study guide and a CD-ROM. The study guide included an outline of learning intentions, a summary of assessment components, success criteria, course calendar, expected course workload, and online class and group discussion expectations. Next, the assignment tasks were explained in detail. In
particular, the problem based learning (PBL) assignment (the focus of Case Study One) was outlined. This covered 23 pages and incorporated several assessment matrices used to assess the different components of the assignment. This was supported by a series of appendices designed to be used at different points within the PBL assignment.
The CD-ROM contained a range of resources, exemplars and software organised in a series of folders. The majority of these were designed for use during the PBL
assignment. In a few instances, students were directed to specific resources on the CD- ROM at certain points in the course. However, the majority were not referred to directly by teaching staff and it was left to the students themselves to explore these
independently. Some students also chose to access and use additional online resources throughout the PBL process that were relevant to their particular investigation. Given that these resources were unique to the problem being investigated within each small group, they were not included as a data source in this study.
Case Study Two students also received a study guide at the commencement of their course. This included an administration guide that encompassed an outline of learning outcomes, the content of the course, online participation expectations, required texts, and recommended readings. This was followed by more detailed information on each assignment that incorporated marking schedules. This included the micro-teaching and reflection assignment (the focus of Case Study Two). The remainder of the study guide was then divided into modules and concluded with a booklet covering a wide range of resources and approaches to social studies. In addition to the above, the lecturer
provided additional online resources in the form of useful websites, social studies exemplars and examples of student work from previous cohorts.
3.8.5 Aggregated data
One final source of data in the form of aggregated achievement and online usage statistics data for all course participants were collected so that comparisons could be made between the two groups (i.e. research participants and non-participants for each case study). This was to determine whether the research participants‟ achievement and online participation were typical of the cohort when compared with non-participants in terms of the assignment of interest and the overall course.
Relevant achievement data from each case study included: the name, final course grade, and assignment mark for all students. This information was provided to a third party (the researcher‟s supervisor) so that the privacy of both research participants and non- participants was preserved. A final list of grades and marks for two groups: participants and non-participants, with all names removed, was then forwarded to the researcher for the purposes of data aggregation.
Comparisons were also made between aggregated online attendance data of research participants and non-participants for each case study. An additional request for this data was not necessary as the usage statistic data originally provided by both course
coordinators included information for all course participants.
A summary of the data generation methods used in this research investigation, their purpose and how they are identified is presented in Table 3.4.