Yin (2009) identifies four general principles that underlie high quality case study analysis. They are: 1) that all the evidence has been attended to; 2) all major rival interpretations have been addressed; 3) the most significant aspects of the case study are
64 With this in mind, this section describes the techniques used to analyse the collected data. While questionnaire data was briefly reviewed prior to the undertaking of interviews with student participants, more formal data analysis occurred once the data generation phase was complete. Analysis of qualitative data, central to this
investigation, is described first, followed by an explanation of the quantitative data analyses undertaken.
3.9.1 Qualitative analysis
“Qualitative analysis transforms data into findings” (Patton, 2002, p. 432). Both
inductive and deductive analysis occurred within this research investigation. While self- determination theory (SDT) (Ryan & Deci, 2000a), described in the previous chapter, provided sensitising concepts with which to explore the qualitative data (Blumer, 2006), an inductive approach geared to allowing additional patterns, themes and categories to emerge from the data, occurred concurrently (Bogdan & Biklen, 2007). The qualitative analysis software package NVivo (www.qsrinternational.com) was used during this research investigation to facilitate and manage the complex, iterative process of analysing large amounts of qualitative data.
A separate NVivo project was set up for each case study and a profile for each
participant was established within the relevant project. Once established, open-ended questionnaire responses, interview transcripts (students and lecturers), and online asynchronous discussion transcripts were imported into NVivo. Interview responses to individual questions from all student participants were then collated using the “auto code” feature (see Appendix T for an example). This was also done as a separate process for open-ended questionnaire responses. Data analyses began with student interviews as these represented rich sources of data and were likely to incorporate the broadest range of themes and ideas among all the data sources. Analysis of all
Table 3.4: Summary of the data generation methods
Data Collection Method
Details Rationale Identifier
Student
questionnaires
Online student questionnaire tailored to each case study
Collect demographic information about student research participants.
Measure students’ motivation to learn using the SIMS scale (Guay et al., 2000).
Gain an initial understanding of students’ perceptions of their experiences in the identified online distance environments.
Pseudonym – Questionnaire CSxSyqz
(where x is the (C)ase (S)tudy number, y is the (S)tudent number and z is the (q)uestion number)
Example: Elizabeth – Questionnaire CS1S8q27
Student interviews
Semi-structured interviews with students in the identified case studies
To investigate, in-depth, students’ experiences and their motivation to learn within the context of an online distance learning environment and social and contextual factors that may
influence this.
Pseudonym – Interview CSxSyqz Example: Adele – Interview CS2S3q3
Lecturer interviews
Semi-structured interviews with teaching staff in the identified case studies.
To explore the teaching approaches and methods used by the lecturers (social and contextual factors) responsible for developing and teaching the identified courses.
Pseudonym – Interview CSxLyqz
(where x is the (C)ase (S)tudy number, y is the (L)ecturer number and z is the (q)uestion number)
Data Collection Method
Details Rationale Identifier
Asynchronous discussion data
Relevant asynchronous messages posted, via the discussion board feature within the relevant WebCT course websites, throughout the assignment and associated activities.
Longitudinal data used to confirm ideas or highlight anomalies within the data collected during questionnaires and interviews.
Asydisc (Topic)CSx(S or L)y
(where x is the (C)ase (S)tudy number and y is the (S)tudent or (L)ecturer number)
Examples: Asydisc PBLGpB CS1S1 Asydisc SID CS2L1
Online usage statistics
Student usage statistics automatically recorded via the Track Student function in WebCT.
Used as indicators of online participation. hits
messages read (or reads) messages posted (or posts)
Course resources
Hard copy study guide including administration guide, assignment details, articles and course resources (1 per course).
CD-ROM containing additional course resources (Case Study One only).
To investigate contextual features of each course that relate to students’ motivation to learn.
To enable references to course structure, objectives and resources made in
questionnaire and interview data to be cross-referenced and explored further.
(S)tudy (G)uide (C)ase (S)tudy (p)age number
Example:SGCS2 pp. 7-10
CD-ROM Case Study One
Aggregated data Achievement data of research participants and non-participants.
Online usage statistics data (see above) of student research participants and non-participants.
To compare the achievement of the research participant group with the non- participant group on an aggregated basis.
To compare online participation of the student participant group with the non- participant group on an aggregated basis.
As Bogdan and Biklen (2007) indicate, this process involved reading and re-reading all student answers to an interview question to get a sense of the breadth of responses and the possible range of codes needed to identify initial themes. Using the “coding” feature, each theme was assigned a code and each coded piece of text was placed at a “node” named in such a way that it described the essence of the idea identified (see Appendix U for sample coding). In this way, chunks of text with similar ideas were able to be stored together. These pieces of text varied in length and were coded at all
relevant nodes. This meant that one chunk of text could be coded at one or more nodes depending on whether single or multiple themes were identified.
As the coding process continued, text coded at established nodes were repeatedly reviewed to ensure coding consistency. In some cases this resulted in the further
refinement of codes and re-coding of some data. For example, chunks of text originally coded at the node self-efficacy, were later re-coded to one of two sub-nodes, lack of self-efficacy or sense of self-efficacy. Consistent with Patton (2002), this iterative
process served to clarify and deepen the researcher‟s emerging understanding of the key themes within the data.
Each node was also assigned a description so that it could be referred to throughout the coding process. As this first coding phase continued, code descriptions were developed and a coding structure began to emerge, where „free nodes‟ relating to similar themes were organised into hierarchical structures through the use of „tree nodes‟ in manner similar to that described by Bogdan and Biklen (2007). An example was the types of challenges students experienced while doing the assignment. Nodes were able to be moved within the branching tree structure as key themes were further clarified.
The “memos” feature within NVivo was used to capture growing understandings, ideas, possible patterns in the data, and references to useful literature, at the level of nodes, participants, groups of participants and the case study itself. When combined, they became early drafts of writing about identified themes and served to emphasise possibilities, false leads and illuminate patterns that needed more in-depth analysis.
68 autonomy, competence and relatedness and the motivation continuum furnished the conceptual lenses to explore the data. This motivation theory acted as an organising framework for themes identified throughout the coding process, thereby revealing the social and contextual influences within each case study. For example, the theme
personal relevance emerged as an instance of identified regulation (a type of extrinsic motivation).
Using the initial coding structure, motivation frameworks and the NVivo functions described above as aids, the remainder of the qualitative dataset was analysed. While questionnaire responses, student interviews and lecturer interviews were
comprehensively coded and analysed, messages posted by study participants within the relevant asynchronous discussions were used as secondary data sources. Asynchronous discussions served to confirm themes or patterns highlighted in interview and
questionnaire data or, alternatively, identify discrepancies thereby ensuring data
triangulation. Methodological and contextual problems associated with the rigorous, in- depth analysis of online discussions, particularly with the removal of postings from non- research participants, have been highlighted previously (Cook & Ralston, 2003; De Wever et al., 2006; Garrison et al., 2006). Therefore, this type of detailed analysis was not undertaken in this investigation.
Administration guides for both cases and the CD-ROM for Case Study One were not analysed in detail. Again they were used as a method of triangulating emerging themes within interview and questionnaire data. For example, a key theme identified within the interview and questionnaire data relating to assignment structure was able to be
explored further by reviewing the assignment information provided in each study guide.
On completion of the first phase of analysis for Case Study One, the created coding structure was imported into the Case Study Two project. This was then further developed and refined as part of the coding and analysis process. This was expected given the different context for Case Study Two. For example, student participants worked individually on the assignment in Case Study Two, whereas small group work was required in Case Study One. Therefore, codes associated with “group processes” were not applicable.
This iterative process resulted in ongoing refinement of the coding structure and demonstrated disciplined subjectivity (Erickson, 1973) on the part of the researcher. A common coding structure across both case studies emerged. While not all “nodes” were applicable in each case, the number of “nodes” not common to both cases due to
differing contextual features was relatively small. This constant review and reflection also highlighted key commonalities and differences in the data across the case studies. Qualitative findings, specifically social and contextual influences, are presented in terms of their relative salience within and across case studies (see Chapters Four, Five and Six). Themes with the highest number of coded instances within the dataset are considered most salient. A summary of the main findings from each case study were sent to the relevant study participants.
3.9.2 Quantitative analysis
While qualitative methods and data are central to the exploratory nature of case study research, quantitative analysis can be used to complement and extend the range of evidence on the topic under investigation (Gillham, 2000a). Cross-referencing
quantitative results with qualitative findings constitutes a form of comparative analysis and strengthens the internal consistency of the case study (Yin, 2009).
Quantitative data collected to support qualitative findings included: student
questionnaire responses to the situational motivation scale (SIMS), achievement results and online usage statistics for each respondent. Additionally, aggregated achievement and online participation data were analysed so that the research participant group could be compared with the non-participant group in each case study. All calculations were performed using the SPSS statistical software package.
Situational motivation scale (SIMS) and self-determination index (SDI) scores Situational motivation subscale (SIMS) scores were calculated for each student
participant by adding the responses to the four questions associated with that motivation type (see Appendix L for complete SIMS scale questionnaire). Responses to each
70 motivation score called the self-determination index (SDI). This follows the weighted calculation described and used in previous research (Ntoumanis & Blaymires, 2003; Ratelle, Baldwin, & Vallerand, 2005; Vallerand & Bissonnette, 1992; Vallerand & Ratelle, 2002). This calculation gives greater weight to the motivation types at each end of the scale (i.e. amotivation and intrinsic motivation). Scores can range from a
minimum of – 72 to a maximum of +72. While the calculation of SDI is a useful indicator of motivation, subscale scores were also retained for analysis purposes as SDI may not account for participants‟ endorsement of more than one type of motivation for engaging in the assignment (Vallerand et al., 2008).
Descriptive statistics, calculated for motivation subscale and SDI data for each case study, comprised medians (Mdn) and interquartile ranges (IQR). Nonparametric statistical calculations were performed because of the small sample size within each case study, the inclusion of ordinal scores in the SIMS motivation scale (Guay et al., 2000), and because normality could not be assumed in the underlying population (Siegel & Castellan, 1988).
Correlations and tests of significance
Nonparametric Spearman rho correlation coefficients (Siegel & Castellan, 1988) were calculated to determine whether any significant relationships existed between student motivation, achievement and participation. This was done using participant SDI scores (a measure of motivation), student achievement data (at both the assignment and course level) and various measures of online participation that included active (messages posted) and passive (WebCT hits and messages read) participation measures.
Mann-Whitney U two-tailed tests of significance (Cohen & Lea, 2003) were carried out, as part of the cross-case analysis, to explore whether the participant motivation subscales scores were significantly different between the two case study contexts. Finally, to determine how representative the participant group were of the entire cohort in each case study, Mann-Whitney U tests were also calculated using the achievement and online usage data (WebCT hits, messages posted and messages read) of research participant and non-participant groups.
3.10 Chapter summary
This chapter has explored both the methodology underpinning the present study and the methods used to generate and analyse the data. Research questions were outlined and a predominantly interpretive research paradigm, on which this study is premised, was discussed. Subsequently, case study methodology was examined and the context of the study explained. Consideration was also given to ethical issues associated with the current investigation. Then attention was turned to the methods used to select the cases, the research procedure and the data collection techniques. The methods used to analyse the data were also outlined.
Having described the methodology underpinning the investigation, research findings and some initial discussion are presented, for each of the case studies, in Chapters Four and Five. Presentation of results is guided by the three research questions. Detailed discussion occurs in Chapters Six and Seven.
CHAPTER FOUR
CASE STUDY ONE
Students are likely to experience intrinsic motivation in classrooms that support satisfaction of these autonomy, competence, and relatedness needs. Where such support is lacking, students will feel controlled rather than self-determined, so their motivation will be primarily extrinsic rather than intrinsic. (Brophy, 2010, p. 7)
4.1 Introduction
In this chapter, the results for Case Study One are presented. The chapter begins with a detailed description of the case. This is followed by the presentation of results separated into two parts. Part One directly addresses the first two research questions, namely the nature of motivation and its relationship with online participation. Comparisons between the research participants and non-participants, in terms of achievement and online participation, are then presented to determine whether the study participants are representative of the wider cohort. Recognising the mutually constitutive relationship of the learner and the learning environment (Hickey & Granade, 2004), Part Two focuses on the salient social and contextual factors that influenced pre-service teachers‟
motivation to learn in this online environment.
Throughout the chapter, the continuum of human motivation (Ryan & Deci, 2000a) and the fundamental premises of self-determination theory (SDT), namely autonomy, competence and relatedness (Deci & Ryan, 1985) are used as organising concepts for the presentation of results. While some initial discussion of findings will be presented, detailed discussion of results in terms of similarities and differences across cases occurs in Chapter Six.