Finding out why science teachers may (or may not) find difficulty in understanding and teaching complex systems ideas can clarify the challenges and issues that may need addressing in future efforts to improve the learning and instruction of complex systems. RQ 3 is answered from the analyses of the written responses to Part C of the Perception questionnaire and the teacher interviews. How the data was analyzed is described in the following sections before a discussion on the validity and reliability of the data analysis.
138
3.6.5.1 Perception questionnaire
Recall that in Part C of the post-workshop questionnaire, teachers were asked if they perceived complex systems ideas difficult to understand, and why. The list of reasons as explained earlier were developed based on a review of the literature and expanded subsequently after a brainstorming process. They were also free to add reasons that were not listed. Unfortunately, as the actual sample size was small (i.e., less than 100 even though the targeted sample was supposed to be more), factor analysis could not be performed on the selected reasons (Ferguson & Cox, 1993; MacCallum et al., 1999). Instead, descriptive statistics were used to provide insight on what the reasons may be.
3.6.5.2 Teacher interviews
The semi-structured interviews aimed to find out the reasons for the perceived difficulties teachers face in understanding and teaching complex systems (RQ 3). A thematic analysis was performed to analyze these interviews for the reasons (Miles & Huberman, 1994; Smith & Osborn, 2003). Thematic analysis involves searching through the qualitative data to identify themes or in this case, reasons for difficulties teachers face in understanding and teaching complex systems. A theme is akin to a cluster of linked reasons conveying similar meanings (Berkowitz, 1997; Patton, 2002). There are generally three distinct phases in this analysis (Elliott & Timulak, 2005).
First, the data was prepared for analysis. All interviews were transcribed in verbatim and subsequently loaded into the ATLAS.ti version 7.1.3 software to facilitate the analysis. Each line of utterance served as a unit of analysis for indexing or coding. Such line-by-line coding allowed the data to be closely scrutinized. Next, an overall
139 organizing structure was introduced to help make initial sense of the data. As I already had some broad ideas of what the reasons might be based on the review of the literature (see sections 2.3.4 and 3.5.2), broad domains of curriculum, ontology, learning and beliefs were used to ‘park’ the units of analysis.
Following this domain grouping, the units were categorized. This was the most intensive part of the analytical process. Each unit was meticulously mined for key points that related to the reasons, and descriptors were assigned. Similar descriptors were then grouped as a first level of consolidation. From these groupings, a second level of consolidation was undertaken where categories of reasons were formed. These categories formed the themes of reasons behind teachers’ difficulty in understanding and teaching complex systems ideas.
Creation of categories was an interpretive process on my part (or any qualitative researcher) in which on the one hand, I had to respect the data and use category labels close to the language of the teachers, and on the other hand, the ideas for the categories were derived from my knowledge of this field (Smith & Osborn, 2003). In this sense, the thematic analysis, like other inductive analyses, emphasizes that the analytical process is dynamic with the researcher playing an active role in the process. It is therefore necessary to reflect upon and reveal my personal assumptions. This, and other issues of trustworthiness, is discussed next.
3.6.5.3 Validity, reliability and trustworthiness discussion
Validity and reliability of Part C of the Perception questionnaire and trustworthiness of the qualitative interview data is discussed in this section.
140
Perception questionnaire. Ensuring the validity and reliability of Part C of the Perception questionnaire was similar to that for Parts A and B of the same questionnaire (see sections 3.6.2.3 and 3.6.3.3) and is briefly reiterated here. The question in Part C of the Perception questionnaire was vetted for face validity with four science teachers. Interviews with these teachers after they completed the questionnaire revealed that the questions were clear and answerable. These pilot teachers were asked to fill in the Perception questionnaire again for test-retest reliability check a week later (Feder, 2008; Trochim, 2006). Although this was a very small pilot size, there was near perfect test- retest reliability for all four pilot teachers; one of the teachers chose three reasons, instead of his original four, during the retest.
Interview data. Four aspects of Creswell’s (2009) criteria for establishing trustworthiness of qualitative data analysis were fulfilled. First, there has been prolonged and persistent immersion in the field as a total of 30 hours was spent video-recording the lessons (between 5 to 6 lessons per teacher). These extended hours excluded the two additional lessons per teacher prior to actual recording. This facilitated honest responses from the teachers during the interviews as rapport was built during the immersion. Second, all teacher conversations were transcribed verbatim, providing rich, thick and exact descriptions of what they said. This allowed the teachers’ interviews to be captured in context, making subsequent interpretations more valid. Third, interpretations of the interview data were cross-checked with the teachers after data analyses to ensure accuracy of the interpretations. The teachers were asked if they agreed – and all did – with the findings. Fourth, the bias the researcher brought to the research was clarified.
141 Aside from what I gathered from the literature review about why complex systems learning and instruction might be difficult, I was also aware that I had prior assumptions about the state of complex systems learning and instruction in Singapore science classrooms based on my experiences as a science teacher in a local high school. I had a hunch that most teachers did not consciously incorporate complex systems into their instructional practices because the existing science syllabi and national assessments did not feature complex systems prominently. I was mindful not to let this personal bias influenced my interpretation of the interviews by using the teachers’ voices as much as possible when the themes were generated.