understanding of people’s sense-making of PSE policy and how it manifests in practice. It has been noted that approaches aimed at answering such questions should take into account a variety of perspectives by triangulating data obtained from several sources (Pratt 1995; Shih 1998; Streubert and Carpenter 1999; Yin 2003). There are different approaches to triangulation, such as the use of different evaluative or theoretical perspectives on the same data set or the application of different methodologies. This study attempted to triangulate different data sources (Patton 1987), which aimed to reflect the different socio-ecological levels of influence and the interactions among them, as suggested by the theoretical framework that has informed this study (McLeroy et al. 1988; Spillane et al. 2002). For instance, processes occurring at the policy level were examined using an approach based on Critical Discourse Analysis principles to understand the meanings within and characteristics of policy texts, whilst interviews with implementers aimed to gain an insight into the interaction between policy context and situated influences on policy sense-making. Practice was examined using lesson observations and focus groups. In order to develop an understanding of how the content and processes of PSE lessons are described in policy documents and people’s narratives it was necessary to draw on qualitative approaches. However, a description of the proportion in
which the different types of pedagogic exchanges were present within lessons employed descriptive quantification.
Quantitative research methods facilitate the collection and synthesis of large volumes of data in a short time (Nutbeam 1998). However, they can limit opportunities for the researcher to capture the meaning of what is explored (Mercer 2010). Although qualitative approaches are more time consuming, they can offer in-depth explanations of phenomena (Holloway and Todres 2003). They can ‘recover themes that are embodied and dramatised in the evolving meanings and imagery of the work’ (Spillane et al. 2002, p. 78), where themes are understood as structures or components of a particular experience. Qualitative research methods can offer the same scientific rigour as quantitative methods if the study aims, criteria for selection, sampling processes and the methods of investigation are as well defined (Denzin 2002; Cohen and Crabtree 2008).
Combining qualitative and quantitative approaches offers an opportunity to take advantage of both approaches (Mercer 2010). It has the potential to provide a detailed understanding of what is researched and facilitates synthesis of complementary findings. The triangulation of methods following the use of differing techniques to explain the same variable can support the validity of conclusions drawn, where mutual confirmation of results can be shown (Gray and Densten 1998). Following a review of over 70 mixed methods studies, O’Cathain et al. (2007) have concluded that the combination of methods can provide ‘confirmation, complementarity, development and expansion’ (p. 86).
However, it is important to be mindful that triangulation of data should not be based on the assumption that ‘the aggregation of data from different sources will unproblematically add up to produce a more complete picture’ (Hammersley and Atkinson 1995, p. 199). Sayer’s (1992) distinction between intensive and extensive research methods has been criticized as vague and reflective of the challenges associated with the triangulation of qualitative and quantitative research approaches described elsewhere (Silverman 1985; Pratt 1995). Paradigms could be incompatible with one
another, and triangulation should not be seen as automatically reducing the bias introduced by inadequately chosen approaches (Duffy 1985; Blaikie 1991; Sim and Sharp 1998). In order to address these challenges, it has been suggested that the processes of how methods have been triangulated should be clearly described (Miles and Huberman 1984; Oberst 1993; Shih 1998).
A number of previous mixed- methods studies have been criticized for reporting findings in parallel without integration (Bryman 2007). The separate reporting of mixed methods findings may lead to disproportionate representations of either approach (O'Cathain et al. 2008). Although there are approaches for combining a variety of qualitative and quantitative research methods respectively (Moher et al. 1999; Sandelowski and Barroso 2003; Mays et al. 2005; Finlayson and Dixon 2008), combining findings from mixed methods studies is more challenging. Thorne et al. (2008; Thorne 2010) have suggested that an agreed framework is needed to guide the synthesis of findings from across different components of mixed methods studies, and suggested a systematic clarification of the research objectives, the approaches taken to research each of these and the findings generated. Such a systematic approach to mapping study findings would highlight overlaps. Mapping would involve listing the research objectives and assigning a numeric code to each of these, and listing the approaches alongside their numeric codes. Key points summarising the findings from each study component are then listed, using the alphanumeric code to identify how they have been derived and which research objectives they respond to. This approach enables a synthesis of key messages and findings across all elements whilst highlighting discrepancies and agreements across the different datasets.
Although previously used for large multi-method research programmes, a simplified version of this approach has been applied to this study to map the research objectives to the methods employed and the data used (Table 1). It aims to identify overlaps and similarities among findings from different data sources, or indeed discrepancies between different study elements. The next section sets out the quality criteria and explains how these different study approaches attempted to generate high quality case study findings.
Table 1. Overview of research approaches and study objectives.
Objectives Approaches and data
To examine how the Welsh Government’s PSE policy framework represents practice (aim of PSE, the content of the lessons as well as the processes for its implementation within the school and the lesson).
Analysis approach based on critical discourse analysis principles of Welsh Government PSE framework and supportive documentation.
To identify how the PSE policy framework is reproduced at school level (representing the aim, lesson content and the processes for implementation within the school and the lesson) and how the organisational context of the school might influence this reproduction.
Analysis approach based on critical discourse analysis principles of school policies.
To examine how implementers represent the PSE framework and school reproductions of it.
Interviews with LEA level coordinators, head teachers, school-level PSE coordinators, and classroom teachers, interview transcripts were analysed thematically as described by Alexiadou (2001).
To identify from representations of PSE policy how individual, social, organisational processes impact on implementers’ sense- making.
Interviews with LEA level coordinators, head teachers, school-level PSE coordinators, and classroom teachers, interview transcripts were analysed thematically as described by Alexiadou (2001).
To describe the PSE lesson context such as classroom setting, the types of teacher-pupil exchanges during lessons including the proportion of competency-focused pedagogic exchanges.
Lesson observations (physical setting characteristics, also in case study diary notes, and lesson recordings). Lesson observation notes and sketches about physical setting characteristics as well as case study diary were analysed thematically. Lesson transcripts from recordings were transcribed and content analysed to generate a quantitative description of the distribution of different types of exchanges.
To understand pupils’ experiences and perceptions of PSE lessons, and the extent to which these reflect understanding and application of PSE relevant knowledge.
Pupil focus group transcripts analysed thematically as described by Alexiadou (2001).