Creswell and Plano Clark (2007) suggest that there are three critical decisions that determine the structure of a mixed methods design. The first is “the timing
decision”, the second is “the weighting decision”, and the third is “the mixing decision”.
5.7.1 Timing and ordering
The timing and ordering decision relate to when the qualitative and quantitative data are collected and whether they are collected simultaneously or
sequentially. In some research designs, qualitative data may be collected to inform quantitative survey design. In such a scenario, the qualitative data would be collected and analysed before the quantitative data. Here the former are often viewed as generating questions to be tested more definitively.
Alternatively, qualitative data may be used to explore quantitative findings in more depth by looking at the processes that give rise to the observable
phenomena. Here, the former explore issues raised by a definitive result or statistical pattern. In such a scenario, qualitative data would be collected after quantitative data had been collected and analysed.
In our study, the quantitative data already existed in administrative health databases and only had to be linked rather than collected. It was our initial intention to analyse the quantitative data first, identify any problems with the implementation of the referral pathway or the DHSW intervention and then explore these and other issues in more depth through qualitative data collection. The quantitative data linkage was conducted by ISD and, as the datasets
required for this study were only a subset of several datasets that were being linked for the purposes of evaluating the wider Childsmile programme, the process took longer than anticipated. In the meantime, we used the results of the process evaluations and the available aggregated data from Childsmile monitoring reports to anticipate the kind of problems that might be identified through the quantitative analysis of the linked data. In the end, the qualitative and quantitative analysis took place simultaneously with the results of each illuminating the other.
The quantitative data available in the administrative datasets related to children who had a health visitor assessment between 1st September 2010 and 30th September 2012. These were the most recent data available at the time the data linkage occurred. The qualitative data were collected from June to
November 2013. Although the “observation” periods for the quantitative and qualitative data differ, we were confident that any changes to organisation structure that may have affected the way the intervention was targeted or tailored between September 2012 and mid-2013 would be mentioned by participants in the qualitative focus groups and interviews and could be taken into account in our interpretation of the data.
5.7.2 Weighting
The weighting decision relates to whether an equal or unequal weight, in terms of relative importance, will be assigned to qualitative or quantitative data. As previously mentioned, the approach I took to this research was to prioritise the research questions and then choose the most appropriate methods. On one hand, there were some research questions that could be answered in more detail with the qualitative data, while the quantitative data corroborated certain points. On the other hand, there were other questions where the quantitative data were able to provide a more complete and detailed overview of implementation while qualitative data offered some explication of the quantitative results. Neither qualitative nor quantitative data were given more weighting in terms of validity or reliability beyond the capability to answer each research question
sufficiently.
5.7.3 Mixing
The options for mixing suggested by Creswell and Plano Clark (2007), and
represented in the mixed methods literature, are (1) transforming the data types so they can be merged and integrated, (2) embedding one type of data within another, or (3)presenting the data separately, but connecting them together to answer the same, or similar, research questions.
I chose to present the data in a separate but connected way for three reasons:
(1) The quantitative and qualitative data related to slightly different time periods
(2) The variables in the quantitative data did not map directly on to themes I wanted to explore through qualitative data collection
(3) The weight assigned to each data type differed depending on the research question.
By connecting the data in this way, this mixed methods study took a convergent design (Creswell, 2011). This meant that, for many aspects of most of the research questions, there was ‘triangulation’ of evidence as one data type corroborated or challenged the findings of the other.
In mixed methods studies, qualitative data is often used to provide illustration for quantitative findings, adding breadth and depth to the analysis; however, we have taken a pragmatic approach and do not accept that qualitative data is subordinate to quantitative data (or vice versa). Rather, our approach has been ‘validatory triangulation’ which involves “using the degree of convergence between different data sources as an indicator of the validity of results” (Davies et al., 2003). In practice, this meant that we looked at the hypotheses
generated by the quantitative and qualitative data relating to how the DHSW intervention was being implemented and looking for corroboration. Where there was discord, a new hypothesis was generated and the data were examined again to test this hypothesis.