One of the problems that confront researchers in the field of language learning is that the efforts learners make to learn and use a language are mainly unseen, taking place in the learners' minds (Nunan, 1992). The difficulty of investigating listening and the complexity of this construct is well-established, calling for a mix of methods to deal with these major issues. In fact, many researchers argue that mixing methods is indispensable in classroom research, due to its highly complex nature (Dörnyei, 2007). A mixed methods approach helps, to use Dörnyei's analogy, put “flesh on the bones” (ibid: 45). The bones in the case of mixed methods research is data driven from the quantitative phase of the study, whereas the flesh is data resulting from the qualitative part of it. Based on the aforementioned reasons, along with the type of data required to answer the research questions, my study followed a mixed methods design.
The term mixed-methods research is used to refer to the combination of both qualitative and quantitative methods within one single study (Dörnyei, 2007). It is defined as a type of research that combines the elements of qualitative and quantitative approaches in a single study (Johnson et al., 2007 , p. 123). Mixed methods research is considered as the third major research paradigm which helps bridge the division between qualitative and quantitative approaches (Johnson and Christensen, 2004, Johnson et al., 2007 ). The main philosophy that supports mixed methods research is pragmatism (ibid), which suggests that the most useful approach to any investigation is actually “what works” to answer the research questions (Cohen et al., 2011). In other words, the bottom-line is that “research approaches should be mixed in ways that offer the best opportunities for answering important research questions” (Johnson and Christensen, 2004, p. 16). Mixed methods research is in fact “an attempt to legitimate the use of multiple approaches in answering research questions, rather than restricting or constraining researchers‟ choice” (ibid: 17).
The fundamental principle behind mixed methods research, Johnson and Christensen (2004) say, is that multiple datasets are collected through the use of different
strategies, methods and approaches in a certain way that the combination would result in “complementary strengths and non-overlapping weaknesses” (p. 18). In many instances, the goal of mixed research is expanding understanding, rather than searching for corroboration (ibid). For a study to be regarded a mixed-method design “the findings must be mixed or integrated at some point” (ibid: 20). Johnson and Christensen (2004) compare the act of mixing methods in research to the use of “several flawed fishing nets” together as an attempt to come up with a new stronger net which functions well in spite of the problems existing in each net individually (p.162). This analogy entails that the use of more than one instrument to collect data for a single study helps overcome the shortcomings present in each single method. However, even though mixing methods is regarded as “an excellent way to conduct high-quality research”, it has to be done skilfully to ensure that the methods complement the strengths of one another and that their weaknesses do not overlap (ibid).
Researchers justify the use of various research methods by stating that “one method alone cannot provide adequate support” (Mackey and Gass, 2005, p. 181). Evidence, Johnson and Christensen (2004) say, is “often greater when you employ a logical mixing strategy” (p. 163). Further, using a mixed-methods approach is helpful in understanding complex constructs, such as classrooms, because it “can broaden the scope of the investigation and enrich the researcher‟s ability to draw conclusions” (Dörnyei, 2007, p. 186). A mixed-methods approach also leads to a multi-level analysis (ibid). Dörnyei justifies this point by saying that “words can be used to add meaning to numbers and numbers can be used to add precision to words” (p.45), hence the methods complement one another. Once the findings support and confirm one another, the validity of the research results is improved (ibid). Finally, studies that are based on mixed methods, as opposed to “mono-method” studies, are more likely to appeal to a larger audience, including second language teachers, who are not researchers themselves but could still benefit from the qualitative side of the study (ibid).
The design of this study was a QUAN + qual, mixed-method one; the quantitative data was given more dominance and the two were conducted concurrently, rather
than sequentially. This design is useful for embedding a qualitative element within a primarily quantitative study (Dörnyei, 2007). In my study, the qualitative data was used to uncover the metacognitive knowledge of participants in the experimental group in more detail and evaluate the intervention from their viewpoint. Quantitative data was given more dominance because it was used to track the development of the experimental group, and to compare the results of the two groups. In a concurrent design the quantitative and qualitative methods are used separately and in a parallel way; one method does not influence the operationalization of the other, and the results are integrated in the interpretation stage (ibid). The major purpose of the concurrent design is broadening the research perspective and consequently reaching a more general picture of the issue investigated or finding out how various findings support or complement each other (ibid). I collected quantitative and qualitative data sets concurrently and analysed them separately. Then I mixed the two databases during the interpretation stage. To conclude, researchers investigating “a construct as implicit as listening” should, Vandergrift (2010) suggests, attempt to use a mixed methods approach to gather “convergent data” (p.168). Before presenting the research design, I establish the ethical considerations I took into account when designing the study. This is the focus of the next section.