A researcher can never be guaranteed that his or her work is entirely valid and reliable; however, if the proper validity measures are applied in an adequate way, then the overall project should maintain a significant degree of integrity (Cohen et al., 2011). Considering the scope of this research project, a substantial amount of care was taken to ensure “honesty, depth, richness and scope of the data achieved” (p. 179). An inherent quality of the adopted mixed methods design is that the triangulation of quantitative data, qualitative data, and literature enhances the probability of validity and reliability. Triangulation is achieved through the utilization of the survey data, qualitative conversation data, and also the literature that has emerged from prior studies on the topic of study abroad intent.
There are certain threats regarding validity in mixed methods studies. Generally speaking, issues that could impact validity of mixed methods research include sample sizes of the quantitative and qualitative sample groups, whether the samples exhibit parallel units of analysis, the process of merging the results from the two phases, and how contrary or divergent results are explained (Creswell, 2015). In the current study, these criteria are satisfied by having a reasonably representative ratio of strong and weak intent members in both data collection phases. Also, qualitative results are later merged with quantitative results, as defined by the subscribed explanatory sequential design. In this particular mixed methods design (explanatory sequential), Creswell (2015) presents several questions that can help ensure validity:
• What quantitative results need follow-up?
• How will you select the sample of follow-up participants? • How will you develop relevant interview questions?
• How will you ensure that the qualitative data indeed explain the quantitative results? (p. 19)
The questions presented in this list are addressed throughout the current chapter. In short, the literature was referred to in identifying quantitative results deemed appropriate for follow up. Also, a sample of strong and weak intent participants emerged for the qualitative phase of the study, in which relevant literature was consulted in developing interview
questions. These interview queries were also designed to address the research questions, with multiple validity strategies employed to ensure that qualitative data offered insight into the survey results.
Simply because both qualitative and quantitative approaches are used in the study, it does not mean that the scope and rigor of each method is compromised or diminished. In fact, in regards to ensuring validity and reliability, the two conjoined methods are viewed as a united piece, and also in isolation, since qualitative validity differs from quantitative validity. In checking that the quantitative component of this study is reliable, many steps are described in this chapter, including the execution of a pilot study, focus groups, and seeking assistance from a professional translator to ensure that there were no glaring omissions or mistakes in the translation. As explained in the description of the quantitative phase, analysis was
conducted in SPSS that yielded a near-excellent degree of reliability for international posture as a whole, and significantly better reliability for the items that determined intent to study abroad.
In brief, qualitative validity means that “the researcher checks for the accuracy of the findings by employing certain procedures, while qualitative reliability indicates that the researcher’s approach is consistent across different researchers and different projects”
(Creswell, 2014, p. 201). This particular study employs validity strategies to assure the reader that findings are accurate. Through researcher positionality in the introduction (chapter 1), honesty is conveyed, which may be helpful to identify bias. Identifying bias through self- reflection “creates an open and honest narrative that will resonate with the readers” (Creswell, 2014, p. 202). Especially with the qualitative analysis, connections are made between opinion and how this might be influenced by factors such as nationality; profession, academic
experience, and culture. Additionally, another practice that can make a design more valid includes the presentation of negative or discrepant information. When appropriate,
unexpected or counter intuitive results are disclosed and discussed. To ensure the adequacy of the current study’s thematic analysis, it was examined through the lens of common pitfalls, described by Braun and Clarke (2006). First, the themes and extracts contribute to
preliminary analysis (chapter 4) followed by more detailed analysis (chapter 5). Second, the themes were not derived directly from the email questions themselves, but from the answers,
and how they related to elements of the theoretical framework. Third, while there is some overlap in themes (e.g. ought-self being found in multiple tables), they are found under discrete categories of the theory of planned behavior and it is explained how the theme contributes to the understanding of each category. Finally, care was taken to ensure consistency in the data extracts, the interpretation of the extracts, and the theory.
3.9. Summary
Considering the scope of the research and the applied theoretical framework, an explanatory sequential mixed methods design was chosen for data collection and analysis. For the initial quantitative component, a pilot was first conducted with 70 students from UoJ. Feedback from this process resulted in a final survey instrument, which 311 first-year
students completed. Following this, analysis was done on the quantitative data, leading to a series of questions that were presented to 10 volunteers of the qualitative email questionnaire: six with strong intent to study abroad and four with weak intent. After receiving responses and making follow-up enquiries, thematic analysis and the theoretical framework helped define of themes and codes that will provide the basis for answering the research questions. In the next chapter on findings and analysis, results of the investigation are organized and presented based on the three elements of the theory of planned behavior, that is, (1) behavioral attitudes, (2) subjective norm, and (3) perceived behavioral control.