As this study employed a mixed method design, the trustworthiness of both the quantitative and qualitative strands of data and the related issues when integrating the two methods in this research design needed to be considered (Creswell & Clark, 2011). This section first discusses the trustworthiness of the quantitative approach concerning its reliability and validity. A description of the parallel issues for the qualitative approach with respect to the credibility, transferability, dependability, and confirmability (Lincoln & Guba, 1985) is then provided. Finally, the strategies used to address the potential threats relating to the merging of the two strands of data are presented.
Quantitative issues
Reliability and validity
Reliability and validity are the two concepts associated with the trustworthiness of quantitative research. Reliability refers to the consistency, suitability or repeatability of the research instrument (Johnson & Christensen, 2008), while validity refers to “the correctness or truth of an inference” (Johnson & Christensen, 2008, p. 253). Due to the practical conditions and the specific purpose of the quantitative approach of this study, the strategies employed to validate the quantitative data of the study included seeking comments from six colleagues of the researcher on the suitability of the questionnaire items with the study focus prior to the pilot, and conducting a questionnaire pilot with seven individuals who had the characteristics of the study participants. The afore-mentioned practices primarily aimed to discover any ambiguous and unsuitable questions that might have negative impact on the reliability and validity of the study.
Qualitative issues Credibility
Credibility is about determining the congruence of the findings with reality (Tracy, 2010, p.842). Two strategies of triangulation and participant feedback were employed to promote the credibility of the study results. Triangulation refers to the examination of the information
or conclusions through the use of multiple data sources or procedures (Johnson & Christensen, 2008). Among four types of triangulation, including data triangulation, methods triangulation, triangulation through multiple researchers and theory triangulation, I adopted the first two types to enhance the trustworthiness of the study, (i.e., data triangulation, and methods triangulation). Firstly, data triangulation involved the use of the data sources drawn from 60 study participants from different groups of the institutional staff in two different vocational colleges. It was also involved with the use of multiple follow-up individual interviews with eight participants. With respect to the method triangulation, the quantitative and qualitative methods were combined in this study to complement each other for better addressing the research questions.
Participant feedback or member checking involves the process of the researcher sharing the data analysis and interpretation of the study data with the actual participants for verification and insight (Johnson & Christensen, 2008). Because of the time constraints of the study, I only applied one part of this process to obtain truthful qualitative interview results. The preliminary data analysis and interpretation were not sent to the interviewees for their feedback. However, the transcripts of the interviews were dispatched to the interviewees within two weeks after the interviews for their review and amendments if necessary. Two participants returned the transcripts with minor changes while the six others totally agreed on the whole transcripts.
Dependability
Dependability in qualitative research refers to the consistency of the research findings (Onwuegbuzie & Johnson, 2006). Dependability requires a researcher to watch out for his or her bias, and to be transparent about the process and procedures of conducting the study. In relation to the research bias, I was mindful that my experience of working in the DVTA – the accrediting body might affect my understanding of the research problem under investigation. For instance, my assumption prior to the study was that some of the external reviewers might not be equipped with sufficient knowledge and skills to complete their tasks and this assumption might influence the evaluation of the policies and practices regarding the external review. To cope with this problem, throughout the study, I was actively engaged in self- reflection in order to monitor and control my potential biases. In addition to make it easy for readers to determine the consistency of the study results, the methods of participant recruitment, data collection and data analysis were also discussed in detail in the study report.
Transferability
Transferability in qualitative research indicates the extent to which the research findings of a study can be applied to other settings (Tracy, 2010). According to Lincoln and Guba (1985), a researcher cannot specify the transferability of their study but he or she can provide sufficient information so that a reader then can determine its relevance for their purposes. In this study, to provide the basis for the researchers to judge the transferability of the study, I described in detail the context of the study, especially, what “accreditation” means in the context of vocational training in Vietnam and the history of accreditation development
Confirmability
The confirmability of research refers to “the extent to which the characteristics of the data, as posited by the researcher can be confirmed by others who read or review the research results” (Bradley, 1993, p.437). In this study, my supervisor looked at the raw data, my analysis notes and personal notes, and supported me in interpreting the data results. In addition, I have provided sufficient detail of the methods and procedures undertaken to enable replication of the study in the future.
Mixed method issues
According to Creswell and Clark (2011), the potential threats when merging data in a mixed research design could be identified in three different areas, including data collection, data analysis and interpretation of a study. As noted by the authors, a specific issue that might compromise data collection might involve inappropriate sample selection for quantitative and qualitative data collection. In this study, the samples for quantitative and qualitative data collection were from the same population; the interviewees were selected from those who responded to the questionnaire. Further, to eliminate the potential threats to the validity of the study resulting from collecting two types of data that do not address the same topics, the questions in the interviews of this study were guided by the survey results, ensuring that both strands of data focused on the research problems.
With respect to data analysis, Creswell and Clark (2011) highlighted the potential threats involving the inadequate and unclear analysis of data that becomes the focus of the follow-up procedures. The initial analysis of the survey in this current study produced clear and detailed description of the significant findings which were followed up in the interviews. Furthermore, the priori codes as the themes identified from the survey data analysis were used in analysing the interview data.
Regarding the interpretation of data findings, Creswell and Clark (2011) suggested that mixed-methods researchers should consider the potential threats resulting from giving more weight to one form of data than the other, not taking full advantage of the two data sets or leaving unattended contradictions. In this study, the quantitative and qualitative strands of data had equal weights in answering the research questions. Any contradictions between the quantitative data and qualitative data findings were noted and explanations were sought in the follow-up interviews. The findings of the survey and the interviews were integrated in discussing the final outcomes of the study. The remaining contradictions were reported in the data findings.