Qualitative research has sometimes been criticised for its lack of rigour inherent in the process of data collection and interpretation (Burns, 2000). Research rigour, by and large, lies in the extent of validity and reliability a research study claims to achieve. Validity (including internal and external validity) and reliability are
133
rooted in the positivist view of research (Kirk & Miller, 1986), but when it comes to qualitative research, these terms are defined and interpreted from interpretive view of research (Lincoln, 1995; Lincoln & Guba, 1985), as summarised in Table 4.5.
Table 4.5: Comparative terms in quantitative and qualitative research (Davis, 1992)
Quantitative Qualitative
Internal validity Credibility External validity Transferability
Reliability Dependability/ Consistency
Internal validity in quantitative research concerns whether a research study actually measures what it is supposed to measure in order to achieve the most truthful results. In other words, how well the results match the reality (Burns, 2000). However, in qualitative research, as Davis (1992) notes, it is more important that “findings and interpretations are credible to those being researched” (pp. 605-606). Thus, the ‘truth value’ lies in the trust participants have for the researcher, the honesty of their answers, the researcher’s understanding of the context and culture, and the use of time and methods to triangulate the data. Also, in case studies, it lies in the researcher “giving a detailed account of how they carried out the study” (Burns, 2000, p. 476). Internal validity in qualitative research can be achieved in various ways. According to Davis (1992), credibility can be enhanced by using “procedures such as prolonged engagement, persistent observation, and triangulation” (p. 606).
In this study, different strategies were used to enhance credibility. Firstly, following Burns (2000), Edge and Richards (1998), to enhance the ‘trustworthiness’ of the study, a detailed account of how this study was conducted is provided in this chapter. This account includes the process of data collection, changes during data collection, how data were managed and stored, and how data were analysed. This account necessarily provides readers with a research-related story of what was going on during the process of undertaking this research. Secondly, I spent roughly five months working closely with the teachers. Such a prolonged engagement (Davis, 1992) gave me sufficient opportunity to get to
134
know the teachers, understand their practices and cultures of teaching, and to build trust. This engagement together with my own experience of the context as a member of the community allowed me to judge what was true and honest and what was not in teachers’ statements.
Furthermore, the process of triangulation suggested by many methodology writers (e.g., Burns, 2000; Cohen et al., 2000; Davis, 1992; Denzin & Lincoln, 2000) was applied. This study adopted a multi-method approach to data collection, which allowed me to view the nature of inquiry from different sources and viewpoints. Since triangulation can take several other forms, such as time triangulation (or prolonged engagement – see above), multiple investigators, and data collection from multiple participants (Davis, 1992), the use of different methods to collect different sources of data is claimed to be “the heart of qualitative research’s validity” (Davidson & Tolich, 2003, p. 34). However, as well as the triangulation of time (prolonged), data sources and research methods, this study’s validity was enhanced by collecting data from different participants (i.e., multiple case studies) in order to validate data across participants.
Another concern regarding validity of a case study is the reactive issue (Burns, 2000), concerning the role of the researcher during the process of data collection and interpretation. This means the researcher’s presence “may affect the behaviour of the observed unit” (Burns, 2000, p. 447). In this study, I was fully aware of how my presence may potentially affect the validity of the data. Therefore, it was important to provide a detailed account of “what the relationship or history was between the researcher and the researched, and what bearing that relationship had on the research process or interpretations” (Duff, 2008, p. 118) so as to make explicit any possible biases derived from the researcher’s presence and activities. As mentioned in 4.4.3.1, of the eleven teachers, two were my college friends, six of them I knew as friends of friends, and since it was such a small town, I occasionally met them in social settings. I got to know the remaining three teachers for the first time during the period of participant recruitment. However, academically all the teachers knew me in the role of a university lecturer and occasional teacher trainer, although until that time I had mainly worked as a teacher trainer to lower secondary school teachers. I participated as a teacher
135
trainer in one VTTN workshop, which involved two teachers in this study, in 2006, and one textbook training workshop for Year 12 textbook in 2008, which involved all the teachers in the province. Although I had clarified with the participants my role as a researcher concerning this particular project, it may be the case that the teachers regarded me as an expert in terms of language teaching. This factual situation explained why I had to frequently remind the teachers that I would like to observe their normal lessons – the type of lessons they practised routinely in their own classrooms rather than ‘observed’ lessons, which required special preparation and technology use, as perceived by the third school’s principal mentioned in 4.4.3.2. This also explained why I chose such methods of data collection as lesson planning sessions, non-participant observation and focus groups to minimise my role during the process of data collection.
However, I admit that my role in the process of data collection may still affect, to a certain extent, the data collected. For example, two teachers chose to use PowerPoint presentations in one of their observed lessons, which, through my experience with the teachers and understanding of the context, was not often the case in normal practice. Therefore, in my interpretation of the data, being aware of the issue, I have tried my best to guarantee that the findings were as trustworthy as possible. In doing so, sometimes I had to look behind the scene relying on my experience and understanding of the situations, as well as checking back and forth through various sources of data in making conclusions about my interpretation. Apart from such particular circumstances, I believe that my participants provided me with data as truthfully as possible.
External validity in quantitative research involves insuring that research findings are replicable. According to Davis (1992), external validity is established when “the findings can be generalized to other contexts and/or subjects” (p. 606). This construct is alternatively referred to as generalisability (e.g., Lincoln & Guba, 1985). In qualitative research, and especially in case studies, “external validity is not of great importance” (Burns, 2000, p. 476). The focus of a qualitative case study is on the characteristics of the case, i.e., its particularity (Stake, 1988). In qualitative research, researchers attempt to claim transferability (Davis, 1992) or relatability (Bassey, 1981) rather than seeking external validity. As such, a
136
qualitative case study may be transferable to other contexts or times depending on the reader: that is, the reader decides to what extent the findings of the study are applicable in their own situations (Burns, 2000). Therefore, it is the researcher’s responsibility to provide a rich, detailed description so as the reader can determine the extent of transferability (Davis, 1992). Like many other case studies, the present study also aims to focus on transferability. Although teachers are different, among themselves, between schools, and across provinces in Vietnam, they may share similar characteristics, such as using the same textbooks and working under similar conditions. The results of this study, therefore, may be transferable to other contexts in Vietnam.
Reliability is concerned with the extent of consistency the results of a research study produce. In other words, are the results replicable (Davis, 1992)? In quantitative research, reliability is assured by the use of testing instruments to make sure that results are stable, consistent, and predictable. However, in qualitative case study research, Burns (2000) argues that “it is impossible to establish reliability in the traditional sense” (p. 475). This means that in qualitative research, testing instruments or measures are not used to seek reliability. In fact, Burns (2000) and Davis (1992) argue that instead of reliability, qualitative researchers focus on dependability, the extent to which “the results make sense and are agreed on by all concerned” (Burns, 2000, p. 475). Different ways of enhancing dependability in qualitative research are identified, including triangulation (Burns, 2000; Davis, 1992), peer debriefing, member checks, inquiry audit (Lincoln & Guba, 1985), and the researcher reporting any possible biases that occurred during data collection and analysis (Burns, 2000). In the present study, dependability was enhanced by triangulation (as stated above), peer debriefing, inquiry audit, and close examination of possible personal biases. Peer debriefing in my study involved working closely and frequently with supervisors during the process of preparing the research proposal, data collection, and data analysis. This procedure resulted in critical analysis of the methods chosen, the data, and the interpretations made. Inquiry audit was made through exchanging data extracts with a colleague researcher to authenticate my interpretation to make sure that with the same set of data, different (qualitative) researchers would interpret to yield similar results. Also, as noted in 4.4.8, the analysis of the data in
137
this research included repeated analysis and re-analysis of sources of data. This means an extract of data was analysed at least twice at two different times. This process can be regarded as another way to enhance the dependability of the research.
I was aware of the possible personal biases that might have occurred during data collection and analysis. The first could be due to the extent of my experience as a researcher, represented in the way I asked questions in stimulated recall sessions and the personal reflection in the observation notes. However, as my data collection process proceeded, I could observe there was improvement in such issues, such as my using more probes than closed-ended questions in stimulated recall sessions. This awareness of the initial limitation was seriously taken into account during the analysis of the data. Frequent checking and re-checking of information across different data sources over time were carried out to validate the accuracy of conclusions. Also, particular care was taken in judging the questions I used to identify whether bias in participants’ answers could be affected by the questions I asked. Secondly, despite the advantages I may have regarding understanding of the culture and context, my role as a cultural ‘insider’ could sometimes hinder me from investigating in-depth the relevant issues during data collection and interpreting the data in an objective way. In several circumstances, I was likely to take some interesting issues such as ‘the role of teacher in English classes’ for granted, and thus necessarily missing some valuable data that may contribute to the overall quality of the study. My role as the cultural insider also affected the process of interpretation. In the initial stage of data analysis, I sometimes felt that the data did not provide me with enough information to analyse, and that the data represented mostly commonsense circumstances. Therefore, I had to frequently take a step back and look at the data as an outsider so as to make the familiar strange (Mannay, 2010; Mercer, 2007). This study had proposed to carry out member checking, i.e., having participants check on the information collected. However, due to the tight schedule of data collection, and the frequent power cuts at the time, little transcription and summary of data was made in the field for the teacher participants to check. Furthermore, only three teachers in this study had access to email, but rarely checked their mail based on the common practice that teachers in the contexts do not use email for work
138
purposes and rarely for personal communication. Due to this situation, the idea of member checking was abolished. While it was impossible to collect further data and seek clarification from participants, the potential problems of ‘cultural insider’ were further reduced by discussing results with supervisors and other PhD candidates within the research group that I participated in throughout the course of the study.
4.5 Summary
This chapter has provided brief accounts of research paradigms, qualitative research, and case studies, followed by detailed accounts of the present study. By reviewing the research paradigms and the nature of qualitative research, and given the claim that teachers’ beliefs and practices are context-bound, a qualitative case study was chosen as the most appropriate design for the present study.
This chapter presented details about a series of TBLT workshops, the preliminary studies, followed by issues of sampling, gaining access and recruiting participants. After providing detailed information about the eleven teacher participants and considering ethical issues, the chapter considered the methods of data collection, data management and analysis. In short, the present study used lesson planning sessions, observation, stimulated recall, and focus groups as methods of data collection. It employed grounded theory approach for data analysis (Charmaz, 2006) in two separate layers of analysis.
The issues of validity and reliability have also been considered. Overall, it is
hoped that I have provided sufficient information about the present study so as to allow for a comprehensive view of what had happened concerning the process of designing methods, collecting, managing and analysing data.
The next chapter will present an analysis of a unit from the textbooks the participant teachers were using.
139
CHAPTER FIVE
5 ANALYSIS OF A TEXTBOOK UNIT
Because this case study takes a socio-cultural perspective, the investigation into teachers’ beliefs and practices needs to take into account full contextual environment. This chapter presents an analysis of the materials the participant teachers were working with. Specifically, it provides an overview of the textbooks and brief analysis of one unit in one of the textbooks, as a contextual factor from which teachers’ beliefs and practices could be more thoroughly understood.