8.1. Threats to validity
Lincoln and Guba (Cited in Robson, 2002, p. 172) indentify three broad threats to validity in flexible design research studies:
reactivity: the way in which the researcher’s presence can interfere with the behaviour of people in the setting being studied
respondent bias: respondents withholding information or trying to provide the information they think the researcher wants to hear
researcher bias: the assumptions and preconceptions that the researcher brings to the research
A number of strategies can be used to reduce these threats (Lincoln and Guba, cited in Robson, 2002, p. 172-4; Stake, 1995; Yin, 2014) and the applicability of these to the current study is discussed below.
8.2. Prolonged involvement
Familiarity with the researcher can encourage trust and help to reduce reactivity and respondent bias. For reasons discussed above, it was not viable to spend long periods of time on the premises but a number of factors helped to compensate for this. I was already known as a local resident and a customer of each of the businesses, I spoke on a number of occasions with the owner of each business to establish trust and I kept interviews informal, placing emphasis on learning about the ways writing was used in the
business from the interviewees. However, for some people writing is a sensitive subject and it was possible some participants may have felt
uneasy, fearing that I would be making a judgement on their writing, and that this could have influenced their responses. To mitigate this possibility I drew on my experience of interviewing adult literacy learners to put participants at ease and ensure they were not made to feel uncomfortable.
8.3. Triangulation
All three threats to validity can be reduced through triangulation. I have already noted the opportunity case studies provide to use multiple sources of evidence. Yin (2014) states that greatest advantage this presents is the
‘development of converging lines of enquiry’ which can ‘corroborate the same finding.’ In relation to this ‘triangulation’ of data, Yin (ibid) distinguishes between facts or events supported by different sources of data,
(convergence) and multiple sources of data that address different facts (non-convergence) (Yin, 2014, p. 121). He argues that the collection of
‘convergent evidence helps to strengthen the ‘construct validity’ of the case study (ibid).
Flick (2004) notes that triangulation is no longer seen primarily as a validation strategy. In a later publication he suggests:
It can be used to describe and formalize the relation between qualitative and quantitative research ... and as a strategy for
promoting the quality of qualitative research ... But it can also be an approach to do qualitative research in an appropriate way.
(Flick, 2009, p. 444)
He discusses four different forms of triangulation originally identified by Denzin. Below I take these in turn, making reference to my own data and analysis.
Triangulation of methods can be ‘within methods’ or ‘between methods’. In terms of the former, my own study uses questions of differing kinds within interviews and interviews of different kinds: both
more formal, scheduled interviews and short conversations related to specific activities, on the job.
In terms of triangulation ‘between methods’, I do not compare quantitative with qualitative methods but do combine data from
‘reactive procedures’ such as interviews, in which the researcher is a part, and ‘non-reactive procedures’ such as the analysis of documents (Marotzki, 1995).
Triangulation of theories. This can serve to provide multiple
perspectives on an issue. However, Flick (2004) warns of the need to take care when combining different methods and theories pointing out the need to be aware that ‘different methods will have developed against different theoretical backgrounds and the need to be alert to
‘incompatible epistemological assumptions’ that different methods may carry with them. (Flick, 2004, p. 181). In this study I draw on approaches to the study of writing that start from practices and on approaches that start from the study of texts (Barton, 2001). However, I have ensured that the frameworks I have chosen share a common epistemology.
Triangulation of investigators was not available for this study. I did,
however, take advantage of opportunities to share parts of my data and my interpretations with peers, on a number of occasions. This provided a check on my interpretation and afforded new insights (Robson, 2002).
8.4 Other strategies
Other strategies to reduce the threats to validity include checking
interpretations with respondents. I did not share written accounts with the participants but, where possible, I used my visits to the workplaces to check my interpretations of previously collected data through conversations and by observation. I checked some questions with the builder and with Maureen at the garage over the phone and with the hairdresser in the course of my appointments. Another strategy involves the deliberate search for negative cases. I did not attempt this but it is worth noting that there were features of the uses of writing that were common across three cases but absent in a fourth. The most significant example was the absence of the need for the builder to share information with others when he was working alone which, in turn, was linked with the fact that his was the only business that did not make extensive use of proformas. This alerted me to a more complex pattern of factors than I might otherwise have considered when undertaking my cross case analysis.
8.5 The Role of the Researcher
Stake (1995, pp. 40-42) underlines the responsibility the researcher holds for ‘interpretation’ in qualitative research.
Standard qualitative designs call for the persons most responsible for interpretations to be in the field making observations, exercising subjective judgement, analysing and synthesising, all the while realising their own consciousness.
The researcher inevitably brings to the research her own experience, presuppositions, feelings and beliefs. To counter researcher bias, I
attempted to adopt a reflexive approach, aiming to identify and acknowledge potential areas of bias and to ‘bracket’ these personal responses in
undertaking the data collection and analysis (Lincoln and Guba, cited in Robson, 2002, p. 172-173).
8.6 Reliability
Unlike a quantitative study, the reliability of a qualitative study cannot be evaluated by its repeatability. However, the ‘trustworthiness’ (Robson, 2002, p. 168) of a study can be enhanced by the transparency of the data
collection methods and approaches to analysis employed. I have
endeavoured to maintain a clear audit trail for my data and in writing up my study have made explicit how my conclusions have been reached. I have also endeavoured to ensure that all claims I make are supported by the presentation of relevant evidence.
I attempted to strengthen the reliability of the research through the
construction of interview guides for both owners and employees. These were not used slavishly but served to keep the interview on track and ensure all significant questions were included on each occasion. They also provided reminders to myself, as interviewer, regarding the need to check the interviewee’s understanding of the focus of the research, the process and what the interview would cover. I have been continually mindful of the limitations of the study and have ensured that I have made these explicit.