The following section discusses validity, reliability and triangulation, and outlines the strategies that will be implemented in an attempt to strengthen the basis of any findings of this research.
5.18a Validity
Validity describes the extent to which an account accurately ‘represents those features that it is intended to describe, explain or theorise’ (Winter, 2000, p597). There is a need for it to be addressed because, as Maxwell (1992) suggests ‘it has long been a key issue in debates over the legitimacy of qualitative research’ (p279). In quantitative research both internal and external validity are considered important. Strauss and Corbin (1990), however, highlight the need to deal with qualitative research differently saying:
‘the usual canons of ‘good science’ … require redefinition in order to fit the realities of qualitative research.’ (p250)
External validity is considered by Winter (2000) to be, ‘the extent to which the results can be generalised and thus applied to other populations’ (p605). For qualitative studies external validity is often considered
irrelevant (Winter, 2000; Cohen et al., 2011) as the research does not seek to generalise but, rather, to represent accurately the phenomenon being investigated.
Generalisability to the wider population in quantitative research can be interpreted within qualitative research as comparability and transferability (Lincoln and Guba, 1985; Eisenhart and Howe, 1992, p647). This can be described broadly as the potential for findings to translate from one research setting to other specific ones rather than a wider population. Schofield (1996) suggests that in order to do this the researcher should provide ‘a clear, detailed and in-depth description’ (p200) but Lincoln and Guba (1985) argue that, ‘it is not the researcher’s task to provide an index of transferability’ (p316) and that readers should judge this potential for
‘researchers should provide sufficiently rich data for the readers and users of research to determine whether transferability is possible. In this respect transferability requires thick description.’ (p316)
This position is reinforced by Denzin (1989) who states that:
‘To use thick descriptions for establishing credibility, researchers employ a constructivist perspective to
contextualise the people or sites studied. The process of writing using this procedure is to provide as much detail as possible.’ (p151).
It could be considered that generalisability in the widest sense is less important in qualitative research than consideration of the ‘particular situations to which findings might be generalisable’ (Bogdan and Biklen, 1992, p45).
Internal validity is important to the credibility of qualitative research. It ‘seeks to demonstrate that the explanation of a particular event, issue or set of data which a piece of research provides can actually be sustained by the data’ (Cohen et al., 2011, p183).
‘Internal validity relates to whether the findings or results of the research relate to and are caused by the phenomena under investigation and not other unaccounted for
influences.’ (Winter, 2000, p604)
LeCompte and Preissle (1993) suggest that internal validity, important in this qualitative research, can be addressed in a number of ways:
‘Using low-inference descriptors;
Using multiple researchers;
Using participant researchers;
Using mechanical means to record, store and retrieve data.’ (p338)
Bias can be viewed as influential within considerations of validity from two different perspectives; that of the researcher and that of the participants. In addition to the objectivity of the researcher, other aspects of their practice such as ‘depth, richness and scope of the data achieved, the participants approached and the extent of triangulation’ (Winter, 2000, p2) might all be influential. In terms of the participants themselves, their opinions, attitudes and perspectives together can contribute to a degree of bias. Gronlund (1981) suggests that validity should be seen as ‘a matter of degree rather than as an absolute state’ (p48) and Cohen et al. (2011) recognise that, ‘at best researchers strive to minimise invalidity and maximise validity’ (p179).
Cohen et al. (2011) say of ensuring validity:
‘The attempt to build out invalidity is essential if the
researcher is to be able to have confidence in the elements of the research plan, data acquisition, data processing analysis, interpretation and its ensuing judgement.’ (p198)
The above suggests that validity can be an issue at different stages of research. At the design stage, the inappropriate selection of a
methodology and subsequent research tools or the biased selection of researchers are examples of the possible problems. During data collection reactivity effects (discussed in Section 5.14), inter-rater reliability and the withdrawal of participants can be issues. Data analysis is susceptible to invalidity though issues such as the selective use of data or the making of inferences that are not supported sufficiently by the data. Finally, at the stage of reporting data the lack of contextualisation or use of data either selectively or unrepresentatively could be problematic. It seems that consideration of each of the stages is necessary in order to maximise validity.
Validity within interviews is threatened by sources of bias, described already in terms of characteristics of the interviewer and participants but
Additionally, leading questions might be used inappropriately, although Kvale (1996, p158) argues that they can be necessary in some instances. Power imbalance is a further issue that potentially affects validity in
interview situations, discussed already as part of the consideration of ethics (Section 5.11).
Observations are at risk of invalidity in the form of bias. The issues include the selective or deficient attention of the observer, interpersonal aspects such as what a researcher may like or dislike about particular participants, reactivity, recording decisions, and the potential problems associated with inference (Moyles, 2002, p179; Robson, 2002, p324-5; Flick, 2009, Chapter 17). The issue of validity in observation can also relate to the indicators selected to represent a particular phenomenon, for example, what might count as ‘vulnerability’ or ‘emotional anxiety’ and how this will be judged fairly by a number of different observers.
Minimisation of invalidity within this research will be achieved through selection of an appropriate methodology and research tools, use of a single researcher to conduct interviews and observations, participant observation, and peer examination of data, as will be discussed in Section 5.18d.
5.18b Reliability
In terms of the relationship between reliability and validity, Brock-Utne (1996) states that, ‘reliability is a precondition of validity’ (p614) and Lincoln and Guba (1985) agree suggesting that:
‘Since there can be no validity without reliability a demonstration of the former [validity] is sufficient to establish the latter [reliability].’ (p316).
However, establishing reliability within qualitative research can still be considered to be an important process. A number of authors (Winter, 2000; Stenbacka, 2001; Golafshani, 2003) have suggested that within qualitative research there are more appropriate terms for reliability, such as, credibility, confirmability, dependability and trustworthiness. Whereas in quantitative research reliability suggests that replication of results should be possible if
the same methods and sample are used, within qualitative studies the individuality of situations that restricts replicability is what provides the approach with its strength. Each study therefore needs to be considered credible.
‘In qualitative methodologies reliability includes fidelity to real life, context- and situation-specificity, authenticity, comprehensiveness, detail, honesty, depth of response and meaningfulness to the respondents.’ (Cohen et al., 2000 p119)
In qualitative research, therefore, reliability can be considered to be the match between what is recorded by the researcher and what actually happens in reality. This can be achieved through, ‘a degree of accuracy and comprehensiveness of coverage’ (Bogdan and Biklen, 1992, p48). The dependability of qualitative research can be strengthened in a number of ways (Lincoln and Guba, 1985, p108-9; Anfara et al., 2002; Golafshani, 2003, p601) including prolonged engagement in the field, the use of reflexive journals, triangulation, and independent audit of research methods.
Reliability in interviews can be strengthened through increased structure, by ensuring that there is consistency not only in the questions asked but in the wording used with each respondent (Silverman, 1993). The advantage of less structured interviews is, however, that respondents might potentially be better able to demonstrate their view of the phenomenon under
consideration. Hence, it seems that increased reliability in this respect might lead to decreased validity and therefore, based on the proposition that the demonstration of validity is sufficient to establish reliability (Lincoln and Guba, 1985, p316), the less structured interview would be preferable for qualitative research.
Reliability of interview data can be compromised through the data analysis process because even detailed transcripts that record silences and non- verbal behaviour can later be taken out of context.
Maximising reliability in observation data can be achieved through
‘habituation’ (Cohen et al., 2011, p474), where the researcher remains in a situation for so long that participants become used to their presence and behave in a natural way. Writing up field notes as soon as possible after events also strengthens reliability (Lofland, 1971, p104-6).
Reliability in observations can relate to the application of the ‘indicators’ such as ‘vulnerability’ or ‘emotional anxiety’ being applied consistently to all appropriate occurrences with no variation in interpretation. This can be problematic even for a single observer and to a greater extent between multiple observers. Statistical tests to measure the degree of agreement in inter-rater reliability can, however, be applied.
Reliability will be addressed in this study through the use of appropriate research tools such as a reflexive journal and semi-structured interviews. The latter will be conducted exclusively by the researcher, to minimise the decontextualisation effect of transcripts, and using consistently applied questions and language. An extended period in the field, including
interviews and observation will support the collection of comprehensive and accurate data. Triangulation through the use of multiple research tools and also relating to the data analysis, as outlined in Sections 5.18c and 5.18d, will also be implemented.
5.18c Triangulation
Triangulation is considered to be, ‘a strategy (test) for improving the validity and reliability of research or evaluation of findings’ (Golafshani, 2003, p603). Its importance in qualitative research is that it can, ‘control bias and establish valid propositions because traditional scientific techniques are incompatible with this alternate epistemology’ (Mathison, 1988, p13).
Two types of triangulation have been identified that bridge issues of validity and reliability. Denzin describes the ‘within-method’ type (1978, p301), checking reliability, and this is compared by Jick (1979, p602) to the ‘between-method’ kind, relating to validity. In short, ‘within-method’
triangulation essentially involves cross-checking for internal consistency or reliability while ‘between-method’ triangulation tests the degree of external
validity. Denzin (1970) also identifies types of triangulation that are
different to the methodological type outlined above. Relevant to education research, he suggests additionally time triangulation (including longitudinal and cross-sectional studies), space triangulation (where a number of geographically separate settings are investigated) and investigator triangulation (when more than one researcher investigates the same phenomena).
As a validity procedure, it seems that qualitative researchers regularly use methodological triangulation, aiming to provide corroborating evidence collected through multiple methods, such as observations, interviews, and documents in order to identify themes. The findings generated are valid because researchers go through this process relying on multiple forms of evidence rather than a single set of data in the study. It is essentially a strategy that can aid in the elimination of bias and allow the dismissal of plausible rival explanations such that a truthful proposition about some social phenomenon can be made (Campbell & Fiske, 1959; Denzin, 1978; Webb, Campbell, Schwartz and Sechrest, 1966).
However, in all the forms of triangulation, one basic flaw potentially exists. As Jick (1979) describes:
‘The effectiveness of triangulation rests on the premise that the weaknesses in each single method will be compensated by the counter-balancing strengths of another.’ (p604)
Whilst each method used will have strengths and weaknesses, triangulation supposes that the strengths are exploited rather than the weaknesses being combined to create a negative effect. Rohner (1977) identifies this potential difficulty stating that, ‘it is assumed that multiple and independent measures do not share the same weaknesses or potential for bias’ (p134).
Denzin (1978) counters this opinion stating:
‘The rationale for this strategy is that the flaws of one method are often the strengths of another: and by
combining methods, observers can achieve the best of each while overcoming their unique deficiencies.’ (p302)
For qualitative research there can be advantages to uncovering disparity between sources of data. Barbour (1998) suggests that:
‘in using triangulation of several data sources in quantitative research, any exception may lead to a disconfirmation of the hypothesis whereas exceptions in qualitative research are dealt with to modify the theories and are fruitful.’ (p353)
In this study elements of methodological within- and between-methods triangulation will be used. Multiple methods are incorporated in the research design including interviews and participant observation recorded in a reflexive journal. Within-methods triangulation will involve peer examination of data as outlined below.