There are different kinds of validity ranging from content validity to predictive validity (Cohen et al., 2000, p. 105). Cohen and his colleagues acknowledge that validity is now a complex and nuanced aspect of contemporary research but also point out that “Validity is an important key to effective research. If a piece of research is invalid, then it is
worthless…” (p.105). They elaborate that validity should not be seen as some absolute state which all research must attain, but rather, as a question of degree. Research should attempt to minimize invalidity whilst maximising claims to be valid.
Maxwell (1996, p. 86) also emphasises that validity is not a gold-standard, it is a goal than a product. Further, “validity is also relative: it has to be assessed in relationship to the purposes and circumstances of the research rather than being a context-independent property of methods or conclusions”. With this in mind, an attempt has been made to
ensure the validity of the current research by examining a range of possible threats to validity and how the research design attempts to deal with these threats.
Validity threat 1: Limited perspective
This threat is in reference to the possibility that the researcher could have adopted a limited perspective on the research phenomena and as a consequence has not ‘attended’ to important issues that this perspective fails to highlight. In response to this threat, use of multiple methods of data collection and analysis (that is, triangulation), was adopted. By using interview survey, participant observations, and focus groups, an attempt has been made to examine the phenomena of interest (i.e. impact of IRI on teachers’ beliefs and practices) from disparate perspectives with each of these different perspectives providing a checking mechanism on the others.
Validity threat 2: Deliberate deception by research participants
This threat is in reference to the possibility that the research participants deceived the researcher, either deliberately or inadvertently. This is particularly applicable to the interview survey. As part of the interview survey, participants could attempt to deceive the interviewer by providing responses that they felt avoided them looking ‘bad’ or ‘incompetent’ (Maxwell, 1996, p. 93). Also, participants could feel inclined to provide responses that they feel the interviewer expects. The design of this research is intended to minimise such threats. For example, the use of external verifications through cross- checking findings against other related other data gathered through the EDC’s monitoring and formative evaluation processes. In addition, the data was cross-checked against current research on Malawian primary education, for example, the ones carried out by the
Malawi Education Institute (MIE), Centre for Educational Research and Training (CERT) and other data gathered through the EDC’s monitoring and evaluation instruments. Furthermore, findings were presented to a panel of twelve PEAS as the process of data analysis was winding up. Another mechanism built in the process of conducting interviews was that each of the interview participants was given the
opportunity to make their responses confidentially (thus allowing them an opportunity to make critical comments). Furthermore, a ‘classroom environment pupil satisfaction and achievement indicator’ (CEPSATI), was adapted for pupils to fill in as a checking mechanism on the responses from teachers about the classroom learning experiences.
Validity threat 3: Choice of research sites is atypical
This threat is about whether the chosen research sites or subjects are atypical of the population as a whole, in which case generalizations drawn could be invalid. In line with its aims, the participants in the current research have been identified according to the contextual focus. As the research is mainly about teacher practice in primary schools, teachers form the largest single group of participants in this study. Up until recently, studies on teacher education in Malawi indicate that three-quarters of teachers in rural areas are either under-or un-qualified (see Kadzamira et al., 1999). Since the IRI programme is implemented nationally and a majority of schools in the country are of rural background, systematic sampling was adopted to ensure an equal representation of rural schools and the population across the country. In this case, this threat is
counteracted, and it would be justifiable to claim external generalisation as a feature or outcome. The current research attempts to illuminate the particular responses within each
of the different individual teachers and their schools as they adopt IRI and the documented responses are most likely to be typical of the wider population.
Validity threat 4: The researcher’s interpretation does not reflect participant interpretations
This is concerned with the potential mismatch when the researcher’s interpretations do not reflect the interpretations held by research participants. This threat can arise for a number of reasons, for example: researcher bias or the imposition of an interpretive scheme by the researcher that does not reflect the local situation and premature closure. This threat is particularly relevant to research that is attempting to promote understanding as interpretation. One of the main ways of trying to refute this threat is by seeking
opportunities for feedback from research participants and others— what Maxwell (1996, p. 94) refers to as “member checks”. Within the context of the IRI project and the current research, this potential validity threat has been addressed through the use of feedback mechanisms built in to the evaluation processes used within the research. Follow up discussions and informal conversations with key participants meant that the outputs from evaluation mechanisms were subjected to verification by project participants. Informal verification was carried out primarily through ongoing focus groups, such as PEAS (and head-teachers) as discussed above, but also individual interactions with these.
Although these processes of verification provide a means by which the validity of aspects of the analysis could be assessed, the final responsibility in addressing this validity threat rests with the researcher through the way in which the empirical analysis and the
discussion of it is presented. According to Peshkin (1988) it is important to be aware of ‘our subjective selves’ and the role that this subjective self plays in research since being
aware is better than assuming one can be rid of subjectivity. Therefore being aware of ‘my subjective self’ here meant being aware of the qualities that would enhance this research as well as the researcher’s beliefs about the value of IRI in the professional development of teachers that could skew the interpretation of the data if the researcher were not aware of them. The suggestion by Eisner (1998) that each person’s history, and hence world, is unlike anyone else’s means that the way in which we see and respond to a situation, and how we interpret what we see, will bear our own signature. This unique signature should not be viewed as a liability but as a way of providing individual insight into a situation.