According to Kirk and Miller “reliability usually refers to the degree to which the finding of a study are independent of accidental circumstances of their production” (Kirk and Miller 1986;
81 Silverman 2006, 282). To increase the reliability of qualitative research Moisander and Valtonen suggest two broad criteria, first, to describe the data collection and data analysis in detail and second to ensure that the theoretical framework on which the analysis is based is clearly laid out (Moisander and Valtonen 2006; here Silverman 2006). Both of these approaches have been used in this study. Data collection and analysis were described in this chapter in section 3.3. The theoretical frameworks on which the analysis is based were discussed in the literature review and in this chapter in section 3.2. They will also be discussed in chapter eight.
Further, Seale (1999) argues that reliability is associated with ”low-inference descriptors”.
According to him this means
“recording observations in terms that are as concrete as possible, including verbatim accounts of what people say, for example, rather than researchers’
reconstructions of the general sense of what a person said, which would allow researchers’ personal perspectives to influence the reporting” (Seale 1999, 148) To collect low-inference descriptors, Kirk and Miller (1986) suggest that the observer keeps different types of notes. First, short notes are jotted while observing, second, more detailed notes are made soon after the observation session and third a field journal is kept on ideas and challenges that arise during the observation sessions (Kirk and Miller 1986; here Silverman 2006). These different types of notes were kept. But in addition the majority of interviews were also recorded and later transcribed by myself. In order to enable increase systematic note taking in the case of observing the operations of groups at their group meetings, a checklist was developed In this case three people took notes: myself, the research assistant and the local assistant-cum-translator and we discussed our notes immediately after the meeting to arrive at as full an understanding of what had happened as possible.
To achieve reliability in carrying out the interviews, great care was exercised that respondents understood the questions and that all the respondents understood the questions in the similar way. In order to ensure this, the interview instruments were pre-tested in the field and unclear questions clarified. Further the interviewers were trained on the instrument and in asking questions in appropriate ways. The interview instruments were devised in English, the research assistants undertook mock interviews of each other translating the questions into local languages who then answered the question first in the local language and then translated the question that she had heard back into English. In this way the research team were able to assess what aspects of the questions were unclear and revise how they were asking them.
82 Several methods have been suggested to assess whether findings are valid or trustworthy (Creswell 2003). First is triangulation; second, is to take the draft report back to the respondent and validate it through the feedback; third is to use “thick description” that provides the reader with detailed and convincing evidence; fourth is to increase validity by spending long periods in the field; and finally, the validity can be increased through “peer debriefing” with people that understand the issues but are not part of the research (Creswell 2003 , 196).
The first method, triangulation of data refers to using multiple methods. In this research for example both observation and interviews were used to assess whether the rules in groups had changed. When similar findings were obtained with both of these methods, this increases the validity of findings. However Denzin and Lincoln note that in many instances using multiple methods does not lead to similar findings. Hence, rather than defining triangulation as a method to increase validity, Denzin and Lincoln write that triangulation “is best understood as a strategy that adds rigor, breadth, complexity, richness and depth to any inquiry” (Denzin and Lincoln 2000, 5). When studying financial performance or whether members were late in their repayment, the records often provided a very different picture than what members had initially revealed in the interviews However, when respondents realised that the research team had also studied the group records, their responses often became more consistent with the picture that the records provided. However, as noted earlier, triangulation between the records and interviews in several groups was problematic because the records were of poor quality. The second approach to improving validity of taking the research findings back to the respondents was not directly used in this research, although our repeated interactions with respondents did mean that we were continually revisiting the earlier information collected and fitting it together with the latest developments. The third and fourth approaches of thick description method and prolonged field time were related in this research. By repeatedly visiting over a period of two years, the circumstances and dynamics of the groups to some extent unfolded. This fitted well with the fact that I was working part time and repeated visits are often meaningful to respondents. The passage of time can build trust, in particular when they become confident that there are no untoward consequences of being interviewed. “Thick description” is in part achieved in this study by providing significant extracts from the interviews in the empirical chapters that follow as well as in depth descriptions of the dynamics of some of the groups.
Regarding the last approach to enhancing validity of “peer debriefing”, after the field work I delivered a draft report of my field findings back to the funder, Financial Sector Deepening Trust
83 (FSD Kenya). FSD had not been directly involved in the research and several useful comments were received. However the implementer, CARE was also present and, as indicated above, they were rather critical and questioned the validity of my findings by claiming that I had not properly understood the methodology, which was not in fact the case. This incident is discussed further in chapter eight.
The longitudinal research design also played an important role in increasing the reliability and validity of the data collection and findings. In the periods between field visits I was able to process and analyse the data which allowed me go back to the field with new question and probe issues further, hence increasing robustness. In particular, it was this approach that enable the extremely important dynamics of numerous default and misappropriation cases to become more apparent during the third and final field visits.