Procedimiento didáctico: la lectura dramatizada

Three principles directed the selection and development of research methods for this thesis. First, following on from the reflections on positionality as well as reflections on the quality of the research process (for example, by seeing translators move into roles of co- facilitators), research methods with the committee members needed to revolve largely around participatory methods. Second, because the overall purpose of the research was to contribute to understanding wider dynamics around decentralisation policies as a dominant feature of today’s development discourse, the research methods also needed to maximise the generalisability of the findings (see: Babbie, 1998: 237). At first sight, there appears to be a tension between these two principles of participatory methods and generalisability of findings.

Generally, participatory methods are perceived to contribute to creating qualitative, case- specific, information. The information generated with participatory methods is primarily owned by the participants and used to mobilise for action that is pertinent and relevant for the participants who generated the information. ‘Generalisability’ does not appear in the list of the many advantages of participatory methods presented by its advocates (see, for example: Kumar, 2002: 27). One of the main reasons for this is probably that the primary beneficiaries of the generalisability of information are ‘outsiders’ (e.g. researchers, policy- makers, aid workers) and not the local group or community that generated – and therefore arguably ‘owns’ – the information. The local group would probably prefer to generate information that is actionable rather than generalisable. Approaching the issue from the other side, critics argue that “information acquired through participatory methods is … context-dependent and not suitable for making inferences [generalisable]” (Barahona and Levy, 2003: 32). However, the two need not necessarily be incompatible.

In 2000, Chambers participated in a “participatory poverty study” for the World Bank that used participatory research methods with “20,000 poor women and men from 23 countries”. An endeavour of which the authors noted: “Something like this had never been attempted before” (Narayan et al., 2000: 2, 3). Many of the methodological challenges in the research for this thesis – e.g. site selection, the pressure of time, establishing trust, and the problem of generalisation – already appeared as headings in the report on this World Bank research. Moreover, it acknowledges there are “continuing ethical challenges …

50 when the prime objective is not to directly empower and benefit the participants, but to help outsiders learn … and influence policy and practice” (2000: 17).33 Nonetheless, after studying a few more experiences, Chambers joins in the conclusions that “recent evidence shows that participatory methods can generate accurate quantitative data” and “participatory methods have been reliably used to investigate and quantify sensitive issues like violence and power relations which would have been difficult through questionnaires” (Mayoux and Chambers, 2005: 271, 279). Others similarly find that participatory methods can generate statistics useful for policy-making: “We claim that the statistics generated in these studies are at least as reliable as statistics obtained from traditional methods such as surveys” (Barahona and Levy, 2003: 2).

It seems that those who experimented with participatory methods to generate statistics for wider policy purposes – as this thesis also aspires – concur that two aspects are of critical operational importance. First, there is a need to employ “statistical principles” in the “selection of the sample of communities in which the research will be carried out” (Barahona and Levy, 2003). And because quality will be compromised “with small sample surveys claiming exaggerated levels of representativeness” (Hulme, 2000: 93), “purposive samples ... are essential for detailed coverage of different groups” (Mayoux and Chambers, 2005: 283). Second, the latter continue that a “degree of standardisation is required to enhance the validity of aggregation across different communities” (2005: 279) or, in the words of Barahona and Levy, “PRA [participatory] tools are adapted through a process of standardisation … taking into account the requirements of comparability of data between sites” (2003: 56). On top of this, careful recording of the large volumes of documentation is required. All these aspects have explicitly been taken into account in the design and large-scale application of Venn diagrams and pairwise ranking exercises in the group interviews for this thesis.34

The third principle that guided the selection for research methods was the principle of ‘triangulation’; using different research methods, locations, information sources and disciplines to analyse the topic of research (Chambers, 1996: 157; Kumar, 2002: 323). This

33

See comments in the preceding section Ethics. 34

Honesty demands that I record here that I designed my research with careful attention for these aspects, but rather on the basis of ‘instinct’ and ‘experience’, and not because I had studied these references when designing the methods in 2004.

51 variety in perspectives would assist in not just capturing most facets of the topic, but it would also allow contradictions to emerge ex post, or allow for deliberate cross-checking when emerging patterns are not quite clear. Moreover, by collecting data in different sources, triangulation also protects researchers from betting on just one horse. (In this field research, for example, the researcher had high expectations from secondary archival research by accessing the university libraries in Uganda and Tanzania, but in practice this proved to render meagre results.) Some analysts distinguish and recommend using various types of triangulation, almost presuming infinite resources, but in the research for this thesis triangulation was applied modestly and predominantly in what is called “methodological triangulation … using different methods in relation to the same object of study” (Mikkelsen, 2005: 96).35 Practically, the research designed or used five different research methods:

- Semi-structured group interviews - Closed individual questionnaires - Open individual interviews

- Primary and secondary archival research

- Reading and ‘re-reading’ primary and secondary sources

The design and actual use of each of these five research methods requires some further presentation, especially as the next two chapters present the findings and analyses that are the results of the application of these research methods.

Semi-structured group interviews

Most group interviews with school committees took place in a classroom or a lunchroom. In dispensaries, the group interview was often held in the nurse’s office. Each group interview opened, of course, with personal introductions and the explanation of the purpose, matters of consent, anonymity, confidentiality, and an explanation of the interview process. A group interview was scheduled to last about two hours, but late arrivals, tours around the school or dispensary, and tea breaks meant most encounters lasted close to three hours. The design for the group interviews was tested twice, once in

35

52 Uganda and once in Tanzania, with members of a school committee in which one of the researcher’s local contact persons was a member.

Table 1 details how the 64 group interviews were distributed over school committees and dispensary committees in Uganda and in Tanzania. Without wishing to open a discussion about sampling, sample size and reliability intervals, the table shows that – with exception of the largest district of Arua – about 4% to 19% of the committees in a district were included in the research sample. By most standards, this would suggest that a reasonable sample was taken and a reasonable degree of generalisability of the findings can be expected, especially as the interviews involved a large majority of the committees’ statutory members.

Table 1: Number of semi-structured group interviews

Committees in Uganda Committees in Tanzania In Schools In Clinics In Schools In Clinics Group Interviews (Total=64) 17 11 27 9 Participating members (Total=420) 127 60 180 53 Statutory Total of members a. 10 5 or 7 b. 9 to 11 c. 8 Average in sample Mode in sample 7.5 6 5.5 7 8.5 6 5.9 4, 5, 6, 7 Ratio of schools/clinics n Sample : n district (%) Moyo= 9:73 (12%) Arua= 4:416 (1%) Kaba.=4:112 (4%) Moyo=4:33 (12%) Arua=4:57 (7%) Kaba.=3:28 (11%) Hai=13:152 (9%) Same=14:176 (8%) Hai=5:27 (19%) Same=4:26 (15%) Notes:

a. Includes an ex-officio secretary (head teacher or nurse in-charge) who may or may not have formal

right to vote.

b. There can be five or seven members, depending on number of administrative units in the area covered

and on type of regulations used (i.e. the ‘1998 regulations’, or the ‘2003 regulations’).

c. The number of committee members ranges from nine to 11, depending on whether the committee

opted to have representatives of the foundation body (church) and a patron.

The semi-structured group interviews revolved largely around two participatory research methods: first the making of a Venn diagram (for technical details see: Kumar, 2002: 234; Pretty et al., 1995: 242) and subsequently a pairwise ranking exercise (technical details in: Kumar, 2002: 246; Pretty et al., 1995: 83). Each of these two methods was adapted – standardised – in form and use to facilitate systematic recording of the results. After the

53 group interviews with the 64 committees, the researcher categorised one-by-one each unique response in categories defined by the researcher, endeavouring “to exercise judgement in allocating participatory research findings to common categories” (Mayoux and Chambers, 2005: 290). For example, the unique descriptions of the contents of the relationship between committees and other actors in the Venn diagrams were categorised by the researcher as either an ‘act of upward accountability’, or an ‘act of downward accountability’. Similarly, the unique descriptions of successes and problems prioritised by the committees in the pairwise ranking were categorised as ‘operational or internal issues’, ‘communication with government’, or ‘communication with local and non-state actors’.36

The Venn diagram in the group interview served a dual research purpose; it provided not only insights into the character and density of the institutional and personal networks around the school and dispensaries committees (see Table 12, Table 19 and Table 20 in chapter 4), but also in the directionality and content of the acts of accountability by the committees (see Table 7 in chapter 4). In fact, as the findings in chapter 4 show, Venn diagrams were used effectively to – what Chambers called – ‘quantify sensitive issues like … power relations’.37 First, committee members listed “all persons, groups and organisations that have some importance for the committee”. Second, in open discussion, the names on the list were sized up as of ‘high, normal or minor’ importance and transferred on to – respectively – large, medium and small pre-cut round cards. Third, the committee members positioned these cards around a large round card that had ‘Our Committee’ printed on it. The cards could touch ‘Our Committee’ if there was a direct connection, or be positioned in a chain to others if contacts were indirect. In a fourth and final step, pre-cut in-going and out-going arrows were placed on top of the direct relations of the committee. Committee members then described the content of the relationship by writing on post-it notes what was received and sent to the direct contact and pasted those notes on top of the arrows. Picture 4 shows images of this four-step process.

36

‘Exercising the judgement’ in categorising and labelling a large number of unique units of data was probably one of the most complex, time-consuming, and sometimes agonising activities in this research. See, for example, the material in Anecdotal Evidence Boxes 7 and 8 at the end of chapter 4. In this light I found Mayoux and Chambers (2005) and Barahona and Levy (2003) may have under-played the complexity of this task of ex post categorisation (especially as volumes of material increases) and the significant risk of researcher bias.

37

Venn diagrams have also been used successfully with dispensary committees in Uganda by other researchers (e.g. Bjorkman and Svensson, 2007: 11).

54

Picture 4: A four-step Venn diagram of a management committee’s network

(clockwise, from left top)

1st_{: Listing all actors important for the committee. 2}nd_{: Sizing up each actor’s importance.}

4th_{: Describing the content of the relationships. 3}rd_{: Positioning actors around the ‘Our Committee’.}

Critical for the research was the careful recording of the complex and detailed Venn diagrams in such a way that an aggregated record could be made. A special ‘recording sheet’ was designed for that, noting the name, size and position of the contacts, as well as the contents and directionality of the content of the relationship.38 Even so, recording the full and final Venn diagram before the wind would blow it away – hence the little stones! – usually took about 20 minutes. (Picture 5 provides an impression of the complexity of the standardised final Venn diagram.) The patterns that emerged from the analysis of the aggregated findings could then be compared – triangulated – with findings from the individual questionnaires (e.g. regarding members’ personal networks), the regulatory requirements for committees’ accountability, as well as correspondence found in the primary archival research.

38

55

Picture 5: The standardised four layers of Venn diagram

(clockwise, from left top)

All actors positioned around the committee. Looking at what this actor sends to the committee.

This diagram is transcripted for later aggregation. Details are written on post-it notes.

The pairwise ranking exercise also served a dual purpose. In a first step the analyses of all pairwise ranking exercises provided insight into the character of the items that had been on the agenda of school and dispensary committees during the last year. In a second step, the pairwise ranking exercise revealed patterns in what had been committees’ biggest successes and biggest problems. (See Table 5 in chapter 4.) These findings could then be juxtaposed – triangulated – with the committees’ regulatory terms of reference and minutes of meetings, which were found via primary archival research. The pairwise ranking consisted of three steps. First, individual committee members brainstormed, noting down their views on successes and problems. Next, in a plenary discussion all cards were considered and a common list of a maximum of six successes and six problems was agreed. Third, the successes and problems were then prioritised in a pairwise ranking exercise. (See Picture 6 for an impression of the sequence.)

56

Picture 6: Pairwise ranking of successes and problems in three steps

(clockwise, from left top)

1st_{: Brainstorming on successes and problems. 2}nd_{: Making one common list of successes…}

3rd: Prioritising both lists via ‘pairwise ranking’. … and another common list of problems.

While the group interviews revolved largely around the Venn diagram and pairwise ranking, there was often also ample room to have introductory conversations with headteachers, nurses and chairpersons (most in English) and make detailed notes of comments about the discussions in the group (translated for the researcher if the co- facilitator was not required to facilitate) or written by individual committee members on cards. Obviously, the researcher’s notes are at risk of a number of contextual biases (e.g. language, spatial, hierarchy) and, of course, ‘researcher’s interest bias’.39 Nonetheless, with this caveat these notes are used in chapter 4 to highlight or underscore trends.

39

My ‘notes from group interviews’ would probably constitute evidence in a risk analysis of ‘selective quoting’ as a researcher takes notes “to play the ventriloquist” with participants’ comments (Swadener et al., 2000: 3).

57

Closed individual questionnaires

A second research method in the field was a classic closed individual questionnaire, investigating individual committee members’ profiles and opinions. At the end of group interviews, each committee member was asked to fill in a questionnaire with 11 closed questions – simply requiring the ticking of a preset box – and two brief open questions (see Appendix 1 for a model of the questionnaire).40 The questionnaire was tested – and subsequently slightly adjusted – in Uganda in the same trial as the testing of the group interview. In Tanzania a Kiswahili translation of the questionnaire was used. In general, committee members took about 10-15 minutes to fill in the questionnaire and none of the committee members reported problems in filling in the questionnaire – aside from the occasional “I forgot my reading glasses”. The data emerging from these individual questionnaires proved to be a surprisingly rich source of information – particularly with regard to members’ personal networks – and a useful backdrop to triangulate information from other sources. Table 2 details the general profile of the 420 respondents.

Table 2: Number of closed individual questionnaires

Ex officio members Local Members Total Uganda School Committee 28 99 127 Dispensary Committee 16 44 60 Sub-total 44 143 187 Tanzania School Committee 68 a. _{112 180} Dispensary Committee 10 43 53 Sub-total 78 155 233 Totals 122 298 b. ₄₂₀ Notes:

a. This relatively high number of ex officio members is due to the fact that Tanzania’s school committees

also have two members elected from among the school’s teachers. For the purposes of this research

these two members are counted as ex officio members because they are not representatives of the

local community.

b. Local members and ex officio members filled in the same questionnaire. However, upon collecting the

filled in questionnaires, the researcher separated the questionnaires by the two groups. Most of the analyses in chapter 4 on the field research are based on these 298 committee members because they represent – through election or selection processes – the local community.

40

The findings from one of the questions in the questionnaire (i.e. question number 13 on the impact of participating in local management committees on people’s trust and understanding of different tiers of civil servants) are not presented in this thesis but are being processed for a topic paper.

58

Open individual interviews

In contrast to the ‘standardised’ participatory methods in the semi-structured group interviews and the closed questionnaire, the third research method used in the field research consisted of a number of “flexible, iterative, and continuous” individual interviews and conversations (Babbie, 1998: 290). In every district, the researcher was introduced – or came across – individuals who appeared interested in the research or were likely to have interesting experiences with local management committees or decentralisation. Taking some ethical safeguards into account (see the earlier section on Ethics) the research often asked these persons for an open individual interview over lunch or dinner in the weekend. With hindsight, these persons have been categorised into four groups: district officials, local politicians, opinion leaders or aid workers, as in Table 3.

In these 28 open individual interviews, the researcher usually broached two issues. Most interviews started seeking clarification on factual matters – for example details of an electoral process, or getting dates and numbers right – but once these ‘how, who, when, where’ questions were addressed, many of these interviews moved into an open conversation about the ‘why’ of matters. In chapter 4, many of the remarks from these individual interviews are presented as illustrations or explanations of trends or patterns. While these conversations cannot be summarised in a few phrases, they certainly assisted the researcher in developing a better understanding of local actors’ perception of decentralisation as a political and historical process.41

Table 3: Number of open individual interviews District officials a. (education or health sector) Politicians a. (district or sub- district councils) ‘Opinion leaders’ a. (church, business, or ‘elders’) Aid workers a. (expatriate and local people) Total Uganda 13 3 4 6 26 Tanzania 7 2 4 4 17 Total 20 5 8 10 43 (28) b. 41

Again, as indicated in footnote 39, I acknowledge that the selection of persons for these individual interviews carried the risk of ‘researcher bias’. Moreover – and more importantly – as these individual interviews had much more the character of a private conversation, a gender bias was inevitably introduced.