M.TERESA DE lESF

Marisa and I sat at the back of the board room in China, considering the expression on the face of the lead surveyor. She told me the survey were were undertaking wasn’t ‘as clear as our usual surveys’. Curious about how this compared with her perception of ‘usual’, I asked ‘How so?’ ‘I think they went for survey too soon’, she told me. Tapping the sheaf of documents on her lap, she reminded me: ‘They only changed the SOP in February. With their volume, only 10 per year, you don’t get the experience, you don’t learn how to do a good review’. Learning to conduct a ‘good review’ came with practice, which in turn came with a committee receiving protocols for review.

The committee we were surveying was one of several in a hospital which was the site of limited research, and as such did not see many protocols. If the committee had waited a little, she

thought, it might have better ‘evidence’ of its capabilities in the form of meeting minutes, archived trials and ongoing files.

If finding means of creating objectivity was the focus of my attention above, here the creation of evidence comes to the fore. The work of Shapin and Schaffer (1985) and Ezrahi (1990) amongst others have shown us a great deal of shared techniques of authority between science and politics, the act of witnessing evidence being central to both. As we saw above, it is in the Lead Surveyor’s presentation that a Survey must provide evidence for its recommendations. It is as important for there to be evidence of good practice as it is for there to be evidence of error, to support

‘recommendations,’ to explain why. During the introductory remarks for a survey in the Philippines, the lead surveyor remarked that the most important thing was to ‘find the evidence’:

if we make comments, we need to have some evidence. Otherwise, we prefer not to say it.[...] Everything you find, you write down as evidence. Use your camera to document.

It is better to let them see the evidence, otherwise they don’t believe it. In the final report, we have to mention everything with the evidence. [We]] cannot make a judgement without evidence.

The day would run on in this way. Photographs were taken of the offending documents - missing signatures on informed consent forms, missing dates on countersignatures. Trainees made diagrams of the committee offices, and leant over the shoulders of secretaries to scrutinize their databases. In the closing meeting, ‘evidence’ would be gathered together from the teams: ‘What protocol number? What page?’ Evidence was collected with the final powerpoint in mind,¹⁶ details and references hoarded for each end of day summary meeting, and the ultimate feedback to the committee under scrutiny. Evidence did not simply serve the Surveyor’s “safety”, however.

Take this Surveyor’s description of the Survey’s approach: ‘The approach is going to be evidence based. We survey the EC then we evaluate the ethical practices based on its adherence to and compliance with national and international guidelines.’ Borrowing from the corpus of ethical principles that the organisation teaches, the organisation is applying to itself and its Survey the same standards for its international operations as it expects for its ethics committees.

Recognition is based on five standards (Figure 13), and the assessment is structured around an evaluation of the quality attained by the committee in relation to each of them. Not only would the Survey be ‘evidence based’ but its methodology was described as ‘just like research

16 As Miller remarks (2003:61) for the Best Value quality control final meetings he observed, the use of PowerPoint to make final presentations is one of a number of factors that mediate the construction and aesthetics of the final document (Harper 1998; Riles 2001:114-42, see also Tufte 2003).

methodology.’ At these events, as much as there was evidence based ethics, there was also an ethic of evidence which ran through the survey, and was inculcated into the teams who were auditing.

Evidence was also oriented towards the making of an envisaged committee — positive feedback meant one resembled it, corrections meant deviation. During a break, my co-trainee and I were discussing how much there was to keep in one’s mind, even within our own designated area. He pointed at the lead Surveyor. ‘She carries it in her head,’ he said, meaning (I suggest), not only the knowledge or criteria of the Committee, but an image. What is desired is already known. This is

‘not open ended journeying but anticipated destinations’ (Strathern 2004: 80). It is ‘just like research,’ but unlike the testing of a hypothesis, the collection of evidence is oriented towards a model already known. Cristina is attentive to what makes a good Surveyor:

They need to be observant enough to capture what’s important, they should have that sense of what’s important. Some people can see very small details but not capture the big picture.

[...] Sometimes they are lost in the small details, and don’t see what’s wrong in the system.

Figure 13: SIDCER Recognition Standards

A social science background helps, seeing the macro-micro relationship, having a systems orientation. If you don’t have that background, that perspective, it takes a while.

The ability that Cristina calls a ‘macro-micro’ view was of most importance in synthesizing findings. End of day meetings brought the teams together, building gradually — in FERCAP’s language — towards a full picture of the committee. Late in the afternoon of the first day of the Survey I attended in Beijing, Heijan, an experienced Chinese Surveyor who had served as FERCAP’s Research Fellow in Bangkok, described the process of the survey to the trainees. She explained that the team would split, the groups would be given different areas of responsibility to examine in greater depth, and they would report back at the end of the day. One team would look at the protocols, for instance, another at the membership records. The chapters of the committee’s Standard Operating Procedures would be divided between the groups for assessment.

At the end of each day, the groups would come together and combine their findings in order to make recommendations. Perhaps because the program was newer in China, or because the language barrier was ‘thicker’, trainers employed two inventive metaphors for this work of synthesis.

The first depicted how the survey would reach its conclusions (Figure 14). Heijan projected a slide with three Chinese characters, ^木 [mù], in the smallest font, was at the back. ^林 [lín] was printed medium size in the middle, and 森 [sēn], the largest, was positioned at the front. The first, meaning leaf, the second, trees, the third wood, she explained that ‘The leaves are the detecting method, the picture of the forest is the analysis - how we draw conclusions and necessary system thinking’.

Figures 14 and 15: Heijan’s Leaf and Elephant slides

Her second slide (Figure 15) was an image of a surprised looking cartoon elephant, surrounded by men in suits, all wearing dark glasses. One held a cane over his forearm, and had his arms outstretched over the elephant’s rump. Another sat on the elephant’s leg, holding the elephants tale with a confused expression. Another gripped its trunk, eyebrows expressing bafflement over his shaded specs. Around the image, eight large yellow question marks.

Heijan warned us that, separated into our teams, none of us could:

expect to know the whole elephant. Each group is only touching only one part at the end of the day - you [in touching the part you do] don’t touch the tail or the head. Try to reconstruct form your observations and reports, what does the EC look like, what are its strengths and weaknesses? How does it do its work?

The image, referenced by Cook, Laidlaw and Mair in the Introduction, is usually used within Buddhist texts and teaching to demonstrate the futility of claims to omniscience. In its use here, blind men we find reinforced the idea of partial perspective. As surveyors, claimed Heijan:

You will say the elephant looks like this, but you only have a part. But together, you’ll be able to describe what the elephant looks like. Different people see different things, they see from different parts. We collect every one’s opinion and truth, we put it together, and may get truth of whole thing.

During the coffee break, I took the handout of the slides Heijan, and asked her to tell me more about the stories these slides referenced. The elephant story was about the move ‘from seeing all sides to arriving at a balance,’ she told me. She framed her explanation by pointing to a clipart image of scales of justice, which adorn many of the circulating powerpoint slides emanating from American literature on risks and benefits. ‘You must see risk as well as benefit,’ she told me, ‘you can’t just see risk or will say its all bad, can’t just see benefit, and forget the risks.’ As I was struggling to process how we had moved from the committee’s strengths and weaknesses to risk benefit analysis, Heijan changed tack:

Teamwork, it’s about teamwork for each may not see the whole, so not everyone can be completely right. The teams, bring finally together each part, and you finally get the whole picture. Everyone may not be completely right, but they may not be completely wrong.

Based on teamwork, we can get to the truth.

And the leaves? I asked:

The survey is so detailed! We look for missing signatures, dates, we look for checked boxes.

How are we going to gather together everything everyone has learned? We need to make evidence, for system thinking, we won’t say, ‘The way you review this protocol is not good,’

we look at the process, on which procedure you need to improve. It’s Quality Assurance and Quality Control. The Survey is Quality Assurance, evidence, to give the system for Quality

Control. The leaves on the tree are the detecting method of the survey, where we get evidence. But the forest is our analysis — how we draw conclusions and use systems thinking.

The story fits with the Lead Surveyor’s comment ‘you see what you see, I see what I see.’ There are two points to notice however, in how Heijan has introduced difference into the combination of perspectives. First, through the concept of balance, she has moved from the survey’s practices of overview and assessment to those of the committee: a risk benefit assessment. The image of perspectives not only comprising the object under study but also holding both it and themselves in balance is one that jumps between the committee’s assessment and the assessment of the committee’s assessment. Second, she has situated in the person their particular ‘difference,’ the perspective they hold from their life experience or identity. It is this shift, (or mapping) of the effect of comprehension from a physical to a conceptual difference that I explore in greater depth in Chapter 5. For now, I move to one of the areas the teams are set to look at in detail: Standard Operating Procedures.