Resultado y análisis de la encuesta

The research demonstrates that evidence in management is not a stand- alone entity, with fixed, stable and transcendental qualities. Rather, we have shown that recognising and using something as evidence is an

irreducibly context-dependent phenomenon. This means that what evidence

is depends on how it becomes part of an activity and on the ways it is linked to practical accomplishments. Even so-called universal or ‘context-free’ sources of information - such as systematic reviews, scientific reports, and NICE guidelines - were probed and reinterpreted in the context of a specific activity. For example, NICE guidelines were used in one context as a

political weapon (when the PCT W commissioner used them to force clinicians to accept that improvements in TIA services were needed). In another context (the diabetes project), guidelines were used as a technical instrument, enabling project members to identify the most effective and safest use of drugs throughout the pathway normally taken by diabetic patients. In yet another context, NICE guidelines and systematic reviews were used as moral objects, enabling IFR panel members to determine whether it was fair for an IFR request to be approved or declined. These different uses of knowledge products echo some of the differences in

evidence utilization highlighted by Weiss, as for example, between problem- solving and political uses (58). From our study, however, we are able to relate these differences not to discrete models of utilization, but to the particular, multi-faceted and dynamic circumstances in which evidence manifests itself.

One conclusion, then, is that any knowledge product (e.g. a Public Health report, a NICE guideline, NHS Evidence etc.) acquires emergent properties. These properties are nested in a wider field of practices comprising

particular socially recognisable sets of activities – for example, persuading rival clinicians to accept the need for a change in the current patient pathway, creating a clinically effective and safe diabetic care pathway, justifying the fairness of an individual decision. This means that in order to understand why evidence is used (or not) we need to consider its properties and status in relation to the task at hand and the stakeholders performing those tasks (118).

To be clear, we do not suggest that knowledge products are not useful. The commissioners and other experts we studied did appreciate and search for ‘good evidence’ because it enabled them to deepen their understanding and to evaluate their problems in more sophisticated ways. For example, the comprehensive health improvement initiative (CHII) was based on robust

and insightful evidence about residents’ lifestyles and attitudes to eating, smoking and drinking. This sort of intelligence was celebrated by both Public Health experts and commissioners in the context of their project. Our

survey findings also suggest that Public Health intelligence was one of the most sought after forms of evidence.

As we showed through our IFR findings, making judgements is also a social activity of proclaiming to be rational. Thus, even at moments of rational choice (‘do we or don’t we agree to this request’), ‘evidence’ was

interpreted and became enmeshed in an activity (e.g. investigating the major causes of health inequalities) and in the pursuit of a practical end

(e.g. commissioning a CHII that would reduce health inequalities). In light of these findings, it is imperative to revisit assumptions, and key questions, about knowledge/evidence utilization in healthcare. In particular, such questions as ‘how can we improve the uptake of evidence’ or ‘how can we put evidence into practice’ seem to be the wrong place to start from. This is because these questions presuppose the existence of evidence prior

to its utilization. They also assume that the role of managers is a largely technocratic one; i.e. to decide on the most efficient technology or process. We have shown, in contrast, that, especially in the context of

commissioning decisions, evidence is not ‘taken up’ in practice. Rather, it is co-produced (or mutually constituted, in Orlikowski’s terms) and becomes a prosthetic device that equips managers, and other actors, in their pursuit of decisions and practical ends(119). In this sense, evidence becomes meaningful as a tool for knowing in practice (79). Managers are artful, moreover, in how they deploy evidence for specific (and sometimes

contradictory) purposes (e.g. political, technical/instrumental, and moral – see also (74).

Our findings thus extend the emerging critique of Evidence-Based Medicine (28) to the domain of Evidence-Based Management. They argue, strongly, for a re-phrasing of fundamental questions on evidence utilization. It may be more fruitful to ask:How are particular knowledge products drawn upon and recast in the context of a management practice? How do specific

objects acquire ‘evidential’ properties when becoming part of a social activity? How do we construct and design sources of information and knowledge products such that they can act as evidence in particular contexts?

These may seem rather abstract questions but they have very practical

consequences. Taking the last, for example, this might mean that, instead of focussing so heavily on ‘universal’ or ‘best practice’ information (and

to take in order to become useful and persuasive in particular contexts. To do this, we need to attend to the way information can support

commissioning groups’ knowing in practice as a social accomplishment (120). Robust case histories (as used in the practice of law) that combine scientific (forensic) results with cost projections, Public Health information and local population intelligence, might be more helpful for redesigning a service, for example, than guidelines focussed on clinical efficacy alone. Whilst scientific reports/information may be helpful for instrumental

problem solving and technical understanding, such case histories might be more relevant, and therefore more readily usable, for managers seeking legitimacy and conceptual understanding, for example. Case histories could be practically deployed as past precedents, which systematise previous experience, relate this to local context, and anchor it in less anecdotal forms.

Our results suggest, then, a shift in emphasis in the design of knowledge products , with a focus, not just on ‘is there enough evidence’, but ’is ‘evidence’ fit for purpose’, taking due account of the different purposes to which it might be applied. Our findings also stress the need to recognise that it is the relationships among different kinds of activities, which interact on an ongoing basis, that provide the contours for evidence utilization (20). Comparing across cases allowed us to identify (at least) three distinct

practices brought to bear in commissioning management that interacted on a recurrent basis. These practices comprise, in Feldman and Orlikowski’s terms, the ‘repertoire’ of commissioning management practice (121).

A) National policy practices; main activities here included national guidance development, setting national priorities, and standards, managing performance of regional and local

commissioners, and demonstrating public concern for maintaining consistent standards across the country.

B) Commissioning organisation practices; main activities here included monitoring contracts and budget over/under spends, implementing national guidance, commissioning policy

development, managing cross-departmental relationships (e.g. between commissioners, finance, Public Health, etc.), ratifying decisions made by individual projects, setting locally and nationally acceptable strategic priorities, reporting to the board,

demonstrating quality assurance, monitoring target achievement, managing relationships with multiple healthcare providers and local authorities, cajoling GPs and other clinicians, doing returns,

attending to national policy makers’ statements, and understanding the extremely complex national payment infrastructure.

C) Project-specific decision making practices; main activities here included organising meetings, managing project timelines and

budget, ensuring effective teamwork, assembling the required expertise, going through a decision making routine, requesting ratification of decisions by organisational boards, tweaking national policies to fit local circumstances, writing and agreeing service specifications, and making individual funding decisions..

In comparing case studies, we found recurrent interactions across these distinct practices that were consequential for the ways evidence was mobilised. Such practices then, were seen to be interdependent –

overlapping and intersecting through the specific activities engaged in by individuals’. For instance, the Diabetes project started as a project for shifting diabetic services from acute to a community-based setting. That was a national priority - i.e. a product of national policy practices - which was then cascaded to the PCT’s CEO for implementation. Thus it became part of organisational practices. The CEO then instructed the Diabetes project lead to ‘move people out of the hospital’ – in this way the national policy practices reached project level practices. The project lead, in turn, communicated to all parties involved in the Diabetes project that this objective was a ‘must-do’. These interactions across practices at different levels (national policy, organisational and project) contributed to the

commissioning management decision, whereby the imperative to develop a community-based setting was taken for granted. The feasibility of this objective was never questioned, nor was the evidence behind it, nor were new forms of evidence proactively sought.

Our findings highlight, then, that practices interact in very complex ways and knowledge utilization in healthcare commissioning is nested within the nexus of practices at different levels - national policy making,

commissioning organisation and project decision making. Figure 22

summarizes this. Being attuned to this nexus of practices (how practices at different levels interact) also helps us to explain why knowledge and

Figure 22. Knowledge Utilization nested within practices