In addition to this protocol (and the completed review that followed discussed in Chapter 4), I found 18 published reviews. Some of these were conducted systematically and others less formally. As they were not contributing original data they were not included in my own review (conducted in collaboration with my supervisors and described in Chapter 4), but were nevertheless useful sources of information for the thesis. None were sufficiently recent or relevant to my work to make our review unnecessary. Some included a range of process interventions that targeted patients as well as providers, and in many of them computerised reminders were just one
intervention among several included in the review. The following paragraphs describe some key insights arising from these papers.
Balas et al (1996) (28) reviewed 98 randomised controlled trials of clinical information systems. This was a comprehensive review, but only 64% of the interventions targeted the health care provider and this review is no longer very recent. Provider reminders were generally found to make a significant difference to process outcomes.
Balas et al (2004) (29) described forty studies of computerised knowledge management interventions to support diabetes care. These included eight studies of the effects on guideline compliance of computerised prompting, reporting significantly improved compliance in six of them.
Bennett, Glasziou and Sim (30) reviewed articles specifically related to medication management, concluding that computerised reminders and feedback were generally valuable in this situation.
Berlin, Sorani and Sim (31) used a previously developed taxonomy of computer-based clinical decision support systems (CDSSs) to describe the current literature. Seventy-four CDSSs were reported in fifty-eight studies, and two distinct subsets were identified: those aimed at patients (via mail or telephone) and online systems directed at physicians in inpatient contexts. These studies were generally not relevant to my current work, but an important conclusion was derived: that CDSSs are heterogenous and dependent on the clinical or workflow setting, limiting their generalisability.
Garg et al (32) reviewed 100 controlled trials of CDSSs both to investigate their effectiveness and to identify features predicting success. They found that the quality of the trials improved over time, and that improvements in practitioner performance were more evident than patient outcomes. Out of 21 trials of reminder systems, 16 produced positive benefits in terms of performance. Automatic prompts were generally more effective than those requiring the user to activate them.
Hasman, Safran and Takeda (33) concluded that reminder systems linked to physician order entry systems were generally beneficial but their use for diagnostic support was more limited.
Kawamoto et al (34) studied 70 articles describing CDSSs and undertook regression analyses to determine the influence of up to fifteen characteristics of the intervention predictive of success in terms of improved clinical practice. Four features produced independent predictors. These were:
Automatic provision of decision support as part of clinical workflow
Provisions of recommendations rather than just assessments
Provision of decision support at the time and location of decision making
Computer based decision support
Thirty out of 32 papers that included all four features significantly improved clinical practice. This suggests the need to embed such interventions into the working environment at the point of care.
A review by Kupets and Covens from 1966 to 2000 (35) identified papers related specifically to improving breast and cervical cancer screening using a variety of techniques. They identified three categories of intervention: physician based, physician/patient based, and patient based. The physician based strategies such as manual and computer generated reminders proved the most effective at improving screening rates. They described the concept of a ‘Number Needed to Intervene’ (NNI), and estimated that in the case of reminder notices 3 physicians need to be exposed to the intervention for one of them to order a screening test. This number was lower (i.e. more effective) than for other types of intervention.
McPhee and Detmer (36) also reviewed approaches to the problem of cancer screening using office based interventions. This review was published in 1993 and so is
my own review described below, the authors drew a distinction not only between physician and patient directed interventions (and both), but also between ‘in-reach’ and ‘out-reach’ activities. In-reach approaches include the consultation based reminders that are of particular relevance to the e-Nudge trial, although other examples included practice based audit which was excluded from our own review. The conclusion of this review was generally positive regarding the effectiveness of office based interventions for cancer prevention.
Mitchell and Sullivan (37) considered more generally the impact of computers in primary care consultations. They identified ‘a descriptive feast but an evaluative famine,’ highlighting the relative lack of high quality, controlled trials of computerised interventions, in contrast to the volume of papers describing interventions, their development, use and acceptability. Out of 89 papers included, 61 reported the effect of computers on practitioner performance, 17 used patient outcomes, and 20 were qualitative studies of practitioner and patient attitudes. This review identified negative aspects related to process measures (including lengthening consultations) but not to patient outcomes. The phenomenon through which effectiveness may fall after withdrawal of the intervention was also identified.
Montgomery and Fahey’s review (38) included 7 randomised controlled trials investigating the use of computers specifically in the area of hypertension management. These studies included 11962 patients and were combined using a narrative rather than meta-analytical approach due to heterogeneity of patient populations, interventions and outcomes, although their methodological quality was similar. A beneficial effect was seen on processes of care such as follow up, but once again the effects on patient outcomes (such as control of blood pressure) were less conclusive.
A meta-analytical approach was, however possible in a review of general practitioner based reminders to support cervical cancer screening reported by Pirkis, Jolley and Dunt (39). Ten studies were identified and a positive effect demonstrated on
a woman’s chance of having a Pap smear if the GP had been reminded. A strong recommendation over the use of such reminders was made.
Shea, DuMouchel and Bahamonde (40) conducted a meta-analysis of 16 randomised controlled trials reporting the impact of computerised reminders in six areas of preventive care (vaccinations, breast cancer screening, colorectal cancer screening, cardiovascular risk reduction, cervical cancer screening, and ‘other preventive care.’) The first four of these areas were associated with benefits of the reminders but not the final two. Ten out of the sixteen interventions evaluated were directed at physicians, the remainder at patients or family. Cardiovascular preventive activities included measurement of blood pressure; follow up of hypertension; cholesterol screening; and dietary assessment and counselling. The overall odds ratio (ratio of odds of completing the target behaviour in intervention and comparator arms) was 1.77 [95% CI 1.38-2.27], and for the cardiovascular risk reduction subgroup 2.01 [95% CI 1.55-2.61].
Shiffman, Liaw, Brandt and Corb (41) reviewed studies of computer based interventions including clinical guideline implementation systems and their impact on clinician behaviour and patient outcomes. Quantitative meta-analysis was impossible due to study heterogeneity. A narrative synthesis concluded that better control of confounding factors would be needed to derive firm conclusions over the effectiveness of such systems at influencing clinician behaviour Seventeen out of twenty systems described used paper based reminders, albeit computer generated. The authors remarked that ‘the paperless office remains a vision of the future.’
Shojania 2006 (42) (note different from Shojania 2009 discussed in Chapter 4) considered only interventions related to diabetes care and using glycosylated haemoglobin level as the outcome, but included any type of quality improvement strategy. Studies using before/after designs were included as well as randomised and quasi-randomised trials. Out of eleven strategies, team changes and case management
finding of more clearly positive outcomes in the smaller studies) was an issue, and the authors also commented on the difficulties in classifying the complex interventions involved in quality improvement when assessing effectiveness.
Tu and Davis (43) reviewed the evidence for educational interventions in the management of hypertension. Reminder systems were only one of a number of interventions that were generally not relevant to my research, including academic detailing, but were apparently the most promising in terms of changing clinician behaviour. However, once again it was the processes of care (such as follow up) rather than clinical outcomes (such as blood pressure levels) that benefited.
van der Sijs et al (44) identified 17 papers describing trials of drug safety alert systems used during computerised order entry. This review was concerned largely with the reasons why physicians over-ride such alerts (in 49%-96% of cases) rather than their effectiveness. Problems include low specificity or sensitivity, unclear information content, and incorrect handling of the alerts. This review is important because it emphasises the need to embed a new intervention such as an alert system in the workflow context if it is to be useful rather than disruptive.
Finally, Dexheimer et al (45) updated a previous review by Balas et al (2000) (46) of both paper-based and computerised prompts related to preventive measures. Reporting nine years later than Shiffman et al (discussed above), they also found a preponderance of paper based rather than fully computerised systems, the latter involved in just 8 out the total of 61 studies. They found an increase in preventive care measures of between 12% and 14% averaged over all studies. Cardiac care and smoking reminders were the most effective.