Tratamientos y ensayos clínicos

Five studies have evaluated the implementation of CCDSSs in ED with an ITS design (Day et al., 1995; Schriger et al., 1997; Schriger et al., 2000; Buising et al., 2008; Gibbs et al., 2012). None of them evaluated a CCDSS for use at Triage. ITS studies collect data a multiple time points before and after the implementation of the intervention (Cochrane, n.d.). Collection of data at multiple points before the intervention reveals the underlying trend, which will have a naturally occurring variation. The analysis, which compares the data points after the intervention can take account of this underlying trend to enable the effect of the CCDSS to be shown (EPOC, 2013b). ITS studies, if conducted appropriately are deemed of sufficient quality to be included in EPOC reviews (EPOC, 2013b). EPOC (EPOC, 2013a, 2013b) stipulate that ITS studies must meet the following criteria to be included in their reviews:

 Secular trends must be analysed (a simple t-test pre and post intervention is insufficient)

 There must be a clearly defined point in time when the intervention was introduced

 There must be a least three time points where data is collected before and after the intervention

These elements are seen as being fundamental to reducing the risk of bias. In EPOC systematic reviews studies that meet these inclusion criteria are then

subject to critical appraisal using the following seven criteria to judge their risk of bias (EPOC, 2013a).

1. Was the intervention independent of other changes? 2. Was the shape of the intervention effect pre-specified? 3. Was the intervention unlikely to affect data collection?

4. Was knowledge of the allocated intervention adequately prevented during the study?

5. Were incomplete outcome data adequately addressed? 6. Was the study free from selective outcome reporting? 7. Was the study free from other risks of bias?

3.9.1 Critical appraisal of the ITS studies

The study by Gibbs et al (2012) evaluated a CCDSS to improve the care of patients presenting with CAP It used a ITS design over a 33 month period analysing the care of 1185 patients in total. Of the four key outcome measures there were statistically significant changes in three: blood cultures prior to antibiotics, antibiotics within six hours of arrival and appropriate antibiotic selection. The fourth measure, mean time to antibiotics decreased by one minute but this change was not significant. A brief conference abstract of the study has been published but no methodological information is available despite email contact with the authors. Therefore any rigorous critical appraisal of the method has not been possible.

Three of the ITS studies were generated from the same academic ED (Day et al., 1995; Schriger et al., 1997; Schriger et al., 2000). The UCLA ED developed an Emergency Department Expert Charting System (EDECS) in the 1980s. EDECS provides a complete EPR including investigations, treatment orders, prescriptions and discharge instructions for patients. Clinical guidelines function in the background and where relevant investigations or treatments are either: strongly recommended, optional or discouraged. Deviation from the suggested actions were always permitted. Clinical guidelines for acute low back pain (Day et al., 1995), health care worker exposure to body fluids (Schriger et al., 1997) and fever

in the under three year old (Schriger et al., 2000) have all been embedded into the system over a five year period and evaluated using ITS. All three studies state they have used an ITS method. The study of patients with acute low back pain was the first to evaluate EDECS (Day et al., 1995). It describes the design as a “prospective,   time   series   comparison   of   control   and   test   periods”  (Day et al., 1995). However in terms of an ITS study its design is methodologically weak. There is no discussion of the time points over which the data was collected. In the “before”  period a random sample of 103 patients were analysed (from a total of 206).   In   the   “after”   period   259   patients   met   the   inclusion   criteria   and   were   included in the analysis. From these details it appears that the method is more consistent with an uncontrolled B&A study.   In   the   “before”   period   data   was   manually abstracted from the hand written charts. There is no mention of a standardised abstraction sheet, training of the abstractors or any inter-rater reliability testing. There was no comparison of the before and after groups to assess if there were any fundamental differences that may account for any change in physician behaviour. No limitations of the study were discussed despite some fundamental risks of bias, namely: selection, performance and detection biases. Together with the issues of regression to the mean and the failure to consider any confounding variables the results of this study do not permit any firm conclusions to be drawn about the impact of EDECS on patient care or physician behaviour.

The remaining two ITS studies from UCLA ED analysed the effect of clinical guidance embedded into EDECS for health care workers exposed to body fluids (Schriger et al., 1997) and febrile children under 3yrs of age (Schriger et al., 2000). Both studies used the same method: an off, on, off design with intention to treat analysis (and with a second off phase). Outcomes measures were similar: quality of clinical documentation, provision of aftercare instructions, compliance with testing and treatment decisions and cost of care. Both studies showed significant increases in quality of documentation and issuing of aftercare instructions. Compliance with testing and treatment decisions only improved for patients with exposure to body fluids (Schriger et al., 1997). This off, on, off design is not a true

ITS study.   The   second   “off’   phase   was   when   the   intervention,   the   embedded clinical guideline, was removed. The aim was to assess if the outcomes measures returned to baseline as this final phase was regarded as a second control group. This method was slightly different to the first EDECS study by Day et al (1995) as it has   a   second   “off”   phase.   Additionally   there   were   attempts   to   manage   some   causes of bias.

In both these studies in the  two  “off”  phases  data  was  collected  from  the  clinical   record by trained abstractors. They were tested to ensure the error rate was <2% and  there  were  periodic  quality  checks.  No  abstraction  was  needed  for  the  “on”   phases as this could be directly exported from EDECS. Both studies compared the characteristics of the three study groups (off, on, off) and the characteristics of the treating physicians. Regression to the mean whilst not mentioned may be less of an issue as the outcomes in the final phase of both studies returned to baseline. There was no consideration regarding confounders (experience of the physician), selection bias (similarity of the groups) and detection bias (training and assessment of abstractors). Data was not collected at regular intervals before and after the intervention and therefore there was no analysis of underlying secular trends. These are fundamental elements of ITS design as stipulated by EPOC (2013b, 21013c). It is difficult to draw any definite conclusions about the true effects of either guideline embedded into the EDECS. Interestingly the study of fever management did not show any changes in test ordering or appropriateness of care (Schriger et al., 2000). There are fundamental issues with the development of the fever guideline and lack of consensus regarding its appropriateness amongst the physicians using it. Undoubtedly this will have had an effect on compliance.

The final ITS study included in the literature review included a time series analysis as part of a B&A cohort study (Buising et al., 2008). This study demonstrated the most rigorous ITS method when investigating the impact of a CCDSS on concordant antibiotic prescribing for CAP. The study took place over a 41-month period  during  which  there  were  3  distinct  phases.  Phase  1  was  the  baseline  “pre- intervention”   period,   which   lasted   12   months.   After   an   11   month   gap   phase   2  

began, lasting 8 months and saw the introduction of an intervention to increase antibiotic concordant prescribing - academic detailing. Academic detailing is a process for face-to- face education aimed at improving prescribing (National Resource Centre for Academic Detailing, 2014). Finally after a gap of 6 months phase 3 began with the introduction of the CCDSS; lasting 5 months. A single trained research nurse collected data on every eligible patient during this time; an infectious disease physician checked five percent of these judgements.

Multivariate logistic regression was used to compare concordant prescribing during the three phases whilst adjusting for disease severity and the age of the patients. However there was no consideration given to the experience of the prescribing clinician as a possible confounder. Clinician exposure to the CCDSS was associated with higher odds of concordant prescribing (OR 2.03 [1.13-3.66]) after adjustment for patient age and disease severity. In general terms prescribing patterns improved over time, as one would expect. However the time series regression analysis in this study revealed that during the CCDSS phase the degree of concordant prescribing was higher than the expected underlying trend. With regard to the EPOC (2013b, 2013c) criteria for ITS studies this research addresses many of the criteria associated with reducing the risk of bias.

ITS studies, if properly designed appear to be an appropriate method for assessing the impact of a CCDSS intervention. However only one study using an ITS method for evaluating CCDSS in EDs considered the underlying secular trend (Buising et al., 2008). It is therefore difficult to draw any overall conclusions about the impact of CCDSS in ED and further studies using well designed ITS studies are required.