162 LOS ESCITAS IMAGINARIOS.

We designed a data extraction sheet to collect data on the studies and the interventions included within them. We were unable to seek additional data from authors in the time frame of the review.

We extracted data on study quality. We chose a dichotomous measure based on allocation concealment, as this is the aspect of trial quality most consistently associated with treatment effect,82,83_{and is particularly}

relevant when outcomes are subjective, such as QoL.84_{Other measures of trial quality in the risk of bias}

tool, such as blinding, are generally less useful in trials of self-management interventions because it is difficult to meet the conditions required for effective blinding. Allocation concealment was judged as adequate or inadequate according to the relevant section from the Cochrane risk of bias tool. We analysed intervention effects on all outcomes (QoL, hospitalisation and costs), grouping by risk of bias (based on the dichotomous measure of the quality of allocation concealment) to assess if results varied by study quality.

We extracted data on the effect of self-management interventions on health-care utilisation and total costs. We also separately extracted data on the methods used in the subset of studies reporting formal cost-effectiveness, cost–utility and cost–benefit analyses. A previously used checklist was employed to assess the quality of the literature.85_{This checklist is based on the Drummond checklist for assessing}

economic evaluations86_{and has been adapted to capture more fully the quality of economic evaluations in}

self-management interventions (see Appendix 3).

Descriptive data on studies, populations and interventions were extracted by two members of the research team working independently. Coding of the type of intervention was conducted on the basis of those extractions by two members of the research team working independently, with disagreements dealt with by discussion. A subset of data on quantitative outcomes were extracted by two members of the research team working independently (n= 50 studies), with the rest of the data extracted by one member and checked by a second.

We also extracted published data on the‘reach’ of each model of self-management support, in terms of the proportion of eligible patients who did not take part in the study, and whether or not long-term conditions additional to the index condition (with the exemption of severe psychosis and dementia) were used as exclusion criteria.

Analyses

Accurate placement of studies on the cost-effectiveness plane requires accurate quantification of the magnitude of both effects on costs and outcomes, which requires particular forms of data beyond simple text descriptions of significance and p-values.

We sought data that would allow us to report a standardised mean difference (or‘effect size’) for health outcomes and costs (Box 2). This generally requires reporting of means, standard deviations (SDs) and sample sizes, although other presentations of those data can be used (such as mean difference statistics), and other presentations (i.e. use of dichotomous outcomes such as rates rather than means) can be

BOX 2 Effect sizes

A RCT assesses the effect of a treatment by comparing the outcomes in the treatment and control groups. Many measures of QoL are continuous, providing a score that varies from 0 up to a maximum based on the number and response range of the items.

Comparing the mean scores of patients in the treatment and control groups gives a good indication of the impact of the treatment. For example, if patients in the treatment group have a mean score at the end of the study of 20, and the controls have a mean of 15, the mean difference is 5 points (i.e. treatment leads to an improvement in QoL of 5 points on average). One difficulty is that it takes an expert to know whether or not a difference of 5 points is important or trivial. A second problem is that studies often use different measures. Knowing that a treatment causes a mean improvement of 5 points when QoL has been measured on two completely different scales makes comparison impossible.

Effect sizes overcome these difficulties by standardising. Essentially, this involves dividing the mean difference from each trial by a measure of the underlying variability of the scores on that outcome (the so-called SD). If scores are generally very variable, then a large mean difference would be required to demonstrate that treatment was better than control. If scores do not vary markedly, then a small mean difference may still represent an important effect of treatment. The mean difference divided by a measure of variability in this way is often described as an effect size.

Standardising in this way means that the difference between treatment and control groups can be described in terms of the same unit (i.e. units of SD). So, if one RCT finds a mean difference of 5 points and the SD is 10, then the effect size is 0.5 (and the difference in QoL is half a SD). A second trial using a different measure might report a larger mean difference of 15 but, if the SD of scores in that trial is 25, then the effect size is actually only slightly increased (15/25_{= 0.6) even though the mean difference is much larger.}

A convention has emerged to judge the magnitude of effect sizes calculated in this way. An effect size of around 0.2 is often described as‘small’, an effect size of 0.5 as ‘medium’ and an effect size of 0.8 as ‘large’.87 These are convenient labels with some validity88,89_{and they provide a useful rule of thumb to assess the effect} of interventions in the context of the wider literature. Nevertheless, decision-makers need to be careful in their interpretation.

Outcomes reported on dichotomous scales (such as proportion of patients using a hospital following treatment) are often reported using different metrics (such as odds ratios, relative risks and NNT). However, they can be translated to an equivalent effect size. For example, a_{‘small’ effect size (0.2) is equivalent to a NNT of} approximately 18, while effect sizes of 0.5 and 0.8 are equivalent to NNTs of approximately 4 and 2.5, respectively.90

Caution should be applied in the interpretation of pooled effects in meta-analyses with‘high’ levels of heterogeneity.

A minority of self-management support trials use cluster allocation to reduce bias associated with contamination. Such studies were identified and the precision of analyses adjusted using a sample size/ variation inflation method recommended by the Effective Practice and Organisation of Care group of the Cochrane Collaboration,95_{assuming an intraclass correlation of 0.02.}

Some studies reported multiple self-management support interventions against a single control. In these cases, we extracted each self-management support intervention as a separate comparison and entered them where relevant in the meta-analysis, dividing the control group sample size appropriately to avoid double counting in the analysis (although this method assumes effect sizes are independent).

The aim of the analysis was to conduct a quantitative systematic review to identify self-management support interventions associated with significant reductions in health services utilisation (including hospital admissions) without compromising outcomes.

The primary analysis was structured by type of long-term condition, with a separate analysis for studies including mixed groups of patients with varying long-term conditions. We also conducted sensitivity analyses to explore the PRISMS categories of conditions (see Table 1) as an alternative typology, restricting those analyses to the two most prevalent categories (PRISMS 1 and 3) (see Table 1).

For each condition category, we present a description of the search and identification of the studies, including the total number identified and the subset of studies including analysable data on QoL, on utilisation and costs and on both outcomes. Our primary interest was on studies reporting both forms of data, because studies that reported only one outcome cannot formally be placed in the cost-

effectiveness plane.

We present the results of the included studies for each condition group according to a permutation plot for all studies reporting both outcomes (i.e. QoL and hospital use and QoL and costs), plotting the effect of interventions on utilisation and outcomes simultaneously and placing them in quadrants of the cost-effectiveness plane depending on the pattern of outcomes (Figure 2). The plot shows the pattern of

Example 2007 Example 1998 Example 2010 Example 2009 Example 1999 Example 2001 – 0.6 – 0.4 – 0.2 0.0 0.2 0.4 0.6 0.8 Utilisation Increased costs

results at the level of the individual study, gives a visual impression of the distribution of studies across the cost-effectiveness plane, and identifies studies in the appropriate quadrant (i.e. those that reduce costs without compromising outcomes) and those in problematic quadrants (i.e. those that reduce costs but also compromise outcomes, or those that compromise both outcomes and costs).