Rather than drawing controls from across the whole of England, we limited our population of potential controls to people who lived in twelve matched areas of the country. So, for our national matching we used a two-stage approach, where first we matched at the area level and then at the
individual person level. We limited the pool of potential controls in this way because hospital utilisation rates vary by area.59,60 An additional advantage
of this approach is that it also greatly improved the computational ease of finding controls. One assumption is that for areas such as Devon, the participating practices were sufficiently similar to the county a whole. We selected four potential comparator sites for each of three virtual ward study sites, based on the Office for National Statistics (ONS) Corresponding Health Areas.49 See Table 4. The ONS selects these sites based on their
squared Euclidean distance for a range of 42 variables related to
demographics, household composition, housing, socioeconomic factors, employment and industry.61,62 The ONS considers health areas to be
“extremely similar” if the squared Euclidean distance (SED) is less than 2.02; “very similar” if it is less than 5.06; “similar” if it is less than 10.12; “somewhat similar” if it is less than 20.24; and “less similar” if the SED is greater than 20.24.49
Table 4. Comparator areas used for national matching
Site Period of study ONS
corresponding health areas Squared Euclidean Distance Similarity of corresponding health area Croydon 15 May 2006–1 September 2010
Enfield 3.39 Very similar Waltham Forest 4.86 Very similar Greenwich Teaching 6.35 Similar
Redbridge 13.22 Somewhat similar
Devon* 1 October 2008– 1 September 2010
Somerset 1.54 Extremely similar Cornwall and Isles of
Scilly 1.77 Extremely similar Shropshire County 1.78 Extremely similar Herefordshire 1.95 Extremely similar
Wandsworth 1 March 2009–1
September 2010
Hammersmith and Fulham
3.61 Very similar
Camden 10.72 Somewhat similar Islington 12.79 Somewhat similar Westminster 16.94 Somewhat similar
We contacted the Director of Nursing at the primary care trust in each potential comparable area to check whether a virtual ward scheme or equivalent was operating during the study period. Based on the responses we received, we excluded two such areas, namely North Yorkshire and York PCT and Dorset PCT. Instead, we used Shropshire County PCT and
Herefordshire PCT as the third and fourth comparator areas for Devon, having confirmed that neither site had virtual wards or equivalent in operation during the study period.
We excluded individual residents of the comparator sites who had
previously been resident in one of the virtual ward study sites, or who had registered with a general practice in the virtual ward study sites, from being controls. This was because such patients might have been affected
indirectly by the operation of the virtual ward (a so-called “spill-over” effect). All residents of the remaining comparator sites were eligible to be selected as control patients, provided they were aged over 18.
Details of our approach for national matching are described in Figure 4. Figure 4. Methods for selecting national controls
We sought to match each virtual ward patient to at least one control based on
variables derived from datasets that were available to us nationwide, namely Hospital Episode Statistics (HES) data, index of multiple deprivation scores, and a dataset from the Office for National Statistics containing dates of death for individuals with a HES record. We chose controls who were similar to the intervention patients in terms of their prognostic score, age, sex, various categories of prior hospital utilisation, total number of chronic health conditions, area-level deprivation score,50 as well as 15
markers of specific health needs from the inpatient hospital record in HES, namely: anaemia, angina, asthma, atrial fibrillation and flutter, cancer, cerebrovascular disease, congestive heart failure, chronic obstructive pulmonary disease, diabetes, history of falls, history of injury, hypertension, ischemic heart disease, mental health conditions, and kidney failure.
We based the prognostic score we used for the national matching on a predictive risk model that we developed using HES data. In two of the sites (Croydon and Devon), this model differed from the predictive risk scores used to identify patients who were offered admission to a virtual ward, as the latter used a model that included GP clinical data, which were not available nationally. In the remaining site (Wandsworth), the PARR model was used, which does not include GP clinical data.
For each study site, we developed a series of prognostic models to predict the
likelihood of an individual’s experiencing an emergency hospital admission in the next 12 months, calibrated according to local patterns of hospital use. These built on variables used in the PARR model 63 but predicted admission rather than readmissions.
In building these models, we excluded any information about patients who were ever admitted at a virtual ward, because we assumed that their pattern of hospital use might have been altered by the intervention.54
We developed the prognostic models using a split-sample approach and we described the accuracy of the models in terms of their positive predictive value (PPV) and sensitivity, as well as the area under their receiver-operating characteristics (ROC) curves.15
After fitting the prognostic score model, we applied the calculated beta coefficients to the intervention group and to patients resident in the comparator sites in order to generate the scores.