derecha Comunicación

or moderate across samples, cohorts and performance tests.

Shortly after the release of theLACE(van Walraven et al.,2010), theLACE+ was de- veloped byvan Walraven et al.(2010,2012) to predict 30-day death or emergency read- mission, based on the administrative data from the Ontario population. TheLACE+ feature space consists of patient age, sex, discharge method, emergency diagnoses and procedures, alternative cares and count of admission methods, which were selected us- ing a stepwiseLR. The performance of the model was moderately high; however, the selected population size was fairly small. Also, one of the most significant features in the model, the Case-Mix Group feature, can only be calculated by the Canadian Institute for Health Information data.

In the study byLyon et al.(2007), hospital emergency readmission to inpatient within 12-month was modelled using the LR model. The Emergency Admission Risk Likeli- hood Index (EARLI) was designed using inpatient, outpatient andA&Edata from the

HESdatabase, andGPrecords, in addition to the mortality records. TheHESrecords from 2002 to 2003 was linked with mortality records in conjunction with data collected from seventeenPCTs, using questionnaire from the patient over 75 years of age. The performance of the developed solution was very moderate and the dataset size was very small.

2.3

Risk Scoring

Risk scoring, in the healthcare domain, relates to a systematic and effective method of identifying risks and predisposing factors that might give rise to a specific event or allow for partial classification. Examples of the application of risk scoring include: identifying patients who are at risk of a heart attack, have unmet needs, represent complex cases or are socially isolated. Furthermore, risk scoring is a useful aid in ef- ficiently identifying and isolating cohorts of patients for which an intervention will be made or for the purposes of stratification and more general analysis of a given patient population.

2.3.1 Comorbidity Risk Index Modelling 20

2.3.1 Comorbidity Risk Index Modelling

Adjustment for comorbidity is common in clinical outcome risk adjustment. The two most common measures (Austin et al., 2012, Baldwin et al.,2006, Khuu et al., 2015,

Kuo and Lai, 2010, Lieffers et al., 2011, Sharabiani et al., 2012) are the Charlson Comorbidity Index (CCI) (Charlson et al.,1994,1987) and the Elixhauser Comorbidity Index (ECI) (Elixhauser et al., 1998), which are used for predicting admission and mortality. TheCCIand theECIcalculate the frequency of some comorbidity categories, weight them based on the proportion of expected admission or mortality and then linearly sum them up. There have been revised versions of the CCIs (Deyo et al.,

1992,D’Hoore et al.,1993,1996,Ghali et al.,1996,Quan et al.,2005,Romano et al.,

1993) and the ECI (van Walraven et al.,2009), including the most recent adaptation of theCCI(Aylin et al.,2010,DFI,2010,HSCIC,2014d) andBottle et al.(2014) and the actively maintained ECIby the AHRQ(AHRQ,2016b). In addition, Gagne et al.

(2011) introduced a combined version ofCCIand ECIindices, and demonstrated that the combined scoring can boost the performance, especially for short-term prediction of mortality and resource usage.

Moreover, the acute conditions in the England NHS can be categorised into three subgroups (HSCIC,2016k): Ambulatory Care Sensitive Conditions (ACSCs), vaccine- preventable conditions and conditions that usually do not require hospital admission. The ACSCs (Blunt, 2013a,Fund,2016c, ACI, 2015) are seen as potentially avoidable and are highly correlated to multiple admissions over time and quality of care (HSCIC,

2016k). The use of ACSCs has had some success in order to hold commissioners to account and reduce the emergency admission (Bardsley et al., 2013, OECD, 2014,

QualityWatch,2016).

At present, twenty-sevenACSCsare used in theNHSOutcomes Framework (Bardsley et al., 2013, Blunt, 2013b) as markers of improved health outcomes. Between 2001 and 2013, the patterns of change over time for each ACSC across all the deprivation levels were similar. The standardised rates of admission per 100,000 population for conditions in acute group3, chronic4 and other5 groups changed by +0.49%, -0.03% and +0.47%, respectively.

3_{The acute group of ACSC: Acute conditions, Cellulitis, Dehydration, Dental conditions, Ear, nose} and throat infections, Gangrene, Gastroenteritis, Nutritional deficiencies, Pelvic inflammatory disease, Perforated/bleeding ulcer, Urinary tract infection/ pyelonephritis.

4

The chronic group of ACSC: Chronic conditions, Angina, Asthma, Chronic obstructive pulmonary disease, Congestive heart failure, Convulsions and epilepsy, Diabetes complications, Hypertension, Iron deficiency anaemia.

5

The other and vaccine-preventable group of ACSC: Influenza, Pneumonia, Tuberculosis, Other vaccine-preventable.

2.3.2 Operations and Procedures 21 Also, there have been other attempts to classify conditions, such as John Hopkin’s Ag- gregated Diagnosis Groups (ADGs) (JHU, 2014) and Selection of Multipurpose Aus- tralian Comorbidity Scoring System (MACSS) (Holman et al., 2005). The ADGclus- tering method is part of the John Hopkin’sACG system and defines thirty-two clusters of diagnoses. It is used to draw five aspects of morbidity: duration, severity, diagnostic certainty, aetiology and speciality of care. Moreover, the MACSSselected 102 comor- bid conditions based on readmission, Length-of-Stay (LoS) or mortality predictability. Based on the validation results on a large population in Australia,MACSSsignificantly outperformed theCCI.

An alternative approach to comorbidity scoring is to use a cost function, like the

UK’sHRG(HSCIC,2016a), and theUS’s Centers for Medicare and Medicaid Services Hierarchical Condition Categories (CMS-HCC) (Kautter et al., 2014, CMS, 2016b). Commercial implementations of such approaches exist in John Hopkin’s ACG system (JHU,2014) and Verisk Health’s DxCG Risk Analytics (Verisk Health,2016). Also, it has been demonstrated (Billings et al.,2012,Li et al.,2010) that use of cost functions, such asHRGand CMS-HCC, can improve the performance of comorbidity models. On the other hand, the use of comorbidity scoring in predictive models is sometimes criticised. Firstly, unrepresentative versions of the comorbidity scoring, like the CCI, are being used widely, even though their accuracies have been shown to be sensitive to the time-frame and population settings (Bottle and Aylin,2011,Bottle et al.,2014,

Quan et al.,2005). Also, the coding accuracy of diagnoses, cost groups and validation techniques are another set of important factors (Bardsley et al., 2013, Bottle et al.,

2013, Hurst and Williams, 2012). Other criticisms (Bottle and Aylin, 2011, D’Hoore et al.,1996,Quan et al.,2005,Romano et al.,1993) of such scoring methods relate to using very small validation sets and not adjusting for key factors, such as age, gender, deprivations, LoSand temporal patterns.

2.3.2 Operations and Procedures

There is increasing evidence that the quantification of high-risk operations and proce- dures with adequate adjustment can significantly improve the quality of mortality and readmission models (Aylin et al.,2013,Finks et al.,2011,Jhanji et al.,2008,Symons et al., 2013). Yet, unlike comorbidity, there is no generic risk model for operations and procedures (Barnett and Moonesinghe, 2011, Mehta et al., 2016, Moonesinghe et al.,2013,Rix and Bates,2007), and the categorisation is typically carried out using

2.3.2 Operations and Procedures 22 clinical groups. In the UK, the NHS uses Office of Population Censuses and Sur- veys (OPCS) Classification of Interventions and Procedures (HSCIC,2016j). And, in the US, the Public Health Service uses the AHRQ’s procedure categorisation scheme (AHRQ,2016a).

Nonetheless, there have been several attempts to define a scoring mechanism for pa- tients with specific conditions, such as the Royal College of Surgeons Charlson Score (Armitage and Van der Meulen,2010),EuroSCORE(Nashef et al.,1999) and the model is developed by Aktuerk et al. (2016) using HES. It has been shown that such tools can potentially increase measurement effectiveness of a patients’ general risk and the risks associated with complications (Keltie et al.,2014).

In contrast to using HRG and CMS-HCC classifications surrounding cost, alternative approaches that are more focused on operations and procedures include (Mehta et al.,

2016, Pearse et al., 2006): the Bupa’s Operative Severity Score (Bupa, 2016) and the Surgical Outcome Risk Tool (SORT) (NCEPOD, 2011, Protopapa et al., 2014). However, they are constrained by their limited data collection range and very narrow population cohorts.

The Bupa’s Operative Severity Score was developed by theUK’s largest private medical insurer using the Bupa private major score procedure database (PROM). It provides a range of information about treatment options including benefits, risks, burden and likelihood of success, which has been proven to be successful in producing more in- formation about risks of readmission, mortality as well as the effect of interventions. But, there is a concern about its accuracy, because of the discrete way of scoring risks (Devlin and Appleby,2010,Protopapa et al.,2014).

Moreover, the SORT was developed by Protopapa et al. (2014) to predict the preop- erative risk of 30-day mortality after non-cardiac surgery, using cases from 326 NHS

hospitals in England, Wales and Northern Ireland. The model usesLR with forty-five features to predict the risk, including American Society of Anesthesiologists Physical Status (ASA-PS) grade, the urgency of surgery, high-risk surgical speciality, surgical severity, cancer and age. The model performance is comparable (Moonesinghe et al.,

2013,Protopapa, 2016) with leading preoperative risk models: the Portsmouth POS- SUM (Prytherch et al.,1998) and the Surgical Risk Scale (SRS). The model incorpo- rates a manageable set of features and has a moderately high performance in overall, but more external validation of the model is necessary to prove its resilience.

In the following chapter, the main complexity levels in data extraction and analysis stages are outlined.

Chapter 3

Complexity Levels of Models

Generally, data-driven approach needs to filter results based on statistical significance, importance and novelty, in order to identify significant correlations from electronic health records (EHRs). A true data interpretation needs development and implemen- tation of guidelines and clinical models to allow unambiguous representation of clinical meaning (Jensen et al.,2012).

There are considerable challenges in comparing the predictive models across interna- tional boundaries (Lewis et al.,2011,Mihaylova et al.,2011). Four distinctive aspects in the analyses of research studies are: data, produced features, modelling, and perfor- mance. In each area, there are several layers of complexity, which influence the quality and interpretability of the models. In the following subsections, these four aspects are presented.