Risk model 1: the point of admission to hospital Development
After exclusions from the full database (as described in Chapter 2), half of the data were randomly selected for the development database. Table 22 displays the variables that were selected via the SSVS method for the Bayesian logistic regression model.
TABLE 21 A comparison of the observed numbers of patients with worsening AKI in each AKI group and the numbers predicted by the model in the second population using the Hosmer–Lemeshow test for the validation of risk model 3
Risk category (%)
No worsening (n) Worsening (n)
χ2a p-value
Observed (expected) Observed (expected)
≤4 412 (433) 33 (40) 44.8 <0.001
>4–10 227 (229) 17 (15)
>10 76 (69) 10 (17)
a With 1 degree of freedom.
TABLE 22 Variables selected for the Bayesian logistic regression model: admission
Variable Median 95% credible interval Mean
Intercept –1.1835 –1.2318 to –1.1375 –1.1839 Variables with probability of inclusion > 0.5
Episode 0.1352 0.0800 to 0.1920 0.1353
Primary diagnosis: diseases of the genitourinary system 0.0988 0.0530 to 0.1398 0.0980 Charlson Comorbidity Index score 0.1480 0.0966 to 0.1975 0.1478 Baseline eGFR –0.2255 –0.2812 to –0.1693 –0.2255 Proteinuria 12-month test count 0.0683 0 to 0.2045 0.0047 HbA1crecent result provided –0.0839 –0.3052 to 0 –0.0760
Variables with probability of inclusion > 0.25
Primary diagnosis: diseases of the respiratory system 0.0009 0 to 0.1069 0.0290 MSU or CSU in previous 2 weeks 0.0007 0 to 0.1045 0.0335 Platelet result available –0.0005 –0.1364 to 0 –0.0246
This model considers only the main effects. SSVS calculates the probability of inclusion of variables in the model. A high inclusion probability implies that the variable is important. A good cut-off point for this probability is 0.5. Based on this, the model should include six variables: spell type, primary diagnosis – diseases of the genitourinary system, Charlson Comorbidity Index score, baseline eGFR, proteinuria 12-month test count and whether or not the HbA1crecent result has been provided. Based on this model we can make the following statements, which are illustrated in Figure 23:
1. The odds of developing AKI are 1.15 times greater for non-elective admission than for elective. 2. The odds of developing AKI are 1.1 times greater for an episode with primary diagnosis = 12; that is,
a diagnosis of a disease of the genitourinary system.
3. The odds of developing AKI are 1.16 times greater for an episode with Charlson Comorbidity Index score = 10 than for an episode with Charlson Comorbidity Index score = 9.
4. The odds of developing AKI decrease by a factor of 0.80 when the baseline eGFR increases by 1 unit. 5. The odds of developing AKI increase by a factor of 1.07 when the proteinuria 12-month test count
increases by 1 unit.
6. The odds of developing AKI decrease by a factor of 0.92 when the recent HbA1cblood result is provided. The effects of a diagnosis of a disease of the respiratory system, of MSU or CSU taken 2 weeks earlier and of the provision of recent PLT are very small, which means that the odds are very close to 1, suggesting that the risk of AKI is the same.
Validation
Validation of the developed Bayesian model was performed as described in the methods. Half of the data set had been randomly selected for use in model development, and half had been selected for
model validation.
For this Bayesian model, the in-sample sensitivity was 35% and specificity was 98%, and out-of sample sensitivity was 32% and specificity was 97%.
0 20 40 60 80 100 120 140 Variables 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 Probability of inclusion
The specificity, which is the probability that the model accurately predicts the episodes when AKI will not develop or, better, who will not develop AKI, is high. The sensitivity is poor.
Summary
The results of the Bayesian modelling demonstrate excellent specificity and hence this model is good at defining patients at very low risk of having AKI; however, the sensitivity is poor and so the model cannot accurately determine which patients are at high risk. This model would, therefore, be good for defining the population of patients who do not need renal function testing to assess for AKI; by inference it can, therefore, determine the population for testing. However, a large number of the population for testing will be false positives.
Risk model 2: predicting new acute kidney injury at 72 hours Development
After exclusions from the full database (as described in Chapter 2), half of the data were randomly selected for the development database. Table 23 displays the variables that were selected via the SSVS method for the Bayesian logistic regression model.
This model considers only the main effects. SSVS calculates the probability of inclusion of variables in the model. A high inclusion probability implies that the variable is important. A good cut-off point for this probability is 0.5. Based on this cut-off the model should include only one variable, the Charlson Comorbidity Index score. Based on a cut-off of 0.25 only one other variable is included, the outpatient attendances in the last 12 months. The second and third columns of Table 23 display the median effect and 95% credible interval (for the effect) of the selected variables. Based on this model we can make the following statements, which are illustrated in Figure 24:
1. The odds of developing AKI are 1.20 times greater for an episode with Charlson Comorbidity Index score = 10 than for an episode with Charlson Comorbidity Index score = 9.
2. The odds of developing AKI decrease by a factor of 1 when the outpatient attendance in the last 12 months increases by one additional attendance. This suggests that this variable has a very insignificant impact on the risk of developing AKI.
TABLE 23 Variables selected for the Bayesian logistic regression model: 72 hours
Variable Median 95% credible interval
Intercept –1.3377 –1.5866 to –1.2238
Variables with probability of inclusion > 0.5
Charlson Comorbidity Index score 0.1744 0 to 0.2714 Variables with probability of inclusion > 0.25
Validation
Validation of the developed Bayesian model was performed as described in the methods. Half of the data set had been randomly selected for use in model development and half had been selected for model validation. For this Bayesian model the in-sample sensitivity was 33% and specificity was 98%, and out-of sample sensitivity was 32% and specificity was 97%.
The specificity, which is the probability that the model predicts accurately the episodes without AKI or, better, who not develop AKI, is high. The sensitivity is poor.
Summary
The results of the Bayesian modelling demonstrate an excellent specificity and hence this model is good at defining patients at very low risk of having AKI; however, the sensitivity is poor and so cannot accurately determine those patients at high risk. This model would, therefore, be good for defining the population of patients who do not need renal function testing to assess for AKI; by inference it can, therefore, determine the population for testing. However, a large number of the population for testing will be false positives.
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 Probability of inclusion 0 20 40 60 80 100 120 140 160 180 Variables
Chapter 4 Qualitative analysis
Analysis of qualitative data
This chapter reports on the findings of the individual interviews with renal consultants and the focus group with outreach nurses from critical care conducted approximately 18 months following the introduction of the alert system. The purpose of this part of the study was to gather information on perceptions of the system, levels of interaction with the technology, communication with medical teams, impacts on patients and clinical practice, and recommendations for improvements.
Table 24 lists the main themes and subthemes emanating from the data. Quotes from the respondents are used in the analysis to justify interpretation, with ‘C’ referring to a consultant response and ‘ON’ referring to an outreach nurse response. Further numerical coding refers to the interviewee code for the
respondents and the page number of the transcript from which the quote was derived. For example, C3:4 = consultant 3, transcript page 4.
At the time it was assessed, the AKI alert system at EKHUFT was delivered through QlikView, a browser-based reporting tool; references to ‘QlikView’ in quotes are, therefore, referring to the AKI alert system.
Each section of the analysis is followed by a short summary highlighting the main points raised.
TABLE 24 Analytical themes and subthemes
Themes Subthemes
1 Introducing the alert system 1.1 Getting to know the alert system
1.2 First reactions to the system 2 Using the technology 2.1 General comments
2.2 Issues with user friendliness
2.3 Timing and workload
2.4 Accessing data and making decisions 3 Interacting with medical teams 3.1 Finding the right doctor
3.2 Discussing the cases
3.3 Awareness of the alert system 4 Monitoring cases 4.1 To follow up or not to follow up?
4.2 Acting on advice
5 Impacts on patient care and clinical practice 5.1 Making a difference to patient care
5.2 Impacts on clinical practice
5.3 Perceptions of cost-effectiveness
6 Recommendations for improvements 6.1 Improving trust-wide knowledge of the alert system