Capítulo III: Evaluación Externa
3.2 Influencia del análisis en cementos Pacasmayo
As mentioned ACM is mainly composed of two sub-systems:
- A predictor model which depending on the input data forecasts the response to anemia pharmacological therapy for a specific patient;
- An algorithm that using the predictor model extracts the optimal policy to achieve the established clinical outcome for anemia management.
The predictor model is the base for the policy extractor that is the model used in runtime environment to generate ESA Iron dosages suggestions. Considering this, the actual risk for a wrong dosage suggestion is related to the possibility of an erroneous prediction of predictive model. Our approach is therefore to evaluate the probability of an erroneous prediction and its consequences in all those situations that could generate a risk for the patient.
First we examine the system using FMEA (Failure mode and effects analysis) and PHA (Process Hazard Analysis) for different errors resulting in an observable (faulty) system behavior. An example of a (faulty) system behavior is the wrong prediction of next hemoglobin level. ACM elaborates therapy suggestions only if provided with the necessary information to perform the prediction of ESA Iron therapy response; therefore, when not enough information is provided to ACM no suggestion will be elaborated and physicians will have to perform their own prescription as they normally do. No risk is therefore associated to possible erroneous ACM behavior due to lack of information.
In a second step we evaluate hazards and respective risks caused by this system behavior. In order to derive risks from hazards we assess the probability and the severity of the respective harm. After the risk analysis, 24 risks have been identified trying to span all different cases, from very low, through normal, up to very high hemoglobin’s values; a couple of examples of defined risks are shown in tables 5.5 and 5.6. Probability is assessed using the available data, for example for risk R0001 given the test dataset (X, Y ) where X ≡ (x1, · · · , xN) is the set of input
vectors (patient data as described in Chapter 4) and Y ≡ (y1, · · · , yN) is the corresponding set
of output (next Hb level), the actual future hemoglobin HBa(t + 1) and the predicted future
hemoglobin HBp(t + 1), the prediction error is e ≡ HBa(t + 1) − HBp(t + 1) we have:
p(Hb < 7g/dl|X) =count(Hb < 7g/l) N p(e > 1.5g/dl|X) = e > 1.5g/l N P robability = p(Hb < 7g/dl|X) · p(err > 1.5g/dl|X) (5.6)
Table 5.5: Example of potential risk generated by the use of ACM.
Risk Number R0001 Cause(s) and conse-
quences
Patient is in special clinical conditions or erroneous data are provided to ACM, thus the model does not contem- plate some important information to correctly perform the prediction of the next hemoglobin level Patient anemic sta- tus is highly critical (Hb < 7g/dl). The predicted next hemoglobin level overestimates the actual one of more than 1.5g/dl.
Probability 1 · 10−5
Comment In some peculiar situation it is possible that patient re- sponse to therapy is strongly influenced by factors other than those considered by the model. The model is thus not able to correctly predict the next hemoglobin level and specifically overestimates it. This could eventually lead to an ESA Iron under-dosage.
Behavior ESA / Iron under dosage
Hazard Very critical anemic status not solved
Impact Patient
Severity Level 4 Probability Level 4
Table 5.6: Example of potential risk generated by the use of ACM.
Risk Number R0007 Cause(s) and conse-
quences
Patient is in special clinical conditions or erroneous data are provided to ACM, thus the model does not contemplate some important information to correctly perform the pre- diction of the next hemoglobin level. Patient anemic status is within target (11g/dl < Hb ≤ 12g/dl). Predicted next hemoglobin level underestimate actual one which increase for two consecutive months, error > 1.5g/dl
Probability 3 · 10−5
Comment In some peculiar situation it is possible that patient re- sponse to therapy is strongly influenced by factors other than those considered by the model. The model is thus not able to correctly predict the next hemoglobin level and specifically underestimate it. This could eventually lead to an ESA Iron over-dosage.
Behavior ESA Iron over dosage Hazard Patient goes over targets
Impact Patient
Severity Level 1 Probability Level 4
Risk Evaluation and Risk Control
Following step is to evaluate the identified risks and to mitigate them when possible. In our case mitigation is basically the fact that ACM is just generating suggestions, thus the doctor can evaluate and in case reject suggested doses that are not optimal for the patient. It is important to distinguish between doses with an undesired but also unexpected outcome from those where the undesired outcome can be expected. In the first case, that can be caused by different reasons like intercurrent events or very unusual clinical condition, there is no real error from ACM and the doctor would have most probably committed the same mistake, so in this case ACM is not generating an additional risk. In the second case the doctor, by his experience and knowledge of a specific patient, may have more information respect to ACM and identify that the suggested dose is not optimal. Anyway it is very difficult to distinguish among these two situation, thus, to be on the safe side, we considered any error from ACM to be potential a risk for the patient. A mitigation is proposed for all the risks ≥ 10. In our case, this range includes both the unacceptable risks and the ALARP risks. For the two risks presented as example the relative mitigation is described in tables 5.7 and 5.8.
Table 5.7: Mitigation of Risk R0001.
Risk Number R0001
Risk Very critical anemic status not solved
Mitigation ACM just provides a therapy suggestion, physicians have to evaluate patient conditions and to prescribe by themselves the actual drug dosages; therefore in this case, as specified in the user manual, if they have indications that ACM suggestions may be wrong they will not follow it and, as they normally do, they will formulate their own prescription and they will perform any other action to solve understand patients critical anemic status (i.e. transfusion). Probability level goes from 4 to 2. Severity level remains 4.
Table 5.8: Mitigation of Risk R0007.
Risk Number R0007
Risk Mild anemic status not solved Mitigation No mitigation needed.
Finally figures 5.6 and 5.7 represent the risk evaluation sheets before and after risk control, it can be noticed that after risk control no unacceptable risks remain.
Figure 5.6: Risk Evaluation Sheet before risk control, in each cell the corresponding number of risks belonging to that category (Probability and Severity) is reported.
Figure 5.7: Risk Evaluation Sheet after risk control, in each cell the corresponding number of risks belonging to that category (Probability and Severity) is reported.