El proceso de evaluación: la evaluación externa

9.1.1 Hypothesis 1: the need for new methods

It is important to propose a new method for analysing observational data and producing answer to queries.

The use of expert judgment is indispensible for analysing certain types of clinical queries as an expert can identify the relations that model a problem domain. Bayesian networks give the flexibility to combine data with knowledge, which we have exploited to generate statistical evidence in the context of an expert-derived BN model. This thesis focuses on using BNs for the problem of clinical evidence derived from observational studies.

159

It is well known that experimental trials are not a possible design option to generate evidence for all clinical questions of interest. Even when a trial is possible, conducting a trial is sometime impractical since the time and cost required are often very high. So observational studies are conducted instead and researchers have commonly used P- values and Confidence Intervals to assess the results produced. However, both the measures have been criticised for their improper use and inadequate interpretation of results.

Bayesian inference can overcome many limitations of the above measures of the classical inference method. This thesis showed that the key advantages that make the Bayesian inference method particularly suitable for evaluating the strength of evidence found from observational data are that it can:

• Report on the probability of interest for evaluation of the strength of evidence. According to critics, the probability of data given the null hypothesis is not essential measure of evidence. Bayesian methods of statistical inference let us calculate what really is required and that is the probability of a hypothesis given data.

• Help to measure evidence and to determine the strength of the evidence. This is done by the use of Bayes Factors. A Bayes Factor, which is the ratio of the probabilities of two competing hypotheses, enables one to decide the strength of the evidence based on its magnitude.

• Support adequate interpretation by quantifying uncertainty.

• Permit inductive inference to assess cause-effect relations. By giving the probability for a hypothesis on the basis of the data Bayesian methods permit inductive inference, which are more appropriate for assessing cause-effect relations

160

A Bayesian network model can represent many types of relations and let us to generate evidence for situations which require many inferences to be performed. In addition, BNs can:

• Model multiple outcomes and therefore, can be used to model complex health care settings such as MDT meetings and waiting times, and derive evidence for its better management.

• Represent both discrete and continuous variables within a framework.

• Provide higher accuracy in incorporating continuous variables using dynamic discretisation algorithms.

To understand the applications of BNs in the clinical and health service domain a survey was performed. This survey covered the types of application, the associated techniques of evaluation and their limitations. We found that:

• Many studies use train-test datasets for evaluation.

• The accuracy of predictions made by BNs has mostly been assessed using the states of one outcome.

• The performance of a BN has commonly measured using: area under the receiver operating characteristic curve (AUC), sensitivity, specificity, positive predictive value, negative predictive value, and HL statistics.

• Models fit to data have been explained using P-values.

Overall, the survey showed that the existing techniques for using BNs in the clinical domain are not sufficient for analysing the strength of associations. This, together with the limitations of the measures of classical inference methods (Chapter 2) and the existing development methods of BNs (Chapters 3 and 6), confirms Hypothesis 1.

161

9.1.2 Hypotheses 2 & 3: The structure of a BN model

from knowledge and data

Data, if available, can demonstrate existence of associations in an expert constructed Bayesian Network model.

Using both the knowledge of experts and data from an observational study we can form a BN to represent associations between its variables

This thesis used a case study and data collected from meetings of a Multidisciplinary Team (MDT) that treats patient suffering with cancer or suspected to have cancer to introduce our research techniques. Initially, it constructed a causal BN by considering an expert’s assumptions about the existence and the direction of causal associations. We evaluated the expert-judged causal associations in the initial BN against data and used the results to revise the initial BN model. The algorithmic steps followed for this are:

• Considered each causal fragment of the expert constructed BN.

• Considered the relations between the fragment variables as a hypothesis.

• Used the available data for testing the hypothesis against all competing hypotheses.

• Revealed if two variables that thought to have a causal relation between them receive support of association from the data.

• Constructed a BN with both knowledge and data.

162

9.1.3 Hypothesis 4: The strength of strong associations

For the BN model we can assess the strength of each association. The results from this assessment can then help to address a relevant query with confidence.

After evaluating the existence of each expert-derived relation, we then assess how strong this association is. The algorithmic steps followed for this are:

• Considered each strong association from the revised BN model.

• Determined the posterior distribution over the parameters of each strong association from data, using an auxiliary multinomial BN model.

• Performed hypothesis tests.

• Produced evidence about differences between the probabilities computed from the data.

• Demonstrated how these differences can be useful for decision-support.

Hence, Chapter 7 confirms Hypothesis 4.

9.1.4 Hypothesis 5: A method for analysing

observational data

It is possible to apply the above techniques to successfully analyse observational data in any domain

Finally, the thesis presented a novel approach for analysing observational data by using knowledge and data. It:

• Combined the above evaluation techniques and presented the complete methodology for the analysis of data.

163

• Sowed that the approach is not restricted to the domain of our case study but can be applied to successfully analyse relations in any domain.

Hence, this part of the research (Chapter 8) confirms Hypothesis 5.