Informes de Auditorias

An electronic medical record system that uses CPGs should allow changes to be applied to its CIGs systematically when changes to CPGs occur. In order to provide automated or semi-automated support to the CIG modeller when applying these changes, an evolv- ing CIG conceptual model is required. The specification of the evolving CIG conceptual model should contain explicit and invariant definitions of all the elements that are af- fected by CPG changes. Recall the types of CPG changes summarised in Table 5.1. A language model and a concrete syntax can then be mapped from this evolving CIG con- ceptual model, enabling the automated creation of tools that can support the authoring and maintenance of CIGs through smart-editing features.

An individual CPG has one or more guideline recommendations. Each of the guideline recommendations has one or more conditions, which must be satisfied in order for a particular recommended action to be executed. Each condition has a decision variable that is tested to determine whether it is appropriate to execute one or more related

Table 5.1: Types of CPG changes [2]

.

Category Type of change

Updating decisions

Adding a decision variable

Changing a value of a decision variable Removing a decision variable

Changing a decision variable

Updating actions

Adding a recommended action Removing a recommended action Changing an action verb complement Changing a recommended action Updating guideline recommendations Adding a guideline recommendation

Removing a guideline recommendation

recommended actions. The actual value of a decision variable that is tested whether a particular set of recommended actions is appropriate or not, is referred to as a vari- able value. Different variable values can be stored in different units of measure. Each recommended action within the guideline has an action verb that determines the type of action to be taken, and an action verb complement that completes the sense of the action verb by referring to a direct or indirect object. When a new version of a CPG is introduced, any of these concepts can be affected.

Let me illustrate this with an example, so as to clarify the context in which evolving CIGs are used. I use the CPG for determining ART eligibility in children and adults aged five years and above from the Malawi guidelines for the clinical management of HIV of 2011 and 2014. Extracts of the CPG from the guideline documents of 2011 and 2014 are presented in Fig. 5.1. The elements that make up the CPG are highlighted and the change has been explicitly pointed out in the figure: when a new version of the CPG was introduced in 2014, a change of type ‘changing a type of a decision variable’ had occurred.

5.3

Materials and methods

In order to evaluate the feasibility of creating a modelling architecture that can support evolving CPG formalization, I started by defining a four-layer CIG modelling architec- ture that is based on a model-driven engineering approach. Thereafter, I defined a formal evolving CIG conceptual model that explicitly specifies the elements that are affected when CPG changes occur. I later systematically map the evolving CIG conceptual model into a comprehensive language model that includes denotational semantics and a formal abstract syntax. I mapped the formal language model into a sufficient concrete syntax specification for a model-based DSL that is tailored towards the modelling of evolving

Figure 5.1: CPG for determining antiretroviral therapy (ART) eligibility in 2011 and 2014 of Malawi HIV CPGs, with each fine-grained element colour coded (see legend on

the right of the figure as insert)

CIGs, that I have named FCIG. Finally, I carried out a scenario-based and an empirical evaluation of FCIG using CPGs from Malawi.

5.3.1 Research questions

Recall the second research question RQ2 in Section 3.1:

Can a model-driven engineering approach adequately support the modelling of an evolving CIG?

In this study, I split the evaluation in two dimensions to address research question RQ2. The first dimension focused on CPG representation adequacy. The second dimension focused on the capacity to support the CIG modeller when applying a specific change. As a result, I formulated two subquestions to guide the evaluations for each of the two dimensions as follows:

RQ2-S1: Can FCIG be used to model an evolving CIG adequately?

RQ2-S2: Can FCIG directly support the application of fine-grained CPG changes? I investigated the representational accuracy and adequacy of FCIG including FCIG’s ability to provide smart-editing support for the CPG elements that are affected by a

change. I obtained CPG documents from Malawi and encoded guideline recommen- dations from the CPG documents using FCIG. To answer sub-question RQ2-S1, I measured the number of guideline recommendations that were encoded using FCIG suc- cessfully. To answer RQ2-S2, I measured the number of language concepts that were directly supported by smart-editing features when a language concept was affected by a CPG change.

5.3.2 Research approach

I used a quantitative approach to conduct the two studies for this evaluation. The first, was to assess the adequacy in encoding an evolving CIG using FCIG. The second, was to assess FCIG’s ability to support, and awareness of, evolving CIG concepts that are affected when a change occurs. I collected quantitative data regarding the number of guidelines and associated guideline recommendations that were able to satisfy each of the two requirements.

5.3.3 Study design

I obtained a convenient sample of CPG documents from the Malawi Ministry of Health. The CPG documents were the ones that the officials from the Central Monitoring and Evaluation Division (CMED) of the Malawi Ministry of Health had provided. From the convenient sample of CPG documents, I selected the Malawi integrated guidelines for the clinical management of HIV and the guideline for community integrated manage- ment of child illnesses (IMCI) because they are comprehensively documented and are operationalised within the national EMR systems in Malawi. I further used the stratified random sampling technique to select representative samples of guideline recommenda- tions from the integrated guidelines. Stratified random sampling allows a researcher to obtain a sample that best represents an entire population under study by ensuring the presence of key subgroups within a population [231, 233]. I used the Malawi medical concept dictionary to obtain a controlled set of medical vocabulary.

5.3.3.1 Evaluation study one: Assessing FCIG’s CPG representation ade-

quacy

In this study, I set out to answer research subquestion RQ2-S1. Noting that the structure of guideline recommendations are usually similar within a particular guideline, I strati- fied the guideline recommendations within the integrated guidelines by guideline. From each guideline, I selected a weighted random sample of guideline recommendations for

inclusion. I derived the randomised sample through a digital randomisation application called Random UX1. Thereafter, I encoded the selected guidelines using FCIG.

5.3.3.2 Evaluation study two: Assessing FCIG’s ability to support CPG

changes

In this study, I set out to answer research subquestion RQ2-S2. Noting that the opera- tions required to apply a particular type of change are similar, I stratified the guideline recommendations from the Malawi HIV guidelines by type of CPG change. For each type of CPG change, I selected a weighted random sample of guideline recommenda- tions from the 2008 version of the guidelines. I derived the randomised sample through a digital randomisation application called Random UX1. I later encoded each selected guideline recommendation using FCIG. For each encoded guideline recommendation, I applied its associated change that was required to update it to its 2011 version.

5.3.4 Criteria to address the research question RQ2

In this section, I describe the criteria that I used to address the second research question RQ2 through subquestions RQ2-S1 and RQ2-S2.

5.3.4.1 Criteria to address research subquestion RQ2-S1

In order to address research question RQ2-S1, I recorded the total number of guideline recommendations in the guideline strata and attempted to model the guideline recom- mendations using FCIG. Recall, from the discussions in Chapter4, that the conceptual models of existing CIG modelling languages do not have explicit concepts for specifying fine-grained CPG representation primitives such as action verb and verb complement. Hence, adequate representation of a recommendation in a CPG was regarded as an in- stance whereby FCIG modelling primitives were used to specify all guideline recommen- dation concepts, including fine-grained constructs. I recorded the number of guideline recommendations from the guideline strata that were adequately encoded using FCIG in order to measure task completion rate.

I formulated the following hypothesis to guide my analyses: H0: FCIG can not be used to complete CPG encoding tasks.

H1: FCIG can be used to complete CPG encoding tasks adequately.

5.3.4.2 Criteria to address research subquestion RQ2-S2

In order to address research question RQ2-S2, I encoded the 2008 versions of the guideline recommendations from the CPG change type strata using FCIG. I recorded the total number of required CPG changes in the CPG change type strata. Thereafter, I attempted to apply the individual changes to the encoded guidelines. I took note whether the evolving CIG semantic elements that were affected by each change were directly supported by smart-editing features. I tracked the number of individual CPG change occurrences whose affected elements were directly supported with smart-editing features. This allowed me to measure task completion rate.

I formulated the following hypothesis to guide my analyses:

H0: FCIG can not provide smart-editing support for CPG maintenance tasks.

H1: FCIG can provide smart-editing support for CPG maintenance tasks.

5.4

A four-layer architecture for modelling evolving CIGs