El marco discursivo liberal sobre el agua

The main reason for using BNs is to enable reasoning under uncertainty. Uncertainty in this context refers to students’ behaviour with respect to problem scenarios. This implies their knowledge and understanding, demonstrations of relevant abilities/skills, and performances, are uncertain and can only be inferred from observed behaviour. For example, in Figure 4.7 (a subset of the LAP model, where the nodes UCV, ACF, MAC, and KNC correspond to nodes

9 (“UseCorrectValues” (UCV)), 8 (“ApplyCorrectFormulae” (ACF)), 10

(“AbilityToMakeAppropriateCalculations” (MAC)), and 28 (“KnowledgeOfConcept” (KNC)), respectively, in the LAP model (see Figure 4.3), the expressions, (p KNC=high)and

( )

p MAC=high , referred to as marginal probabilities, represent measures of beliefs in the hypothesis represented by the variables. It cannot be stated with certainty that a student has knowledge of the addressed concept(s) or that he/she has applied the correct formulae for a laboratory activity yet to be undertaken. The marginal probabilities can be evaluated using the

parameter entries in the CPTs and PPTs of the nodes. For example,

( ) ( | ) ( )

p KNC =high = p KNC =high ACF =applied × p ACF =applied +

( | ) ( )

( | ) ( )

p KNC=high ACF =notApplied ×p ACF =notApplied (4.1)

All the values required to evaluate this expression are derived from the parameter entries in the CPT of the node, KNC, and PPT of the node, ACF. For the purposes of this example, it is assumed that there is no other node with a link to the node, KNC.

UseCorrectValues (UCV) used partiallyUsed notUsed 33.3 33.3 33.3 ApplyCorrectFormula (ACF) applied partiallyApplied notApplied 33.3 33.3 33.3 AbilityToMakeApproCalculations (MAC) low high 50.0 50.0 KnowledgeOfConcept (KNC) low high 50.0 50.0

Figure 4.7: A sub-model of the LAP Model

If after undertaking the laboratory activity, it is evidenced that the student had applied the correct formulae (ACF = applied) and used the correct values (UCV = used) for the specified calculation, the evidence is entered into the model by clamping nodes ACF and UCV to their evidenced or instantiated states. It still, at this point, does not mean with certainty that the student definitely made appropriate calculations or that he/she definitely has knowledge of the addressed concept(s). However, there is increased probability (belief) that the student made appropriate calculations, and that he/she has knowledge of the addressed concept(s). In this case, the value of the expression, p KNC( =high ACF| = applied), referred to as conditional probability, is directly obtained from the CPT of node KNC. That is, the measure of believe in the student’s knowledge of the addressed concept is revised after receiving the evidence.

The most important use of BNs is in revising measures of belief in the light of actual observations of events. Suppose that it was not known (not evidenced) that the student applied the correct formulae, probably because the given laboratory activity did not include a task that would elicit the necessary behaviour from the student to facilitate the derivation of the evidence, but the instructor had ascertained, through some other means of assessment (e.g. physical observation), that the student definitely has knowledge of the addressed concept. This information can be entered into the model by clamping the node, KNC, to high, and then used to determine the revised probability that the student; applied the correct formulae,

( | )

p ACF =applied KNC =high ; made appropriate calculations, p MAC( =high); and

calculated using Bayes formula, values from the CPT and PPT of nodes ACF, and KNC, and the marginal probability of KNC, (p KNC=high), as:

( | ) ( )

( | )

( )

p KNC high ACF applied p ACF applied

p ACF applied KNC high

p KNC high

= = × =

= = = ₌

The calculation of the revised marginal probability of MAC, (p MAC=high), will depend on the value derived for p ACF( =applied KNC| =high) and no longer on the value of

( )

p ACF =applied in the PPT of node ACF. Thus, any evidence or information received is

used to revise the measures of belief for all affected hypotheses. This is referred to as network update or evidence/information propagation.

4.5

S

The construction process of the LAP model, with the assistance of domain experts, has been described from structure realization to calibration. The model belief estimation process has also been highlighted. Though the assistance of domain experts were used for the construction of the model, BN-based models can also be constructed from data. The next chapter investigates the optimal approach for the construction of the assessment model for the performance assessment of students’ laboratory work in the VEL environment, with respect to expert-centred and data-centred BN construction approaches. That is, to empirically investigate if the expert-centred approach is the best approach for constructing the LAP model, or if it is possible to derive an improved or better model using the data-centred BN model construction approach.

The whole difference between construction and creation is exactly this: that a thing constructed can only be loved after it is constructed; but a thing created is loved

before it exists.

Charles Dickens 1812 - 1870

CHAPTER 5

INVESTIGATION

OF

THE

OPTIMAL

CONSTRUCTION

APPROACH

FOR

THE

BAYESIAN NETWORK-BASED ASSESSMENT

MODEL

5.0

I

A BN -based model (the LAP model) for performance assessment of students’ laboratory work, in the VEL environment, was constructed with the assistance of domain experts. The details of its construction process and evaluation are given in Chapter 4. Though BN model constructors (researchers working with BN models), ab initio, relied only on domain experts to define both the structure and parameters of a model (which line was toed in constructing the LAP model), currently algorithms exist to construct BN models from data. Hence, BN model construction can be categorized under two approaches: expert- and data- centred. Consequently, there are three techniques to BN model (structure + parameters) construction:

total expert-centred (tecen)

total data-centred (todacen)

semi data-centred (sedacen).

In tecen approach, the BN model is a product of domain analysis, whereby domain expert(s) completely specify both the qualitative and quantitative components of the model. The todacen approach uses algorithms to generate both the qualitative and quantitative components of a BN model from data. The generation of the qualitative component from data is referred to as structure learning, and the generation of the quantitative component referred to as parameter learning. The sedacen approach is a hybrid framework whereby domain experts assist in the creation of the qualitative component of a BN model, while the quantitative component is learnt from data.

Having constructed the LAP model with the assistance of three domain experts, it was necessary to find out if the LAP model (a tecen model) is the best or optimal model for the assessment of students’ laboratory work in the VEL environment. That is, to empirically investigate if the expert-centred approach is the best approach for constructing the LAP model, or if it is possible to derive an improved or better model using the data-centred BN model construction approach. This required the construction of sedacen and todacen models from data, based on the same set of domain variables as the tecen model. The aim is to compare the performances of the sedacen and todacen models to the performance of the tecen LAP model (the reference model), based on a set of performance metrics.

There are two possible sources of sample datasets that can be used for the construction of the sedacen and todacen models. First, sample domain historical sample dataset(s) on students’ laboratory work performance assessment, with respect to a set of performance indicators and their respective criteria, could be used if available. This option was not possible as there are

no existing historical sample datasets, for the domain of engineering students’ laboratory education (with respect to performance-based assessment of students’ laboratory work), to the best of the author’s knowledge. The second option is the use of simulated sample datasets. Often, researchers needing to undertake empirical investigations, with respect to structure and/or parameter learning, create frameworks that would allow them to generate the required sample datasets from a reference model. This approach has been adopted by a number of researchers including [181][183][184][185]. The procedure starts with an existing model (the reference model), generates datasets from the Joint Probability Distribution (JPD) represented by the model, and then uses learning algorithms to attempt to retrieve the reference model from the datasets. The retrieved (learnt) model is then compared with the original model that generated the dataset [185]. This procedure is commonly used for evaluating learning algorithms, and was deemed appropriate for this empirical investigation because of lack of existing historical sample datasets. The idea is that if, using the above procedure, the algorithms fail to retrieve (induce) the reference model, a comparable model, or a better model (in terms of performance) from the sample datasets generated using the JPD encoded by its structure, then it may imply that the algorithms will also fail to induce a comparable model, or a better model from sample datasets generated from other sources. That is, failing to retrieve the reference model, the algorithms should at least learn a model whose performance is comparable to or better than that of the reference model. It is assumed that the reference model is the existing optimal model. If the algorithms learn a model whose performance is significantly better than that of the reference model, then the learnt model is taken to be the optimal model, else the reference model is taken to be the optimal model. Optimal, in this context, refers to the model which is best in terms of the adopted optimality criteria [186]. First, section 5.1 details the different BN model construction approaches. Section 5.2 describes the procedure for the empirical investigation, highlighting how the different sedacen and todacen models used in the investigation were constructed, and the model test process, while section 5.3 highlights the criteria and comparative tools used to compare the models. The results of the investigation are presented in section 5.4, while the observations and attendant discussion are given in section 5.5. The chapter is summarised in section 5.6.