SELECCIÓN DE VARIABLES

KERMIT was extended with dialogue capabilities to facilitate self-explanation (SE), in my M.Sc. research. For a detailed discussion on how KERMIT was enhanced to support self-explanation see (Weerasinghe, 2003; Weerasinghe & Mitrovic, 2006). Some of the important details are discussed here. As mentioned in Section 2.3, KERMIT-SE provides a problem-solving environment and uses dialogues as additional learning support. User responses are limited to selecting the correct explanation from a pre-defined list. All the systems that facilitate self-explanation prior to KERMIT-SE prompt students to explain most of the problem-solving steps, requiring students to point out the definitions/theorems used. We believed this approach puts too much burden on able students. Therefore, our tutor, KERMIT-SE prompted for self-explanation only when the student violates a constraint, which indicates missing/erroneous knowledge or a slip. The tutor is thus able to customise self-explanation based on the student solution so that the knowledge construction is facilitated for students who have misconceptions or gaps in their knowledge without disrupting others (Bunt, Conati, & Muldner, 2004).

The model to facilitate SE consists of two parts: an error hierarchy and tutorial dialogues. The error hierarchy categorizes all error types in the domain of conceptual database design. At the lowest level, an error type is associated with one or more violated constraints, which form leaves of the hierarchy. The error types are then grouped into higher level categories. Self-explanation is facilitated through tutorial dialogues, one of which is developed for each error type. When there

61

are multiple errors in a student solution, the hierarchy is used to select the error most suitable for discussion and the corresponding dialogue is then initiated. In response to the initiated dialogue, learners are able to provide answers by selecting the correct option from a list. Each component is now described in detail.

Error hierarchy: Since a student solution can contain several errors (i.e. several constraints can be violated) in a single submission, the pedagogical module (PM) needs to decide on the pedagogically most suitable error to initiate the self-explanation process. Selecting the most suitable error to discuss is crucial as this helps students to build a comprehensive mental model of the instructional domain. The simplest approach is to use the order in which the constraints are specified in the constraint base. The constraints are ordered according to traditional ER modelling procedure, starting with entities first, then relationships and finally the attributes. This ordering alone is not sufficient to select an error to prompt for self-explanation, as most solutions violate more than one constraint. At the same time, some semantic constraints are not specific enough to guide self-explanation effectively. For instance, the constraint A construct that should be a regular entity has been represented by another type is violated when a regular entity is modelled either as a simple attribute or a weak entity. Different approaches are required in these two cases. Self- explanation in the first case should help the student to clarify the definitions of entities and attributes so that the student understands when to use them. In the latter case, the student should understand the differences between regular and weak entities, which will enable the error to be corrected and the correct design decisions made in future. Also, the pedagogical module should enable students to build a more comprehensive mental model of the domain knowledge by giving them an opportunity to learn basic concepts before complicated ones. For instance, students need to understand the reason for a certain component to be modelled as a regular entity before understanding the relationships that this entity participates in or attributes of that entity. Our approach to select the pedagogically most suitable error for discussion was based on an error hierarchy that categorizes all the errors in the conceptual database design. This hierarchy was used to introduce a high-level structure of constraints to the constraint base of KERMIT. The development of the hierarchy was done in consultation with a domain expert. We also used students’ answers for an assignment in an introductory database course and students’ responses to

62

ALL ERRORS Syntax Errors

Errors dealing with simple connections 1 2 3 4 5 6 101 7 8 9 10 21 Semantic Errors

Using an incorrect construct type Extra constructs

Missing constructs

Connecting an attribute to an incorrect construct Errors dealing with cardinalities and participation

Figure 2.20: Overall view of the error hierarchy

pre- and post-tests of a previous evaluation study conducted for KERMIT (Suraweera & Mitrovic, 2004). The overall view of the error hierarchy is shown in Figure 2.20.

In CBM domain knowledge is represented as a set of constraints on correct solutions. These constraints identify the correct solutions from the space of all possible solutions. Constraints represent only declarative knowledge of the domain. Incorrect solutions can be identified as solutions that contravene the semantic and syntax rules of the domain. In other words an incorrect solution violates one or more constraints specified for a domain. Incorrect domain knowledge is much greater than correct knowledge and violates one or more constraints for that domain. The proposed error hierarchy divides the space of incorrect knowledge into syntax errors and semantic errors. Each one is further divided as presented in Figure 2.20.

Violated constraints (which are specified by the constraint numbers in Figures 2.20 and 2.21) for each type of error are represented as leaves of the hierarchy. Constraints for the nodes in Figure 2.20 are given in separate lines to indicate that each constraint deals with a specific type of error.

Our experience in teaching conceptual database design in a traditional classroom setting at the University of Canterbury indicates that it is easier for students to understand and correct syntax

63

errors than semantic ones. Therefore, the error hierarchy focuses on dealing with the syntax errors before the semantic ones. This is achieved by placing the node Syntax errors, which deals with all the syntax errors before the node Semantic errors, which deals with all the semantic errors (Figure 2.20).

The nodes in this hierarchy are ordered from the basic domain principles to more complicated ones so that a student can be guided systematically towards building and repairing his/her mental model of the domain. Therefore, when the hierarchy is traversed to find the first leaf that corresponds to one of the violated constraints, the simplest error for a given student solution can be found.

Syntax errors: The constraint base of KERMIT contains two types of constraints: syntax and semantic constraints. The syntax constraints focus on the syntactic accuracy of the student solution. On the other hand the semantic constraints deal with how a student solution matches a given scenario by comparing it to an ideal solution that a human teacher deemed as pedagogically the most suitable solution. Syntax constraints are of varying complexity. For instance, constraint 1 is violated when more than one ER construct is used to model a phrase of the problem statement (i.e. when the phrase “CHAPTER” is modelled by both a regular entity and a simple attribute), whereas constraint 45 is violated when a regular entity does not have a primary key attribute. Thus, constraint 1 deals with a relatively simpler error than constraint 45. As a result, it is sufficient to use a simple feedback measage when constraint 1 is violated. However it is better to initiate a longer dicussion for constraint 45 as it is more complex. The syntax constraints for which a simple feedback message is sufficient have been categorised under the node Syntax errors in the error hierarchy (Figure 2.20). The node Semantic Errors categorises both the remaining syntax constraints for which a detailed discussion is needed and the semantic constraints.

Semantic errors: Each of the semantic error types are further divided into sub error types as indicated in Figure 2.20. For instance, the node Using an incorrect construct type is divided into two child nodes: (i) Using a completely different type of construct and (ii) Using a different variation of the correct construct. In ER modelling, two types of design decisions need to be made. The first type of decision requires deciding whether a certain phrase in the problem statement can

64

be modelled as an entity, relationship or an attribute. The second type of decision focuses on deciding whether it is regular or weak in the case of an entity; regular or identifying in that of a relationship etc. The error types associated with the first step are classified under the node Using a completely different type of construct. The node Using a different variation of the correct construct

deals with the errors related to the second step.

Including constraints in the hierarchy: Some leaves of the error hierarchy contain a single constraint for a single node. For example constraint 37 that focuses on binary relationships (it

Using an incorrect construct type

Using a completely different type of construct

Using an entity to represent another type of construct

Using an to represent a relationship

(11 or 200) and (27 or 28)

Using an entity to represent an attribute

(11 or 200) and 203

Using another type of construct to represent an entity

Using a relationship to represent an entity

(13 or 14) and (19 or 211)

Using an attribute to represent an entity

(13 or 14) and 202

Other representations

Using an attribute to represent a relationship

(27 or 28) and 202

Using a relationship to represent an attribute

(19 or 211) and 203

Using a different variation of the correct construct

Entity

Using a regular entity to represent a weak entity

11 and 14

Using a weak entity to represent a regular entity

13 and 200

Relationship

Using a regular relationship to represent an identifying relationship 28 and 19

Using an identifying relationship to represent a regular relationship 27 and 211

Attribute

Using a different type of attribute

Complete attributes

54 or 55 or 56 or 57 or 58 or 59 or 80 or 83 or 84 or 85

Component attributes

41

65

checks whether there is a matching binary relationship in the student solution for every binary relationship in the ideal solution) is categorised under the node Missing constructs (Figure 2.22). The more complex case is having multiple constraints for a single node. This can happen due to different reasons: (i) Multiple constraints are needed to define an error precisely; (ii) multiple constraints specify multiple instances of an error type; and (iii) A single error type is covered in multiple constraints.

To illustrate case (i), consider the regular entity CHAPTER in the student solution in Figure 2.7(a). Modelling CHAPTER as a regular entity violates constraints 11 and 14. Constraint 11 deals with extra regular entities in the student solution which should be modelled using some other type of construct. (i.e. constraint 11 checks for the existence of a matching regular entity in the ideal solution for every regular entity in the student solution). Constraint 14 focuses on weak entities which are represented as some other type of construct in the student solution (i.e. constraint 14

Missing constructs Entity 15 or 16 Relationship Binary 37 N-ary 38 Missing connections 17 22 23 25 26 Missing attributes Complete attributes Keys/Partial keys 44 45 46 Other attributes 204 and (60 or 61 or 62 or 63 or 64 or 67 or 86 or 87 or 88 or 89) Component attributes 40 66 68 71

66

checks for the existence of a matching weak entity in the student solution for every weak entity in the ideal solution). This situation is described by the node Using a regular entity to represent a weak entity which is a child node of Using a different variation of the correct construct (Figure 2.21). The two constraints (i.e. 11 and 14) are combined with ‘and’ to specify that both constraints need to violated by the same construct i.e. CHAPTER is an extra regular entity in the student solution (constraint 11) and should be modelled as a weak entity (constraint 14).

As another example, consider the error using an attribute to represent an entity. A student can make this error in two possible ways: (i) using an attribute to represent a regular entity (ii) using an attribute to represent a weak entity. When a regular entity is represented as an attribute, constraints 13 (focuses on regular entities which are represented as some other type of construct in the student solution) and 202 (focuses on attributes in the student solution that are used to represent another type of construct) will be violated. Similarly when a weak entity is modelled as an attribute, constraints 14 (deals with weak entities which are represented as some other type of construct in the student solution) and 202 will be violated. As a result, all these constraints are grouped together as (13 or 14) and 202 and assigned to the node Using an attribute to represent an entity (Figure 2.21).

In case (ii), multiple constraints that specify multiple instances of the same error type can be grouped together so that a single dialogue customised to reflect the current error in the student solution can be used. For example, constraints 54, 55, 56, 57, 58, 59, 80, 83, 84 and 85 are grouped together to specify the error Using a different type of attribute (Figure 2.21). Constraint 54 deals with simple attributes connected to an entity, modelled as some other type of attribute (i.e. constraint 54 checks for the existence of a matching simple attribute in the student solution for every simple attribute in the ideal solution). Similarly constraint 80 deals with simple attributes belonging to a relationship. Constraint 57 deals with derived attributes connected to an entity whereas constraint 87 deals with derived attributes belonging to a relationship. As all these constraints denote different instances of the same error type using a different type of attribute they are grouped together using ‘or’.

The final scenario involves an error type covered in multiple constraints. Incorrect cardinality and participation are two such errors. Two cardinalities have to be specified for each binary relationship. For example two cardinalities for the binary relationship CONTAINS are: (i) the

67

cardinality between TEXT_BOOK and CONTAINS (ii) the cardinality between CHAPTER and CONTAINS. In addition to having to specify two possible cardinalities for each binary relationship, there are two possible values (1 or N) for each cardinality. Therefore, constraints 92 and 93 deal with cardinality being ‘1’, whereas constraints 94 and 95 deal with the value N. As a result, all these constraints are grouped together using ‘or’ and assigned to the node Cardinality

which is a child node for Errors dealing with cardinalities and participation (Figure 2.23). Similarly there are four constraints that deal with the two possible values for participation (i.e. total and partial) Constraints 96 and 97 focus on total participation whereas constraints 98 and 99 deal with partial participation. All these constraints are grouped together (Figure 2.23).

Tutorial dialogues: Error remediation is facilitated through tutorial dialogues. A dialogue is designed for each error type (i.e., each leaf node in the hierarchy). As the domain model of constraint-based tutors is represented as a set of constraints, violations of constraints indicate the domain concept that the student has difficulty with. Each dialogue discusses the domain concept associated with an error as well as providing an opportunity to reflect on that error within the current problem context.

There are two types of dialogues: (i) Single-level dialogues and (ii) Multi-level dialogues. Single-level dialogues handle errors associated with simple syntax errors for which a detailed feedback message is sufficient to explain the error. Hence these dialogues are limited to a single

Errors dealing with cardinalities and participation Cardinality Regular relationship (92 or 92 or 93 or 94 or 95 Identifying relationship 33 Recursive relationship (102 or 102 or 103 or 104 Participation Regular relationship 96 or 97 or 98 or 99 Identifying relationship 34 Recursive relationship 105 or106 or 107 (34)

68

feedback message. There are 12 such dialogues. An example of a single level dialogue is “You have connected an entity A to entity B directly. Entities cannot be directly connected to each other”. This is associated with constraint 7 (Suraweera, 2001). Multi-level dialogues handle other more complex errors for which a series of prompts is necessary to guide students to self-explain both domain concepts and problem-solving steps. These dialogues consist of four levels:

(i) Informs the student about the incorrect modelling decision and asks to focus on the corresponding domain concept

(ii) Prompts the student to understand why his/her modelling decision is incorrect (iii) Prompts the student to specify the correct modelling decision

(iv) Prompts the student to review the domain concept learnt through a review question There are two exceptions to the 4-level dialogues. The first refers to those dialogues that focus on the concepts of cardinality and participation; they consist of only three levels. The second exception is the different focus of the level 2 prompt when discussing missing constructs. Both these exceptions will be discussed in detail at the end of this section.

Dialogue presented in Figure 2.24 discusses the error that CHAPTER has been modelled as a regular entity. This error was one of many errors identified in the evaluation of the student solution in Figure 2.7(a). This error was selected as the simplest error to discuss as it is the first one in the error hierarchy that corresponds to one of the violated constraints.

We now discuss what happens in each stage of the dialogue. In the first stage of the dialogue, the system informs the student about the error and focus on the corresponding domain concept (KERMIT-SE1 in Figure 2.24). If the student requests the correct explanation or fails to provide the correct one, the student is given more help by explaining the domain concept or by guiding him/her throughsimpler questions (KERMIT-SE2 in Figure 2.24). If he/she still fails to understand the mistake through the simple prompt provided by the tutor, the correct answer is provided as a last resort. Providing the correct explanation at any stage, moves the dialogue to the next stage and the next prompt is displayed.

In the second stage of the dialogue, the student is guided to explain why his/her modelling decision is incorrect (KERMIT-SE4 in Figure 2.24). If it is difficult for the student to understand why his/her decision is incorrect (Student3 in Figure 2.24), the tutor guides the student to search for more information in the problem statement. For instance, when the student fails to understand

69

why modelling CHAPTER as a regular entity is incorrect, he/she is asked to search for any key attributes for the entity type CHAPTER (KERMIT-SE5 in Figure 2.24). As students need to be actively engaged in finding out whether there are any key attributes for CHAPTER, the possible answers are not given for the question immediately. Instead, the list of possible answers is displayed only when the student informs the tutor that he/she is ready to continue with the self- explanation process. By delaying the display of possible answers when students are expected to search for information in the problem statement, they are encouraged to actively engage in understanding why their modelling decision is incorrect. Then he/she is given the opportunity to