DEFINICIÓN EL ENFOQUE TRADICIONAL Y EL ENFOQUE MULTICRITERIO

The Geometry Explanation Tutor (Aleven, Popescu, Ogan & Koedinger, 2003) is the result of adding dialogue capabilities to the Geometry Cognitive Tutor (Anderson et al., 1996), which addressed the current geometry curriculum in high schools in the United States. The focus of the Geometry Explanation Tutor is on the Angles Unit which deals with the properties of angles in various kinds of diagrams. Students are presented with a diagram with a set of known angle measures, and are expected to find some unknown angle measures. Students are expected to explain their steps using geometry definitions and theorems. A previous version of this tutor expects the students to either type the name of the theorem or select it from a glossary of geometry knowledge (Aleven & Koedinger, 2002). The glossary listed the relevant geometry theorems and definitions and provided further information about each rule on demand. In the version with menu options, a typed explanation which seems to express the correct idea is considered correct even though it was not mathematically precise. In the later version, the Geometry Explanation Tutor engages students in natural language dialogue and guides them to explain their steps in their own words and produce mathematically precise explanations.

The Geometry Explanation Tutor uses a knowledge-based Natural Language Understanding (NLU) component and a simple dialogue management algorithm to assess and respond to student explanations. Each student input is assumed to be an attempt at stating an explanation and is processed in three steps.

First, the system parses the student’s explanation, using the LCFLEX left-corner chart parser (Rosé & Lavie, 1999) and creates a semantic representation of the explanation, implemented in the Loom term description logic system (MacGregor, 1991).

Next, the semantic representation is classified based on an “Explanation Hierarchy”, a fine- grained set of approximately 200 explanation categories. The categories represent students’ incomplete or incorrect explanations commonly observed. The set of categories was developed by analysing several corpora of student explanations. Each category is associated with a sequence of messages that will be presented as feedback for an explanation classified under that category.

45

Finally, feedback to be presented is decided based on (a) the set of categories under which the explanation was classified and (b) the set of geometry rules that can be used to justify the current step. The rules applicable for the current step are identified by running the system’s cognitive model for geometry problem solving.

The system produces feedback in the following manner. First, if the explanation categories indicate that the student explanation contains any geometric concepts that are not relevant to the correct geometry rules, the tutor indicates the concept(s) that need not be mentioned. Step 2 of the second dialogue in Figure 2.15 illustrates this scenario. If the explanation is totally irrelevant to any of the correct rules, the student is told that he/she is focusing on the wrong rule.

Second, if the student input is classified as a complete and correct explanation of the correct geometry rule justifying the current problem-solving step, the tutor accepts the explanation by means of implicit feedback. This scenario is illustrated by the last step of both dialogues in Figure 2.15.

Finally, if the explanation is an incomplete statement of a correct geometry rule, the feedback focuses on hinting at or indicating what is missing. This is done by selecting the category (from the set of categories under which the student explanation was classified) closest to the one that represents a complete and a correct statement of one of the correct geometry rules. Then the system presents the first one from the series of feedback messages associated with that category. This case is illustrated in step 2 of the first dialogue in Figure 2.15. If the subsequent attempts do not indicate

1 Student: the bottom angles are the same

2. Tutor: You are getting there, but you need to be more precise. Are base angles always congruent? 3. Student: the bottom angles are congruent

4 Tutor: Base angles in what type of geometric figure are congruent? 5. Student: the bottom angles in an isosceles triangle are congruent

6. Tutor: [accepts as correct and complete explanation by means of implicit feedback] 1. Student: three angles in an isosceles triangle sum to 180 degrees

2. Tutor: Your explanation is close to being perfect. Except for the following: your explanation should not be specifically about isosceles triangles. Better delete that from your explanation. 3. Student: the sum of the three angles in all triangles is 180 degrees

4. Tutor: [accepts as correct and complete explanation by means of implicit feedback]

Figure 2.15:Two dialogues that students had with the Geometry Explanation Tutor, focused on the Isosceles Triangle Theorem and the Triangle Sum theorem (Aleven et al., 2003)

46

an improvement of the explanation (i.e. the explanation is categorised under the same set of categories), the student receives the next feedback message in the sequence that provides more specific feedback. The first dialogue in Figure 2.15 illustrates this scenario. The student changed the explanation from “the bottom angles are the same” (step 1) to “the bottom angles are congruent” (step 3) to “the bottom angles are congruent”. As both these explanations are categorised as having the same meaning, the next more specific feedback message attached to the chosen category (step 4) is presented.

A classroom study was conducted to test the hypothesis that students will gain a deeper understanding when they explain their problem-solving steps in their own words, as compared to explaining using a menu (Aleven, Popescu, et al., 2003) . The study took place within the context of a course based on the Integrated Mathematics Curriculum, which includes concepts both from algebra and geometry. It was conducted during three class periods, all taught by the same teacher. All students were honours students: they were the most gifted and diligent students within the given age group and school.

The students were assigned to two conditions, a “Dialogue” condition and a “Menu” condition, at the beginning of the study. Two entire classes were assigned to one of the conditions. The students in the third class were assigned randomly to one of these conditions.

Prior to the system interactions, the teacher and students covered the textbook chapter on proofs, which involves many of the geometry theorems that are covered in the system’s Angles unit. Then students participated in an in-class, paper-and-pencil pre-test. During the same session, students watched a demonstration of the Geometry Explanation Tutor. All participants interacted with the system for four 40-minute sessions. In the final session, all students took a paper-and- pencil post-test.

The students in the Dialogue condition were asked to explain their reasoning steps in a (restricted kind of) dialogue while interacting with the Geometry Explanation Tutor. The students in the Menu condition explained their steps by specifying the name of a geometry definition or theorem. i.e. they used the previous version of Geometry Explanation Tutor, mentioned above. They could either type the name or select it from an on-line glossary of geometry knowledge, which listed the relevant geometry theorems and definitions. The glossary was available freely to

47

all the students in the study, but it functioned as a menu only for the students in the Menu condition. The two tutor versions were the same in all other respects.

Both pre-test and post-test included regular “Numeric Answer” and “Explanation” questions similar to the problems that students had encountered while interacting with the system. The tests also included two types of transfer items to assess the improvements in students’ understanding. In some of these test items students were expected to determine whether there was enough information to find a particular unknown quantity. Items that involved a quantity whose value could not be uniquely determined are called “Not Enough Info” items. On the other hand, items that had a quantity whose value could be determined with the given information, were grouped with the Numeric Answer items. Some questions termed as “Verbal” items required students to determine the accuracy of a given general statement and to correct it, if applicable.

Of the 71 students, 62 completed the pre-test and post-test. The analysis focused on 46 students who worked on the tutor for at least 80 minutes, excluding the idle time. Even though the 80 minutes time threshold may seem somewhat arbitrary, Aleven and colleagues (2004) noted that results were not sensitive to the threshold. Dialogue condition included 21 students and Menu condition, 25 students. The results revealed that the participants who explained problem-solving steps by engaging in a dialogue with the system did not learn better overall than their peers who explained steps using a menu. This might be due the high pre-test scores. However the participants perform significantly better on the explanation questions. There was no significant difference in the improvement between the groups for other types of questions.

One of the limitations of these dialogues is the expectation to modify the complete explanation even when just a single word is missing. For instance, when the student justifies a step by saying “The angles in an isosceles triangle are equal” and the tutor responds with “Are all angles in an isosceles triangle equal?”. It is not possible to say “No, it’s just the base angles”. Instead, the student is expected to modify the complete explanation to say “The base angles in an isosceles triangle are equal.” This is due to system's inability to break down the knowledge construction process through new non-rhetorical questions and multi-step plans.

48