The Model Tracing approach to ITS is based on the ACT-R theory of An- derson [Anderson, 1993]. The knowledge is constructed using a rule based production system.
There have been a variety of Model Tracing Tutors built for the various domains of instruction, e.g. LISP programing [Anderson and Reiser, 1985], Geometry [Anderson et al., 1985], Algebra [Koedinger and Anderson, 1997]. We give a brief overview of the most important systems below.
2.3.3.1 The PUMP Algebra Tutor (PAT)
The PUMP Algebra Tutor (or PAT) is a Model Tracing Tutor for teaching introductory algebra [Koedinger and Anderson, 1997]. Each problem in this system involves some algebraic problem with several quantities. Using the complex multi-part graphical interface, students have to represent quantities as spreadsheet columns, answer the questions in the spreadsheet rows, con- struct algebraic representation of the relationships between these quantities and graph these relationships.
PAT is a classical Model Tracing tutor, which solves each problem step- by-step with the student and provides assistance if necessary. It tries to trace the student’s solution path and checks it against many possible solution paths in its domain model. Tracing the student step-by-step gives an opportunity to provide individualized instruction, concentrating on the current solution state. The system gives flag feedback on errors, and in case a typical error is diagnosed by matching the student’s answer against a buggy rule, the system presents feedback that indicates what is wrong with the answer.
In addition to the error feedback, PAT also provides help on request. The system gives three types of advices, such as a reminder of the current goal, a general description of how to achieve the goal (conceptual hint), and finally a description of exactly what problem solving action to take (procedural hint). For each of these three types of feedback, several levels of detail are possible. The PAT tutor uses two student modeling techniques: Model Tracing and Knowledge Tracing. Model Tracing is used to monitor the student’s progress within one exercise and Knowledge Tracing is used to monitor student’s
2.3. INTELLIGENT TUTORING SYSTEMS 23 growing knowledge from problem to problem. As the student solves problems, the PAT tutor estimates the probability that each domain concept is in the learned state, based on the students actions. This process involves a simple Bayesian decision process described in [Corbett et al., 1997].
The PAT tutor implements a mastery learning strategy for the outer loop. The students continue solving problems until the probability that each involved domain concept is at the learned state has almost reached certainty. 2.3.3.2 Andes Physics Tutor
Andes is an intelligent problem solving environment for quantitative prob- lem solving in introductory college physics [Van Lehn et al., 2005]. It was designed for the purpose of an intelligent homework environment. It does not have an automated outer loop, but suggests a hierarchical menu from which the student can select problems manually.
There are two major versions of the Andes system: Andes1 and An- des2. Both systems share the same domain model and the user interface.
The domain model of Andes (1,2) consists of a set of production rules that are used to automatically generate solution spaces of problems in the domain of Newton mechanics. Apart from rules, for each of the major me- chanics principles Andes has developed a problem solving method, that incorporates a multi-step hierarchical plan for generating an application of the principle. Solving a problem is realized as a form of state-space search, that iteratively chooses the problem solving methods to be applied. For more details on problem solving in Andes, see [Van Lehn et al., 2005] (pages 30-31).
Andes also possesses a library so-called called error-handlers which can match typical errors of the student and then help to generate error-related feedback.
Andes (1,2) possesses a complex user interface, in which the students can read the problem statement, draw coordinate axes and vectors representing Newtonian forces, define variables and input equations. The system has a mathematics package, that solves equations for students to free them from mathematical steps and let them concentrate on aspects of physics.
Andes (1,2) implements an elaborate but fixed tutorial strategy for feed- back. The system gives unsolicited feedback on slip errors and several types of feedback on student’s request: what’s wrong help pointing to the error and providing error specific feedback. It also generates a sequence of hints on
24 CHAPTER 2. BACKGROUND AND RELATED WORK how to proceed (next step help) with different levels of detail. Typically the next step help consists of a teaching hint containing information about the concepts involved, and the bottom out hint giving out (parts of) the correct solution. The module for Next Step Hint generation is called Procedural Helper.
Another pedagogical decision in Andes is to relieve students from al- gebraic manipulation tasks by providing them with algebraic manipulation tools such as the equation solver. This way the students can concentrate on actual physics principles rather then on algebraic calculations.
The main difference between Andes1 and Andes2 is the content of feed- back produced by the Procedural Helper. The Andes1 has a plan recognition module, based on the Bayesian network student model. The system attempts to recognize the plan of the student and gives the next step hint following this plan [Gertner et al., 1998].
Empirical studies have shown that although Andes1 was successful and increased learning outcomes by 1 standard deviation in comparison with traditional instruction, the Bayesian student model was not the key to its success. It turned out that the elaborate tutorial strategy was key to its success. Also, since the Andes system does not have an automated outer loop and the next exercises to tackle are manually assigned to students, the Bayesian student model was not used for selecting the next tasks.
Inconsistencies in the plan recognition results suggested that in most cases of erroneous reasoning the learner does not have any expert solution plan. So the Procedural Helper in the Andes2 is giving hints directed to return the learner back to the fixed standard expert solution path, rather than trying to recognize the student’s path.
Another method used in Andes2 was the Conceptual Helper module, which was called when Andes2 decided that the student is unfamiliar with a specific principle of physics or has a misconception. In this case the Con- ceptual Helper presented so-called ”mini-lessons” to the learner, which are adapted to the student’s problem solving context.
Years of research, continuous evaluation and deployment allowed Andes researchers to build a powerful learning environment for introductory physics and make significant contributions to the methodology and architectures of Intelligent Tutoring Systems. It is now used widely in the US and elsewhere in more then 200 schools.
2.3. INTELLIGENT TUTORING SYSTEMS 25 2.3.3.3 Ms. Lindquist
The new generation of model tracing tutors with an advanced tutorial model was announced by Ms. Lindquist - a system using multiple tutorial strategies for teaching ”symbolization” in an algebra domain [Heffernan et al., 2008]. The kind of problems considered contain a modeling step in which the stu- dents have to formalize some aspect of the real world. This introduces the need for a more strategic approach to student feedback then in simple incre- mental problem solving.
Ms. Lindquist implements an elaborate tutorial dialogue architecture, called ATM (Adding Tutorial Model), built on top of a model tracing tu- tor. In addition to the traditional outer and inner loops, this system defines a so-called knowledge-search loop. This means, that when the student has reached an impasse in his reasoning, the system starts asking him ques- tions, the answers of which are not always parts of the problem solution, but elicit the knowledge construction of the student that helps him to solve the problem. Such scaffolding also appears in Atlas-Andes within the modified Conceptual Helper in which the static ”mini-stories” of Andes2 are replaced by interactive knowledge-eliciting dialogues, realized as multiple-choice ques- tions.
In comparison to classical model tracing tutors, that generate feedback from text templates coupled with production rules, the ATM architecture generates a dialogue plan for each error student makes. The tutorial model of Ms.Lindquist contains seventy seven rules to generate these plans.
After the student model has made a diagnosis of the student’s action, there are three types of responses: give a bug message, give a hint or use the tutorial sub-strategy addressing the error. There are two kinds of strategies involved - so-called Knowledge Remediation Dialogue and Knowledge Con- struction Dialogue that invoke multi-step plans to deal with particular errors. Evaluations of Ms.Lindquist have shown the effectiveness of multiple tu- torial strategies applied to algebra ”symbolization” problems.