Características

The three systems described in this section offer a collection of exercises and provide detailed feedback on form-related errors. They have been used and tested within for- eign language programs in universities.

E-tutor

The E-tutor system (previously known as German tutor) has been developed by Trude Heift and colleagues at the Simon Fraser University in Canada (Heift and Nicholson, 2001; Heift, 2003, 2004, 2010a). It is used by students of German as part of their regular language classes and covers the content of the first three beginner courses. In addition to texts that introduce the topic and grammar structures of each chapter, the core of the system consists of exercise activities. These exercises comprise listening and read- ing as well as writing tasks. For specific grammar-focused tasks the system is able to generate automatic feedback. These exclusively text-based exercises are gap-filling, sentence building, translation, and dictation. The feedback is implemented through a combination of generic non-linguistic matching algorithms and a linguistic analysis using constraint relaxation. The generic error module identifies spelling errors, miss- ing or superfluous words, and incorrect word order by comparison to the set of correct answers. The NLP-based module diagnoses grammatical errors based on the syntactic analysis of the learner answer (using the Head-driven Phrase Structure Grammar (HPSG) formalism (Pollard and Sag, 1994)). It can identify agreement errors, e.g., mismatches between subject and verb or unsatisfied case requirements of verbs and prepositions (more details are given in Heift and Nicholson (2001); Heift (2003)). Feedback mes-

44 CHAPTER 3. DIALOG FOR LANGUAGE LEARNING sages are explicit and provide different amounts of information that are specific to the level of the learner (see for more details Section 5.5.2). The activities in E-tutor that provide automated feedback do not allow free input. The developers argue that their goal is high accuracy of feedback which would be impossible to provide reliably for unconstrained input Heift (2003). The system is evaluated in terms of accuracy of the feedback it provides. In addition, learner errors and learner behavior in response to different types of feedback have been studied extensively, as described in more detail in Section 5.5.2 and Heift (2001b, 2004, 2010b). Heift (2004) refers to SLA feedback studies and the value of interaction and noticing (Section 4.2.3) as SLA principles and motivations for the system.

Robo-Sensei

Robo-Sensei is a system for learning Japanese, developed by Noriko Nagata at the Uni- versity of San Francisco (Nagata, 2002, 2009). It covers grammatical structures that are contained in a standard 2- to 3-year Japanese curriculum. It is intended as a supple- ment to a text book, and its core consists of sentence production exercises. Learners are provided with a communicative context in English and an English paraphrase of what they should produce in Japanese. The system then provides immediate feedback to the learner response. Although the task is embedded in a real-life scenario, the sentence to be produced by the learner is not part of a larger dialog and the learner utterance is very much constrained by the English prompt that is to be translated.

The error diagnosis and feedback is based on a linguistically informed comparison between the correct answer and the learner answer. The linguistic analysis employs word segmentation, morphological and syntactic analysis and errors can be diagnosed at each of these levels. The error diagnosis can identify unknown, missing and unex- pected words, modifier errors, word order errors, and predicate form errors, which include tense, negation, style, and auxiliary form errors. The feedback is explicit and very informative as it indicates not only the location of the error but also provides an explanation of the grammar rules that were violated. Some common spelling and conjugation errors are anticipated and handled in the morphological analyzer. Other errors are recognized by matching the syntactic structure of the correct target response with the syntactic structure of the actual learner response. In this way, errors are diag- nosed through recognizing the difference to the model response, which can be consid- ered as one instance of pattern-matching approaches (see Section 2.3). This means that errors are not anticipated explicitly, but, since the possible mismatches are identified related to very specific phrase structure rules, the feedback messages contain detailed information about the nature of the rule violation. The system, and in particular the learning effect of the feedback it provides, have been thoroughly evaluated (Nagata, 1993, 1997). We will summarize the results of this in more detail in Section 5.5.2.

TAGARELA

The Teaching Aid for Grammatical Awareness, Recognition and Enhancement of Lin- guistic Abilities - TAGARELA(Portuguese for “talkative”) was developed by Luiz Ama- ral, Detmar Meurers, and colleagues at the Ohio State University (Amaral, 2007; Ama-

3.2. STATE OF THE ART IN EXISTING ICALL SYSTEMS 45 ral et al., 2011). It is conceptualized as an “electronic workbook that offers on the spot individualized feedback on spelling, morphological, syntactic and semantic er- rors” for learning Portuguese (Amaral and Meurers, 2011, page 14). The system pro- vides listening and reading comprehension, picture description, rephrasing, fill-in- the-blanks, and vocabulary tasks as exercise activities. The linguistic analysis of the learner input comprises tokenization, spell checking, morphological analysis, lexical lookup and disambiguation for lexical information, bottom-up chart parsing based on a small custom-built grammar, and semantic interpretation based on shallow matching strategies. The feedback given by the system depends on the type of activity, which entails different kinds of learner input. Feedback for reading and listening compre- hension and description tasks is meaning-based, while the rephrasing task provides feedback about syntactic errors. Vocabulary exercises, which expect a noun phrase as response, and gap filling exercises involve feedback about morphological or lexical er- rors. The work on TAGARELA is based on a number of SLA concepts, that we will discuss in the next chapter – task-based instruction andFOCUS-ON-FORM. The evalu-

ation of TAGARELA is limited to small-scale usability studies and the observance of some specific problems for feedback efficiency (Amaral and Meurers, 2009). However, until now, there has been no principal evaluation in terms of learning gains that the system can support.

Summary

We have described E-tutor, Robo-Sensei, and TAGARELA as examples of systems that offer relatively focused and well-defined exercise activities and detailed feedback on form-related errors. This feedback is enabled by a combination of several steps of linguistic processing which at least comprise morphological and syntactical analysis. These systems are relevant for this thesis because they illustrate the state of the art in form-related feedback, and in the scope of this thesis, we will examine the effect of different types of such feedback. Since, for our study, we plan to provide feedback in the context of communicative interaction, we will now describe ICALL systems that focus on communicative activities in a meaning-based context.