Una industria al servicio de la revolución

of the learning gains it enables. Such an in-depth evaluation requires a considerable amount of human resources, in shape of learners willing to participate. Limited ac- cess to participants then prevents us from attempting a broader exploration covering a more extensive set of instances from the parameter space. Instead, we constrain our- selves to only a few instances, but compare those in more depth regarding their effect for language acquisition, in an experiment following the rigor of SLA studies.

Contributions

Based on our approach we hope to be able to make the following contributions. We provide an example of how to integrate the goals of SLA research and ICALL develop- ment in one unified approach to examine and compare the effect of instructional pa- rameter on language learning and put them in relation to the implementational efforts required to realize them. Through that we evaluate the benefit of using NLP-informed ICALL. We further show how human-centered ways of instruction can be transfered and implemented in a human-computer setting and examine if the effects of instruc- tions in both contexts are comparable. The experimental results allow us to make a concrete contribution to the existing research in SLA. The contribution to ICALL lies in the development of an application that supports both learning and learning about learning. Related to that, the contribution to the field of NLP/CL lies in investigating how different levels of effort and sophistication in NLP afford different instructional parameters of ICALL that in turn lead to different experiences and effects in language learning.

1.2

Outline of the thesis

We describe the structure of the thesis chapter by chapter below.

Chapter 2: Computer-Assisted Language Learning

This chapter provides the first part of the technological background to this thesis. It starts by discussing the motivation for the use of NLP methods for developing ICALL applications and describing the pertinent challenges. It then focuses on one of the chal- lenges by presenting an overview of the different approaches for error diagnosis. The chapter finishes with a review of relevant research in the field of computer-mediated communication between humans. Even though this is an area that does not depend on NLP, it provides insights to the non face-to-face mode of synchronous written inter- action that resembles the communicative setting between humans and computers that we will explore in the present study.

Chapter 3: Dialog for Language Learning

This chapter provides the second part of the technological background by focusing on the computational treatment of human-computer dialog. In the first part, we will discuss every aspect of dialog modeling, comprising a characterization of the features

6 CHAPTER 1. INTRODUCTION and structures of natural dialog, a description of the general architecture of dialog systems and the role of the components, and finally a review of the approaches to dialog modeling and management. The second part presents the current state-of-the- art by providing a comprehensive survey of existing interactive ICALL systems.

Chapter 4: Second Language Acquisition

This chapter presents the pedagogical background of this thesis by discussing the rel- evant concepts, theories and empirical findings related to SLA/FLL. We discuss the different approaches to language instruction that differ in respect to (a) how much em- phasis they put on meaning versus form and (b) how explicit or implicit they are. The chapter further discusses properties of linguistic structures that influence how easily they can be learned. The chapter finishes with a presentation of conversational inter- action and task-based instruction.

Chapter 5: Feedback

This chapter completes the background discussion of this thesis by reviewing the rel- evance of feedback. The chapter integrates the two angles SLA and ICALL by clas- sifying feedback that is provided in SLA contexts and relating it to the technological conditions to provide such feedback through an ICALL application. The chapter fin- ishes with an in-depth inspection of recasts and metalinguistic feedback and the ex- isting empirical evidence for their effectiveness, since these are the feedback types we further examine in this study.

Chapter 6: The Approach

Based on the background of the theoretical and empirical material expounded in the preceding chapters, this chapter discusses the approach we used for exploring how language technology can support language learning. It identifies important parame- ters from both the pedagogical and implementational perspective that serve as a means to constrain the general goal of this work and render it into a more focused study with concrete experimental research questions that seek to compare the effect of different instructional parameters. The chapter then introduces the research design we adopt and justifies the choices we make.

Chapter 7: The Experiment

This chapter describes the details of the experimental setting that we employ to an- swer the concrete questions and compare the parameters. This comprises the selection of linguistic target structures and the specification of the tasks and interaction that con- stitutes the experimental instruction. Furthermore it also includes a discussion of the measures that we use to assess the development of linguistic knowledge and skills. Finally, we describe the procedures and conditions of the practical implementation of the experiment.

1.2. OUTLINE OF THE THESIS 7

Chapter 8: The System

This chapter describes further details about the design and implementation of the ICALL dialog system that we use for providing the experimental instruction. Fur- thermore, it provides a detailed analysis of the performance of the system during the experiment. The chapter concludes by presenting the results of the learner survey regarding their perception and enjoyment of the system.

Chapter 9: Findings

This chapter presents the results of the experiment in detail. First, this includes the development of grammatical accuracy for the target structures in terms of test scores along the four test times during the experiment. Second, we describe the development of the communicative spoken language skills as measured by holistic ratings and tem- poral analysis of fluency in speech samples.

Chapter 10: Discussion

This chapter discusses our findings in the light of the concrete and general questions we sought to answer with the experiment. We will also discuss the limitations and suggestions to address them in future work.

Chapter 11: Concluding Remarks

The final chapter concludes this dissertation by summarizing the contributions and giving a brief outlook on possible continuations and extensions of the presented work.

2

Computer-Assisted Language Learning

2.1

Introduction

This chapter discusses computer-assisted language learning (CALL) and the role of natural language processing (NLP) for CALL. In general, CALL refers to technology and software applications that support people in learning foreign languages. CALL applications can be used as supplement to traditional teacher-dependent language in- struction or as a substitute, in case teachers are unavailable or unaffordable (Nerbonne, 2003). Both as substitute and as additional resource, one key motivation for CALL is that it fosters the autonomy of learners (Benson, 2001).

The use of computers for language instruction dates back to the 1960s, even before the advent of personal computers. The PLATO system (Programmed Logic/Learning for Automated Teaching Operations) (Curtin et al., 1972), which provided grammar drills on a mainframe computer, is often cited as one of the first CALL efforts (Levy, 1997). Since the days of these first approaches, many other systems and tools have been developed, and today CALL is a broad discipline which covers a variety of activ- ities and applications.

These applications can be classified according to the language areas and skills that they target. Levy (2009), for instance, in his review of CALL technology, distinguishes the following target areas: grammar, vocabulary, reading, writing, pronunciation, lis- tening, speaking, and culture. A different classification has been proposed by Zhao (2003), who distinguishes three functions of CALL applications:

1. Providing access to linguistic and cultural material 2. Providing opportunities for communication 3. Providing feedback on learner responses

Access to linguistic and cultural materials refers to the context-dependent provision of ad-

ditional lexical, morphosyntactic and cultural information, which supports the learner 9

10 CHAPTER 2. COMPUTER-ASSISTED LANGUAGE LEARNING in understanding authentic material that originated from native contexts but was not specifically targeted at learners (see, for instance, Lyman-Hager (2000) and Nerbonne and Dokter (1999)). Other examples of features that enhance comprehensibility are captions for videos or the option to slow down the speech rate of audio material (Shea, 2000; Zhao, 1997). Authentic material can also be automatically enhanced to empha- size linguistic forms and make learners more aware of them (Meurers et al., 2010).

With regard to opportunities for communication, Zhao (2003) distinguishes two areas. One is concerned with technology that enables learners to communicate remotely with other learners or native speakers – this field is known as computer-mediated commu- nication (CMC). The other area refers to technology that allows learners to conduct near natural conversations with a computer program, we know these as dialog sys- tems and conversational agents.

Finally, the provision of feedback on learner utterances comprises corrections on er- rors in pronunciation, orthography, morphology, syntax, semantics, and even prag- matics. It also includes the development of learner models based on a record of previ- ous errors.

In this and the next chapter we will see examples of CALL applications that serve one or more of these three functions. Many of these CALL applications require rather sophisticated, i.e., intelligent techniques. Indeed, with the beginning of the 1990s, when artificial intelligence (AI) technologies had reached a sufficient state of maturity, they brought forth a subdiscipline of CALL - Intelligent CALL (ICALL). In a general sense, intelligent CALL comprises the use of techniques such as knowledge representation, expert systems, intelligent tutoring systems (ITS), user modeling, natural language processing (NLP), automatic speech recognition and speech synthesis, and machine translation (for reviews see Gamper and Knapp (2002); Levy (2009); Schulze (2008)). Most often, however, ICALL is used in a narrower sense in which intelligent refers particularly to the automated analysis and generation of natural language. To eschew this ambiguity, some prefer to call it parser-based CALL, referring to the process of parsing, which describes the syntactic analysis of natural language (Schulze, 2008). We will use the term ICALL to refer to the NLP-supported CALL in this thesis.

This chapter has three parts which present different aspects of CALL and ICALL. In Section 2.2, we will discuss the reasons to use NLP techniques in ICALL, describe the challenges and introduce a general strategy to deal with the challenges. As we will see, one of the challenges is the frequent occurrence of errors in learner language which an ICALL application has to account for in some way. In Section 2.3 then we will review the range of approaches to error diagnosis which is a prerequisite to provide feedback. In the last section of this chapter (2.4) we will take a step back to non- intelligent CALL by giving an account of computer-mediated communication between humans. This area does not rely on using NLP methods but it is relevant for this thesis since the remote, non face-to-face mode of interaction resembles the communicative setting between humans and computers that we will employ for the current study.