ANÁLISIS DEL AGUA DE RIEGO

Earlier in this chapter we introduced ECAs that can be implemented in the pedagogical role of a tutor, expressing cognitive and empathic feedback as successful strategies for encouraging learning in web-based tutors. We identified techniques for modelling empathic behaviour in Section 2.3 and approaches to affective tutorial feedback in Section 2.4. In this section we look at the specific methods of generating ECA empathic behaviour that is recognisable to humans. We begin with an overview of multimodal output in Section 2.5.1, followed by a more detailed analysis of methods related to generating speech, facial expression and gesture in Sections 2.5.2 to 2.5.4.

50

2.5.1 Multimodal output

ECAs have been developed to imitate human-to-human conversational properties. Cassell et al. (2000, p.52) describes these as follows:

 “The ability to recognize and respond to verbal and non-verbal input.  The ability to generate verbal and non-verbal output.

 The ability to deal with conversational functions such as turn taking, feedback, and repair mechanisms.

 The ability to give signals that indicate the state of the conversation, as well as to contribute new propositions to the discourse.”

Therefore ECAs could imitate human-to-human conversational properties through the use of more than one communication channel (or multimodal communication) that may include one or more of the following: speech, facial expression, gesture and posture. This is demonstrated by REA an ECA developed with multimodal input and output capabilities in the domain of Real Estate (Cassell et al., 2000) and Greta an ECA developed using communicative acts (Pelachaud 2005). One constraint to consider is that human users can synchronize their non-verbal behaviours to improve conversation smoothness; however an ECA may not have the same ability to improve synchrony and adapt its behaviour to the other party as the conversation continues.

This study uses multimodal output to develop a believable empathic ECA that is able to imitate human conversation through the use of speech, facial expression, gesture and

51

posture. However, the ECA in the current study accepts self-reported emotion in addition to task related information such as the selected answer as input.

2.5.2 Speech

Speech is an important part of human-to-human communication, implementing appropriate speech using ECAs is not a trivial task. A key consideration is whether to use synthesised (machine-generated) speech in comparison to pre-recorded speech in tutoring systems. A number of studies suggest that pre-recorded speech using a human voice may improve student engagement (Baylor et al., 2003), student learning and usability (Atkinson et al., 2005). Another study suggests voice characteristics such as “voice pleasantness” and “listening effort” required is more important than the type of speech (Moller et al., 2006). Pre-recorded speech is less flexible as the speech cannot be adjusted easily whilst synthetic speech can introduce usability issues such as timing although advances in this technology have improved this. Forbes-Riley et al. (2006) suggests little difference in learning, as learners can read the tutor transcript and are not entirely dependent on speech, in their spoken dialogue tutoring system.

Investigations carried out during the study presented in detail in Chapter 4 emphasise the use of redundant channels of communication to ensure that communication goals are met during every interaction. In addition, although pre-recorded speech is less flexible, it is used in the current study to encourage user engagement during interaction with the ETS; based on a pilot study (see Appendix 1) we conducted where

52

learners gave negative reactions to synthetic speech, as described in Section 3.3 and Section 3.4.

2.5.3 Facial expression

Facial expression is related to what is occurring in concurrent conversation (Cassell et al. 1994). The same authors describe facial expression as being related to speech in conversation in the following three ways:

 Syntactic functions - describe facial movements such as raising eyebrows that are synchronized with specific words or accented syllables.

 Semantic functions - emphasize speech by referring to a word or emotion.  Dialogic functions - control speech flow between two people.

Cassel et al. (1994) identify a number of parameters that impact on the functions above including speaker and listener characteristics such as social identity and the listener’s reaction to the speaker’s utterance.

Pelachaud (2005, p.686) describes the automatic generation of facial expressions for GRETA an embodied agent using temporal parameters. These parameters define attributes such as “sustain” which “is the time during which the expression maintains maximal intensity”. A number of studies build on the work of Eckman and Friesen (1977) on facial expression in realizing affective states in learning environments which is beyond the scope of this thesis; see Poggi and Pelachaud (2000).

53

2.5.4 Gesture and posture

Non-verbal behaviour such as gesture provides various functionalities in communication including signalling affirmation or rejection (McClave 2000; Kendon 2002). Gesture or gaze can be incorporated into ECA speech using nonverbal behaviour generation rules that are described by Lee and Marsella (2006). These rules are based on analysing surface text, after which relevant non-verbal behaviours are added to the interaction. Each nonverbal behaviour rule has a priority and a set of associated key words that occur in close proximity to the rule. The challenge during implementation of any generated behaviour, for example a pointing gesture, is maintaining the synchrony of the communication act with the speech and facial expression to ensure that the meaning is clearly conveyed.

Further non-verbal communication such as gaze and posture are beyond the scope of this study, although the current study uses automatically generated off-task non- verbal behaviour to imitate human behaviour such as breathing, blinking and direction of gaze. Furthermore, the current study uses automatically generated posture for specific affective states such as concern where the ECA leans forward; see details of implementation in Chapter 3 (Section 3.4.1).