Robots are being developed in research laboratories all around the globe and are expected to be part of our daily lives in near future. Particularly japanese institutes and companies have a strong tradition in robotic research, popular examples are the humanoid robots QRIO by Sony or Honda’s Asimo. While much research effort is invested in the improvement of fusion and control of electronics and mechanics and cognitive abilities to enable robots
3.3. ANTHROPOMORPHIC AND ANIMALISTIC INTERFACES 79
to perform complex tasks, some projects also consider the social component and examine affective impact of human robot interaction. Particularly aspects such as expressing emotion and modelling convincing behaviour are relevant for this work as well, therefore we selected two sophisticated examples of social robots, which we will introduce in this section.
3.3.4.1 The Paro Seal Robot
Paro is a pet-like seal robot (see Figure 3.37), which has been developed by the Intelligent Systems Research Institute of the National Institute of Advanced Industrial Science and Tech- nology in Japan since 1993, and later in 2004 commercialised in collaboration with Intelligent System Co. Ltd. The robot is designed for the elderly to substitute or complement Animal Assistive Therapy and to study the effects of robot therapy (e.g. [Wada and Shibata, 2007] or [Inoue et al., 2008]). Animal therapy is expected to have three effects: Psychological ef- fects (e.g. relaxation, motivation), physiological effects (e.g. improvement of vital signs) and social effects (e.g. stimulation of communication among patients and caregivers). The motivation raises from the difficulties of implementing Animal Assistive Therapy. Most hos- pitals and nursing homes refuse animals due to potential allergies and infections that could be caused by bites or scratches. In some places, it is not allowed for people to take care of animals in an apartment at all. Apart from that, some elderly who live alone would have difficulties in taking appropriate care of a pet by themselves.
Figure 3.37: The Paro seal robot (source: http://www.parorobots.com)
Paro is equipped with four main types of sensors:
1. Light: To distinguish bright from dark environments
2. Audio: For rudimental speech recognition and to determine sound source direction 3. Orientation: To determine whether it is being held by somebody
Actuators include motors that facilitate vertical and horizontal neck movements, front and rear paddle movements, and independent movement of each eyelid. The robot is also able to render sounds that imitate a real baby harp seal. Several behaviour patterns are implemented to generate a convincing animalistic interaction. Since Paro is able to detect greetings or praise, it can learn to behave in a way that the user prefers. For example, if the robot is caressed after performing a certain action, Paro will remember the previous action and try to repeat it later. Hitting Paro will have the opposite effect and cause it to avoid the previous action. Furthermore, basic emotions such as surprise or happiness are expressed through blinking eyes, or movements of the head and legs.
The behaviour generation system of the robot consists of two hierarchical process layers, which generate three types of behaviour: proactive, reactive, and physiological [Wada and Shibata, 2007] (see Figure 3.38).
Figure 3.38: The behaviour generation system of the Paro seal robot (source: [Wada and Shibata, 2007])
Proactive Behaviour: Proactive behaviour is based on internal stimulation states, de- sires, and internal rhythm and is generated by two layers: a behaviour planning layer and a behaviour generation layer. The behaviour planning layer mainly consists of a state transition network and internal states that reflect basic emotions. Each state has a level, which changes according to sensorial stimulation and decays over time. Certain interactions with the robot change internal transition network, therefore defining Paro’s character. The behaviour plan- ning layer sends basic behavioural patterns to the behaviour generation layer. These patterns include several pre-defined poses and movements. The behaviour generation layer generates low-level control statements that enable the actuators to perform the determined behaviour. The parameters of a control statement depend on the levels of internal states and their com-
3.3. ANTHROPOMORPHIC AND ANIMALISTIC INTERFACES 81
binations. For instance, various parameters can change the speed of actuator movements and create many variations of the same behaviour. Therefore these theoretically almost infinite variations of a finite set of basic patterns are intended to convey a lifelike impression of the robot seal. These parameters are further adjusted according to the priorities of reactive and proactive behaviours based on the levels of the internal states, which makes its behaviour again more unpredictable.
The Paro system further applies Reinforcement Learning to implement a long-term mem- ory: It places positive value on preferred external stimulation, such as stroking and negative value on undesired stimulation, basically beating. Thus, the system can be gradually tuned to the preferred behaviour of its owner. In addition to that, the robot is able to memorise a frequently articulated word as its new name.
Reactive Behaviour: The Paro robot reacts to external stimulation, such as turning to the direction of sudden and loud sound sources. There are several patterns of combining external stimulation with a predefined reaction, which are assumed to be preconditioned and unconscious.
Physiological Behaviour: The system follows a day-night rhythm, which enables the robot to follow spontaneous needs, such as sleep.
To investigate the psychological and social effects of the robots, several user studies in different scales were conducted. A more recent study [Wada and Shibata, 2007] reported on results from tests with elderly residents in a care house, when two robots were introduced into the facility, and activated for over 9 hours each day to interact with the residents. All subjects were interviewed, and their social network was analysed. In addition, the activities of the residents in public areas were recorded by video cameras. For physiological analysis, urine of the residents was obtained and analysed for two hormones6This study was the first
attempt to investigate the impact of robots such as Paro in situations where subjects can interact freely with the robot. However, the results were obtained from limited number of subjects (12) and a control group did not exist. Summarising the predominantly positive results, it could be shown that the average amount of time spent in a public space of the care house increased from about 1.4 hours to over 2.5 hours. In particular, 1.5 hours of this time included interaction with the Paro robot. Typical situations arose when one person was seen with Paro by another resident, who then stopped to communicate. This time decreased after two weeks, when people were accustomed to the new toy, but still remained on a higher level than before. Further, the values of the hormones significantly improved during the test phase. In particular, significant improvements in the ratio were shown, which was considered as a positive physiological reaction of the residents’ organs.
6
1) 17-ketosteroid sulfate (17-KS-S), which has high levels in healthy individuals and decreases with failing health or the progress of disease. Also, the 17-KS-S value exhibits sensitivity to changes in psychological and social factors and correlates strongly with a person’s will, desire, and energy. 2) 17-hydroxycorticosteroids (17- OHCS), which has high levels in individuals under stress [Selye, 1970].
3.3.4.2 Leonardo
Leonardo is a humanoid, immovable robot with a flexible torso offering 65 degree of free- dom. It’s outer appearance resembles more an animal like creature than a human and the fact that it is not able to speak [Breazeal et al., 2004] further shifts the perception of this robot toward a more animal-like interaction partner. Nevertheless, it has been designed for social interaction utilising various gestures and facial expressions, particularly for learning/teaching situations in collaborative processes.
Figure 3.39: Left: Leonardo offers to perform an action. Right: Leonardo asks for help. (source: [Breazeal et al., 2004])
The overall goal of this work was to improve the intuitiveness, efficiency and enjoyment of human-robot interaction in working and teaching scenarios, by modelling these properties as essentially collaborative processes requiring natural human social skills and conventions. The chosen approach is two-fold, it includes the ability to teach a task to Leonardo through the course of a collaborative dialog with gesture and facial expressions, and the ability to coordinate common intentions to perform a learned task collaboratively.
Cooperative behaviour in this context is considered an ongoing process of maintaining
mutual beliefs, sharing relevant knowledge, coordinating action and demonstrating commit- ment to the shared activity. In order to support this, Leonardo employs a variety of gestures
and other social cues to continuously communicate its internal state to the human, such as the robot’s estimation on who is supposed to do an action or whether a goal has been reached. For example, when then human collaborator has changed a state of the world, the robot ac- knowledges this action by briefly glancing towards the area of change and then looking at the human partner. This behaviour reassures the user of the robot’s awareness of what she or he has done. If this action at the same time fulfils a goal, Leonardo adds a quick confirming nod while redirecting its gaze towards the human. Likewise, nods are used while looking at the human partner to indicate that Leonardo believes that it brought about the completion of
3.3. ANTHROPOMORPHIC AND ANIMALISTIC INTERFACES 83
a task. This kind of social and reassuring communication is particularly beneficial when the team does not work linearly on a joint plan but instead when each partner works in parallel on different parts of the task, or when unexpected actions of the human occur. Furthermore, the robot has the ability to assess his own capabilities and react accordingly. For instance, if Leonardo would be able to complete a step of the task he will offer to do so, but allow the human partner to override this verbally or by completing it by her- or himself. Accordingly, when Leonardo is not able to carry out an action, it would ask for help. The robot indicates intents to perform an action by pointing to himself and taking up the corresponding pose (see Figure 3.39, left). Similarly, Leonardo expresses the inability to fulfil an assigned task by gesturing towards the human in a help-seeking pose (see Figure 3.39, right). In addition to these gestures, Leonardo shifts its gaze between the human and the object in question to direct the human’s attention unambiguously.
The software architecture consists of several modules, comprising speech understanding, vision and attention, cognition and behaviour and motor control. The vision module parses objects from the visual scene, including humans and objects that can be acted on, e.g. but- tons. Objects attributes such as colour and location are associated with each perception and forwarded to the cognitive system. Pointing gestures of the human are also recognised and linked to their object referent by spatial reasoning. The continuous stream of perceptions from the vision and speech understanding modules flows into the cognition system where it is integrated into coherent beliefs about objects in the world and their features, e.g. location, colour, ON/OFF state. On top of these processing modules, a set of higher-level capabilities are employed, including goal-based decision making, task learning and task collaboration, which can be reviewed in more detail in [Breazeal et al., 2004].
Leonardo realises several nonverbal behavioural features that are known from graphically embodied conversational agents, but also integrates concepts that are particularly relevant in a physical and spatial context. For example, the posture of its head indicates his field of vision and therefore defines the range of its visual perception. Also, spatially referencing physical objects and the user by gaze and indicating that certain objects are out of reach, are specifically relevant for robots and other entities in the physical realm.