This section presents projects involving the use of robots to display emotional facial expressions. FACE and Probo were the robots from this list already employed in studies with children with ASD.
The humanoid robot FACE (Mazzei et al., 2011) was built to allow children with ASD to deal with expressive and emotional information. The expressions and movements of FACE were modelled to be harmonized with the feelings of the user. HEFES (Hy- brid Engine for Facial Expressions Synthesis) is a system created by the same authors to generate and control facial expressions both on physical androids and 3D avatars (Mazzei et al., 2012). The system used in FACE was tested on a panel of 5 children with ASD and 15 typically developing children interacting with the robot individually under therapist supervision. The evaluated facial expressions were happiness, anger, sadness, disgust, fear, and surprise, defined as the basic emotions by Ekman (Ekman & Rosenberg, 1998). These emotions are going to be referred from now on as basic emotions or basic facial expressions. The participants labelled each expression and this labelling was scored by the therapist as correct or incorrect. Their results showed that both children with ASD and typically developing children were able to label happiness, anger and sadness performed by FACE with good accuracy. However fear, disgust, and surprise had not been labelled correctly, especially by participants with ASD. The results for FACE’s recognition rates with children with ASD were the following: anger - 100%, disgust - 20%, fear - 0%, happiness - 100%, sadness - 100%, surprise - 40%, and the average of all emotions was 60%. The results for FACE’s recognition rates
Chapter 2. Literature Review 45 with typically developing children were anger: 93.3%, disgust: 20%, fear: 46.7%, hap- piness: 93.3%, sadness 86.7%, surprise: 40%, and the average of all emotions was 61.1%. The authors justify these results claiming that fear, disgust, and surprise are emotions which rely greatly on gestures to convey its expression, and facial expressions on their own were not enough for an efficient recognition.
Probo (Saldien et al., 2010) is an animal-like robot, designed to act as a social in- terface. The authors used Probo as a platform to study human-robot interaction and it was capable of performing facial expressions. These were represented as a vector in the two-dimensional emotional space, valence and arousal, based on the Russell’s circumplex model of affect (Russell, 1980). The recognition of the robot’s facial ex- pressions were evaluated by 23 typically developing children, giving an identification rate of 96% for anger, 87% for disgust, 65% for fear, 100% for happiness, 87% for sadness, 70% for surprise, and the average of all emotions of 84%. In their opinion, a better recognition of the robot’s facial expressions contributes to the general social acceptance. In addition, the recognition of the facial expressions is important for an effective non-verbal communication between a human and a robot.
Kismet (Breazeal, 2000) was designed with the possibility to process a variety of social cues from visual and auditory channels, and delivered social signals to humans with whom it interacted. Kismet’s facial expressions were generated using an interpolation- based technique over a three-dimensional, multicomponent affect space: arousal, va- lence, and stance (Breazeal, 2004). In this model, valence and arousal were used to construct an emotional space, based as well on the circumplex model of affect defined by Russell (Russell, 1980), which has as well been implemented in the robot EDDIE (Sosnowski et al., 2006). EDDIE similarly to Kismet is a robotic head, and they were evaluated by 8 typically developing children between the ages of 5 to 8 and 16 adults between the ages of 25 to 48. The study consisted of a total of 32 questions. Participants had to choose their best guess for a displayed emotion. The results for Kismet’s recognition rates were: anger: 76%, disgust: 71%, fear: 47%, happiness: 82%, sadness 82%, surprise: 82%, and the average of all emotions was 73%. For EDDIE, the recognition rates were: anger: 54%, disgust: 58%, fear: 42%, happiness: 58%, sadness 58%, surprise: 75%, and the average of all emotions was 57%.
The humanoid robot WE-4RII was designed to communicate naturally with a human partner by expressing human-like emotions (Itoh et al., 2004). The authors measured the recognition rate of the emotional expressions performed by the robot, including facial expressions and gestures. Eighteen adult participants watched films of the six
basic emotional expressions exhibited by WE-4RII, and chose an emotion correspond- ing to the expression. The recognition rates were: 100% for anger, 100% for disgust, 66.7% for fear, 94.4% for happiness, 100% for sadness, 100% for surprise, and the average of all emotions was 93.5%.
SAYA (Hashimoto et al., 2011) is a tele-operated android robot, that can display human-like facial expressions. SAYA’s face includes actuators distributed on its sur- face in order to improve the structure of the facial muscle-like movement. The facial expressions were designed based on control points of the face, and the directions of movement of those control points were designed empirically or from the anatomical knowledge of the facial muscle morphology on the facial skin. To evaluate whether the designed facial expressions could be recognized, 20 adults observed videos of SAYA performing each facial expression and chose one of six options corresponding to the ba- sic emotions. The authors found a high recognition rate for all the six basic emotions: anger: 92%, disgust: 92%, fear: 100%, happiness: 100%, sadness 100%, surprise: 100%, and the average of all emotions is 97.3%.
From the projects mentioned above only the facial expressions of the humanoid robot FACE were evaluated by children with ASD. Overall, the average recognition rate of the studies presented in this section is 70% for typically developing children and 80.2% for adults. The evaluation of the expressions performed by Kismet and EDDIE were included in the group of adults since detailed information was not provided by the authors, and the group was mostly composed by adults.
Table 2.2 compares the facial expressions’ recognition rates of all the projects presented above, where A = Anger; D = Disgust; F = Fear; H = Happiness; Sa = Sadness; Su = Surprise; Avg = Average.