Humans rely heavily on visual cues, particularly non-verbal visual communication to negotiate
social contexts (Beattie & Ellis, 2014). Birdwhistell (2010) postulated that during a dyadic
conversation, only one-third of the social meaning is conveyed by verbal components, the
Figure 18. Combined verbal and non-verbal social signals during a dyadic interaction (reproduced from Vinciarelli, Salamin & Pantic, 2009, p.43). Verbal cues include language, tone of voice, and intonation. Non-verbal cues include posture, personal space, expressions and gestures. The combination of these signals allows humans to decipher a social situation, for example, with the above image, hostility, aggressiveness and disagreement.
Indeed, Mehrabian (1971) proposed that up to 93% of a conversational message is conveyed
by non-verbal components. These non-verbal components are usually visually processed and
include aspects of human behaviour such as eye contact, facial expressions, body language and
gestures (Knapp & Hall, 2002). According to Argyle (2013), there are five main functions of
non-verbal communication; to express emotion, facilitate interpersonal relationships,
accompanying speech to create a feeling of synchronisation, self-presentation, and to uphold
cultural rituals, for example, handshaking. The ability to correctly identify, assimilate and
engage paralinguistic features, non-verbal and contextual cues from the social environment is
vital in behaving appropriately during social situations (Mah, Arnold & Grafman, 2004).
The present research was interested in exploring the visual non-verbal communicators of mood
most communicative modes utilised during social interactions, and correct identification of
facial expressions is central to adaptive social functioning (Hinojosa, Mercado & Carretié,
2015; Jack & Schyns, 2015). Human faces display visible signals of social intentions,
motivations, and communicate internal emotional states (Hess & Hareli, 2015; Schmidt &
Cohn, 2001). It is generally accepted that there are six basic facial expressions/emotions which
are recognisable across cultures; anger, fear, disgust, happiness, sadness and surprise (Ekman
& Keltner, 1997). Nevertheless, there is still a lack of consensus between theorists regarding a
common taxonomy of emotions (Izard, 2009, 2010; Turner & Stets, 2006), mainly because
they are investigating phenomena from different theoretical positions.
As well as emotions, human faces also display intention gestures and signals, such as the
eyebrow flash which indicates a desire to communicate (Frith, 2009), yawning, which is
theorised to promote emotional contagion (the transfer of emotion and mood to other
individuals (Ferrari & Coudé, 2018), and eye gaze (Slonimska, Campisi & Ozyurek, 2015),
which is the basis for joint attention; the ability of two or more individuals to concurrently
utilise gestures or eye gaze to focus attention on salient objects or events in the environment
(Jones & Carr, 2004). These gestures are critical for adaptive social behaviour as they allow humans to perceive and interpret other people’s intentions, thoughts, and behaviours during
social interactions, as well as mediating the generation of contextually appropriate social
responses (Green et al., 2008). Interestingly, Elder (2018) explored the use of emojis
(ideograms displaying facial expressions) in textual computer-mediated communication. The
research concluded that the use of emojis had a positive effect on individual well-being and
interpersonal communication, directed attention, expressed or acknowledged difficult
emotions, and increased altruistic tendencies, highlighting the human reliance on facial
Facial expressions can also be used to build rapport consciously through friendly displays
(Grahe & Bernieri, 1999), and unconsciously through a process of mirroring (the chameleon
effect which is the unconscious imitation of facial expression, gestures or speech patterns;
Lakin, Jefferis, Cheng & Chartrand, 2003; Tramacere & Ferrari, 2016). The evolutionary
benefit of mirroring is that it drives humans to unconsciously produce survival behaviours. For
instance, a fearful face is indicative of a threat and the physiological facial adaptations
associated with fear (e.g. widening of the eyes and nostrils) enlarge the visual field, inspiratory
capacity, and sense of smell (Susskind et al, 2008).
Humans exhibit an innate preference for faces and evidence has proposed that the human visual
system rapidly identifies human faces compared to other visual stimuli (Frank, Vul & Johnson,
2009). Eye-tracking research has reported that human saccades (quick movements of the eye
in preparation for a fixation) are exhibited as early as 100 milliseconds after stimulus onset for
happy faces (Crouzet, Kirchner & Thorpe, 2010). When faces are paired with other objects (e.g.
vehicles), initial saccades are biased towards faces, even when the other object is the target,
suggesting that these quick saccades are not under conscious control (Crouzet et al., 2010).
This unconscious bias appears to extend to social cues from facial expressions (e.g. eye gaze).
Deaner and Platt (2003) reported that when participating in a peripheral visual target detection
task, macaque and human performance levels were both facilitated when eye gaze was
consistent with the target location. These findings indicated that primates displayed innate
covert attention to face stimuli. Abnormalities within this social perception system, for example
avoiding looking at the eye area of a face, have been related to poor social functioning in
autistic cohorts (Tanaka & Sung, 2016).
The neural networks underpinning facial affect recognition are extensive and beyond the scope
low and high-level processes, activating early sensory cortices (discussed in detail in chapter
three), subcortical structures, and widespread cortical regions. In brief, Phillips, Drevets, Rauch and Lane’s (2003) neurobiological model of emotion perception suggests two emotion
perception neural systems. Firstly, a ventral system, predominantly important for identifying
emotion valence and producing appropriate affective responses, and included the amygdala,
insula, ventral striatum, and ventral regions of the anterior cingulate gyrus and prefrontal cortex.
Secondly, a dorsal system, necessary for the regulation of affective states, and included the
hippocampus and dorsal regions of the anterior cingulate gyrus and prefrontal cortex. TBI, even
mild, has been associated with structural and neurochemical brain changes, including DAI, in
the aforementioned regions (Eierud et al., 2014; Mayer, Mannell, Ling, Gasparovic & Yeo,
2011), and could account for the frequently reported emotion perception impairments post-TBI
(Babbage et al., 2011).
In summary, facial expressions are arguably the most important conveyor of human mood state
and aid social behaviours such as emotional contagion and joint attention. Even in the ‘digital age’, where there has been a reduction in face to face, humans have perceived the need to
develop a facial expression system to facilitate the message they are trying to convey,
highlighting human dependency on this intrinsic ability (Elder, 2018). Facial affect
recognition draws on a diverse set of brain networks including; early sensory cortices,
subcortical structures, and widespread cortical regions, notably the frontal cortex and the
motion detection regions of the parietal cortex (Adolphs, 2009; Phillips et al., 2003). Facial
affect recognition was identified as the primary focus for the current thesis because facial
expressions are considered to be the most important non-verbal conveyor of mood state in
humans (Croker & McDonald, 2005; Frith, 2009), and impairments in understanding facial
al., 2017). The next section critically reviews the effects of TBI on social cognition with a
specific focus on facial affect identification.