SELECCIONAR TECNOLOGÍAS Y DISPOSITIVOS PARA

The following analysis identifies specific behavioural practices and resources, including repair strate- gies, that participants were seen to employ during interaction. CA combined with gaze data forms the foundation of the analysis, and extracts describing participants’ behaviour during both correct and incor- rect responses to grab instructions are presented. Participants’ use of movement and gaze as resources during the experimental task is then explored. Finally, post-hoc analysis of eye tracking data to explore the specific gaze practice of ‘glancing’ is presented. Glancing was used extensively by all participants as a repair strategy. This analysis is primarily concerned with interaction related to grab instructions,

5.3. Analysis 147

Figure 5.10: CA transcript of tracked gaze AMC. Confederate, “Rob”, issues a grab instruction to participant, “Tim”, who responds by grabbing the correct cube.

rather than position instructions. As suggested by the task performance analysis, responses to position instructions were not as revealing as those to grab instructions. This is further discussed in Section5.4.

Responses to Grab Instructions

Participants responded to grab instructions either by correctly or incorrectly identifying the cube indi- cated by the confederate issuing the question. Figure5.10shows how an example case of correct and unproblematic cube identification and grabbing unfolds between a participant, “Tim”, and the currently questioning participant, “Rob”. At line 01, Tim is gazing toward the puzzle base. Rob summons Tim’s attention by calling his name and subsequently eliciting his gaze as indicated at the beginning of line 02. Rob produces a pre-instruction, also at line 02, which Tim acknowledges both vocally and with gaze at line 03. At line 04, Rob issues the instruction, during which Tim continues to gaze toward him. As he produces the deictic term “this”, at line 04, Rob turns and shifts his gaze to the designated cube. After a moment, and as Rob utters “now” at line 05, Tim follows Rob’s direction of gaze, shifting his own to focus on the correct cube. Tim points towards the cube at line 06, and requests confirmation that his decision is correct. Tim performs this request verbally, with “Is that this one”, followed by a return of gaze to Rob. At line 07, following Rob’s affirmative response, Tim focuses his gaze back on the cube, and grabs it.

In contrast, Figure5.11shows a case of problematic interaction during a grab instruction. In this extract, a participant, “Owen”, firstly requests clarification as to which confederate is currently talking, and subsequently selects an incorrect cube. At line 01, Rob issues an instruction as Owen gazes at various cubes in the VE and then at the bystanding confederate, “Paul”. During the 3.2 seconds of

5.3. Analysis 148

Figure 5.11: CA transcript of model gaze AMC highlighting problematic interaction and subsequent repair during a grab instruction. Confederates are “Rob” and “Paul”. Participant is “Owen”.

silence that follows during line 02, Owen alternates his gaze between the two confederates. Realising that Owen is unsure of which confederate just issued the question, Rob establishes that it was he, at line 03. At line 04, Owen vocally acknowledges this, as does Paul, who also points towards Rob. At line 05, as Owen clarifies Rob’s instruction, he shifts his gaze from Rob to a group of potential cubes. Rob confirms this action at line 06, and after 1.3 seconds of deliberation, Owen grabs an incorrect cube at line 07. Responding to this error, Rob states that the choice is incorrect, and Owen returns his gaze to Rob at line 08. Owen apologises at line 09, and returns the incorrect cube to its original position. At line 10, Owen states that he will try again, and moves very close to Rob’s avatar in order to examine its direction of gaze. At line 11, Owen identifies and grabs another cube, and asks Rob for confirmation that his choice is correct. Rob confirms that Owen has grabbed the correct cube at line 12, completing the grab instruction.

The above cases of successful and problematic interaction demonstrate how varying responses to instructions have the potential to significantly reduce task performance, in terms of both number of errors and time taken. The two cases also illustrate participants’ strategic use of confederate avatars’

5.3. Analysis 149 gaze in order to inform their own action. This is particularly clear in the case of problematic interaction, in which, following an incorrect response to a grab instruction, the participant closely examines the instructing confederate’s gaze in an attempt to repair the progress of interaction.

Movement as a Resource

When deciding which cube to grab, participants used confederates’ proximity to cubes as a central cue. Correspondingly, confederates made extensive use of the ability to move freely within the VE, position- ing themselves at an appropriate distance and angle (both vertically and horizontally) to the intended cube, thereby aiming to provide the participant with less ambiguous information. As the CA extract in Figure5.11illustrates at line 10, participants often moved very close to the instructing confederate’s avatar, in an attempt to more clearly determine their direction of gaze. This is one category of a number forms of locomotion that participants engaged in. In a small number of cases, participants were observed to take up an opposite position to the instructing confederate, so that potential cubes lay between the pair, in order to identify the target cube. However, a more common strategy adopted by participants was taking a position similar to that of the instructing avatar, by standing beside them, looking over their shoulder, or even standing “inside” them as allowed by the ICVE system. By adopting a common perspective, participants were able to share the confederate’s view of the virtual scene. In this way, participants made use of the spatial characteristics of the ICVE system, employing movement strategically in an attempt to perform the task more effectively.

Gaze as a Resource

Gaze was employed as a general nonverbal resource during the multiparty interactions, and participants distributed their own gaze naturally between confederate avatars and cubes during periods of instruction, conversation, and silence. Participants were frequently observed to base their action on the direction they perceived the instructing confederate to be gazing. During both grab and position instructions, participants attempted to determine which cube or location was being indicated by focusing on, and following the instructing confederate’s gaze. As exposed by task performance analysis, the gaze model condition in particular caused difficulties during interaction due to the unfaithful display of confederate gaze. The impact of this incorrect nonverbal information frequently surfaced during interaction with explicit participant vocalisation types including questions (“Are you looking at it now?”), complaints (“You’re looking up! Not down.”) or accounts (“It’s a little difficult for me because your eyes keep moving about.”).

These moments of problematic interaction, caused by confederate avatars displaying unfaithful gaze behaviour, highlight the operational importance of gaze as a communicative resource that is highly informative to subsequent action. Participants perceived the gaze of confederate avatars to reflect that of their embodied user, and thus, instinctively attempted to use the cue as a bidirectional resource, similarly to its role in collocated interaction.

5.4. Discussion 150 Table 5.5: Mean number (and standard deviation) of glances to and from the instructing avatar and target cubes per grab instruction.

Condition Static Model Tracked

# of Glances 2.42 (1.5) 3.58 (2.74) 2.25 (1.57)

Glances Per Grab Instruction

Following a grab instruction being issued be a confederate, a strategy adopted by all participants was to alternate gaze between the instructing confederate’s eyes and the potential cubes. Over all twelve participants and experimental sessions, a total of 180 grab instructions were issued, and this strategy of glancing between confederates and cubes was employed in 176 (98.9%) of cases. By gathering infor- mation from confederates’ gaze, participants were able to develop their judgement as to which cube was being indicated before taking action. The method was also employed extensively as a repair strategy following an incorrect grab. The number of glances per grab instruction was analysed with a repeated measures two-way ANOVA calculation, with the three gaze conditions and the five grab instructions as factors. A significant difference between conditions (F(2,4) = 8.23; p < 0.0005) and also between instructions (F(2,4) = 4.54; p < 0.002) was found. Similar to analysis of grab instruction errors and timings, post-hoc Tukey tests determined that the differences lay between the gaze model and the other two classes of static gaze (F(1,4) = 8.81; p < 0.005) and tracked gaze (F(1,4) = 10.95; p < 0.001). No interaction effect was found between condition and grab instruction.

Table5.5shows the mean number of glances performed per grab instruction under each gaze condi- tion. These values are representative of a gaze control method’s ability to clearly and efficiently commu- nicate attentional information to an observer, with sufficient fidelity upon which to base critical action. Hence, when engaged in tracked and static gaze AMC, participants needed to performed fewer glances before responding to grab instructions than when engaged in modelled gaze AMC. This is likely to be an influential factor to the increased task performance (reducing the time taken and number of errors) observed in the tracked and static gaze conditions. Although the questionnaire did not expose significant differences of between gaze conditions, the user experience is also likely to be improved, with more effi- cient interactional flow, and reduced need to employ repair strategies. Although not statistically superior to static gaze, tracked gaze is identified as the highest-performing method of gaze control by this metric.

5.4

Discussion

Analysis aimed to measure quality of communication in terms of task performance, user experience, and interactional behaviour. In the categories of task performance and interaction analysis, tracked and static gaze AMC were seen to significantly outperform the gaze model condition. The poor performance of the model was explored in Section5.3.2. This demonstrated how the simulated gaze behaviour was in contrast to the actual tracked gaze behaviour of participants, particularly in terms of eccentricity, saccade magnitude, and fixation duration. However, analysis of both task performance and interaction analysis failed to expose statistically significant differences between tracked and static gaze. Therefore, the initial hypothesis that tracked gaze avatars would enhance quality of communication during object-

5.4. Discussion 151 focused collaboration is not supported. Both intuitively, and when considering the enhanced throughput of nonverbal information delivered by tracked gaze, it is somewhat surprising that this was found to be the case. The following discussion explores influential factors toward this result, focusing particularly on the impact of gaze on the experimental interactions, and the technical properties of the ICVE setup. The lack of a significant result in subjective user experience is also discussed.