The overall user study consisted of three sub-studies, conducted concurrently: Study A evaluated hypotheses H1 and H2; Study B evaluated hypothesis H3; and Study C evaluated hypothesis H4. The three sub-studies used the same recruitment pool to enroll participants, though participants were each enrolled in a single sub-study, and they were not be told ahead of time which it was. From the participants’ perspective, a single study was conducted, with no distinct sub-studies. The same gesture system was used for all three sub-studies, and participants in each completed both defined interaction tasks. Because the interaction tasks are both susceptible to learning effects, and it would be very difficult to control for task difficulty across conditions for a single user, all three of the sub-studies were between-subjects.
A total of six study conditions were evaluated in these experiments, exactly two of which are assigned to each of the three sub-studies, with no repetition of conditions. This allowed binary hypothesis testing to be done for each of the four research hypotheses, for direct and clear comparison. Objective interaction performance measures was compared
70
for the pair of conditions in each sub-study (to evaluate hypotheses H1, H3, and H4), and subjective responses was compared as well for the first sub-study (to evaluate H2).
4.6.1. Study A: Conditions 1 and 2
Study A tested hypotheses H1 and H2 using a direct comparison of two study conditions; comparison was done with objective interaction performance measures and subjective participant responses. Recall that hypotheses H1 and H2 are: Including visual affordances and feedback will result in better interaction performance, and Including visual affordances and feedback will result in higher user satisfaction, respectively. Both hypotheses were tested by comparing the gesture system with and without the corresponding types of affordances and feedback; these modifications to the gesture system are organized into study conditions 1 and 2.
Condition 1: No explicit affordances and only required and application feedback. In this study condition, the gesture interface remained intact for participants to use in the interaction tasks. However the designed affordances and feedback methods were not included, with two exceptions: the cursor and the application feedback. The cursor is the minimal required feedback needed to allow the interaction to take place. This gesture interface is driven primarily by pointing gestures, which are universal in human-human interactions in part because of people know the other party can see their hands, and thus know when and where they are pointing. To maintain this universality in a human-machine interaction, the same real-time visibility must be supported by the other party. In this case, that means letting the user know where the system thinks they are pointing, which is done with a cursor. Additionally, application feedback remained, as it is implicit to the
71
interaction task. So for example, if the user moved a card in the interaction space for the sorting task, the card will actually move.
Condition 2: All affordances and all feedback. In this study condition, all the affordance and feedback methods described the System chapter were included in the gesture interface for use by participants in both interaction tasks. This included a total of six types of affordances, and nine types of feedback. Hypotheses H1 and H2 predicted that this condition would outperform Condition 1 in the objective and subjective evaluation measures.
4.6.2. Study B: Conditions 3 and 4
Study B evaluated hypothesis H3 by comparing objective performance measures for two study conditions. Hypothesis H3 predicted that: Affordances indicating how to do something in interface are more important to interaction performance than affordances indicating what the user can do. To evaluate this hypothesis, interaction performance was compared in two study conditions (Conditions 3 and 4), each using a version of the gesture interface that includes only one or the other of these categories of affordances. Both interface versions included all the feedback of Condition 2, though.
Condition 3. All feedback, but only affordances indicating what the user can do. In this study condition, the only type of explicit affordance included in the gesture interface were those that a user may perceive as classical action potentials – that is, only clues telling the user what they can do in the interaction. The affordances that were included in this condition were: indications to the user that the interface supports hand
72
gesture interaction, indications that it supports pointing gestures, indications that an item in the interface can be selected, and indications that an item can be moved.
Condition 4: All feedback, but only affordances showing the user how to perform gestures. In this study condition, the only type of explicit affordance included in the gesture interface are those serve to teach or show the user how to perform a [potentially unfamiliar] gesture. Two affordances are included in this category and therefore in this study condition: showing the user how to select an item with tap-to-select gesture, and showing the user how to release an item with a withdraw-to-release gesture. The affordances included in Study B’s two study conditions form mutually exclusive sets.
4.6.3. Study C: Conditions 5 and 6
Study C is the final part of the overall user study, and evaluated hypothesis H4, again by comparing objective performance measures for two study conditions. Hypothesis H4 predicted that Feedback indicating system status is more important to interaction performance than feedback acknowledging user actions. Evaluation of this hypothesis was done by comparing interaction performance in Conditions 5 and 6. Each of these conditions included all affordance types, but only those feedback types in one or the other of the two feedback categories.
Condition 5: All affordances, but only feedback acknowledging user actions. In this study condition, the gesture interface the participants encountered included only visual feedback that acknowledges to users that the gesture system has detected and recognized a particular action they’ve performed. Four types of feedback were included in this category and thus formed part of the gesture interface in this study condition: feedback
73
acknowledging that the user has raised their hand into view, feedback acknowledging that the user is pointing within the interaction space, acknowledgement that the user has performed a tap gesture, and acknowledgement that the user has performed a release gesture.
Condition 6: All affordances, but only feedback indicating system status. In this final study condition, the gesture interface included only feedback that provided the user with unprompted information about the status of the gesture system. Two types of feedback are included in this category and study condition: feedback indicating to the user that their hand is not visible to the system, and feedback indicating that their hand is outside the bounds of the interaction space. These are in contrast to the types of feedback included in Condition 5, and again the feedback included in the two conditions compared in this sub-study form mutually exclusive sets, however the two exceptions from Condition 1 apply here as well. Feedback that acknowledges pointing gestures performed by the user – a cursor – must still be included here, despite being outside of the defined category. And the application feedback, being an implicit part of the application interface, must also remain present.