The human being is well-equipped to interact with physical objects in the real world. However, interacting within virtual environments is considerably more challenging for us - as virtual objects do not provide tactile feedback8. Because AR derives several interaction patterns from VR, it is important to better understand their affordances and limitations. This section briefly addresses theoretical concepts and interaction techniques that relate to our work.
Benford et al. introduced a general, theoretical framework for mutual awareness in collaborative virtual reality called the spatial model of interaction [56]. The key
components to the spatial model are space and the objects that inhabit that space, aura –
which is a sub-space that surrounds an object and acts as an enabler for interaction, focus
– which is the direction of attention (increasing awareness of the object), and nimbus – which is the capability for objects to make themselves available to others (i.e. increasing the nimbus makes an object more noticeable). The work of Curry in tele-immersion extended Benford‟s model, identifying major types of awareness including presence, attention, environmental and action awareness [57]. To maintain gaze awareness, Hindmarsh recommends rendering the view frustum9 of participants during referential tasks [58].
It is possible to divide selection techniques into the more primitive components of 1) an indication of the object of interest, 2) a confirmation the selection by the user, and 3) feedback that the selection was completed [59, 60]. Variables that affect selection performance include target distance, size, number, density of surrounding objects and target occlusion [61]. We can classify interaction techniques as either isomorphic - in
which there is a direct mapping between the real and virtual hand position - or non-
isomorphic - which use linear or exponential scaling to perform selections at a distance.
Poupyrev further categorized interaction techniques as either ego-centric (i.e. from the perspective of the avatar – first person) or exo-centric (i.e. any other viewpoint) [62]. Exo-centric metaphors include World-In-Miniature (WIM – where the world is scaled to fit in the palm of your hand in order to overcome distance) or automatic scaling
techniques. Ego-centric metaphors are further decomposed into virtual hand techniques (such as the “classical hand”, “Go-Go” 10
and “Indirect Go-Go”) and virtual pointer metaphors (like ray-casting, aperture, flashlight and image plane techniques) [61, 63].
A primary benefit to using the pointing metaphor is that it naturally affords selection of objects at a distance, allowing the user to remain stationary. The most common method is ray-casting, in which an infinitely-long line emanates from the virtual hand to intersect with a virtual object. However, this technique suffers from the magnification of small errors (from tracking or user ability) over distance, its inability to reference general space, and the possibility for the ray to intersect with more than one object. Several theme and variations are derived from this, including two-handed pointing, where one hand controls the direction vector and the relative distance between the two hands controls its length [64]. Using two hands, another technique is the curved virtual pointer, where a twisting of the hands bends the pointer to select occluded objects [65]. Still another variation is the flashlight technique where a conical volume is projected from the hand [66]. This alleviates some of the distance-related inaccuracy problems of ray casting, but may be too course of a technique in clustered scenarios. To reduce this limitation, the aperture technique allows the user to control the radius of the conic volume [67]. The “fishing reel” method allows users to control the length of the virtual ray through the physical manipulation of a specialized device, such as a slider. Image plane techniques allow users to select objects by their projection onto a 2D virtual image plane - essentially framing the object. For example, the “head-crusher” technique
10 Named after the children’s TV show Inspector Gadget, in which the appendages of the main character
allows a user to select an object by placing their hand between the eye and the object, and positioning their index finger above and their thumb below the object (allowing someone to virtually “crush the head” of another person) [68]. In general, Bowman et al. recommend pointing metaphors for selection in virtual environments [61].
The virtual hand metaphor includes the linear hand, the Go-Go technique, and the World-In-Miniature [69]. Simple hands are isomorphic – directly mapping to movement in the real world and serving as a 3D cursor in the virtual world. Its graphical representation can take the form of an actual hand, or can be a semi-transparent volume, such as the “Silk Cursor” - which uses a semi-transparent bounding box which allows the user to view occluded virtual objects and yield important depth cues [70]. The Go- Go technique is similar in behavior to the classic hand - where movements within a fixed distance from the user‟s origin are isomorphic; however, once the hand extends beyond a pre-determined distance, the hand becomes non-isomorphic, scaling exponentially. A final variation on the virtual hand is the world-in-miniature technique, in which the entire virtual world is scaled to fit within the user‟s virtual hand. Users can then select virtual objects in the shrunken world, and have those selections reflected in the fully-scaled environment. A drawback of this technique is its inability to work in crowded or very large environments, or when selecting small objects.