The characteristic that I use to differentiate how CDOM interaction techniques refer to destination displays is spatiality. An interaction technique is spatial when the action required by the user encodes spatial information of the location of the destination display in the environment. For example, consider the early Put-that-There technique [Bolt, 1980]. In this technique, the user points with the finger (or with some other device) to indicate the intended destination (see Figure 8). Note that although the Put-that-There technique uses voice to activate the selection and release of the object, the information about where to put it is specified only by the pointing gesture.
Figure 8. The Put-that-There technique as envisioned by Bolt [1980] (from left to right). Most existing CDOM techniques are spatial, although there are several different varieties. Laser pointing techniques [Oh & Stuerzlinger, 2002, Voida et al., 2005, Myers et al., 2002, Parker et al. 2005] (also called remote pointing, laser beam, or distance pointing techniques) belong in the same group than Put-that-There. Variants usually differ in the device used to point and the activation method (e.g., voice, a button integrated in the device, a button in the other hand, a hand gesture, and dwell time), but are essentially identical in how the direction of the arm (or the device) indicates the destination.
Direct input techniques such as Pick-and-Drop [Rekimoto, 1997] (See Figure 9), or Passage [Prante et al., 2004] also make use of spatial referencing, but these techniques require direct physical contact with the destination display. Other examples of spatial interaction techniques are those based on the world-in-miniature paradigm (WIM – graphical representations of the physical space) [Bieh and Bailey, 2004, 2006; Swaminathan and Sato, 1997] and those in which the cursor, steered by a mouse or any other indirect input device, jumps from one display to another according to the physical location of the displays [Wallace et al., 2008; Johanson et al., 2002; Baudisch et al., 2004; Benko and Feiner, 2005, 2007]. Notice that even though in this last group the spatial relationship between the gesture and the physical space is indirect, the gesture still relates to the spatial arrangement of the physical space. For example, in world-in-miniature techniques, the gesture relates to the location on the miniature (map), and this in turn, is a representation of the physical reality. If the relationship between the map and the physical layout of the MDE is not spatial (e.g., if the different displays are arranged in the map according to their frequency of use, their owner, etc.), then the technique cannot be considered spatial, but would fit in the non-spatial category described below.
Put
This brief summary of spatial techniques should give an idea of the variety possible within spatial techniques. Although there are more existing and possible spatial techniques, I postpone its discussion to Chapters 4, 5 and 6, which deal in more detail almost exclusively with differences in performance of spatial techniques of different types. This chapter, however, is focused on differences due to how techniques reference to displays.
Figure 9. The Pick-and-Drop sequence. A) The user touches the object with a pen and clicks a button. B) The user moves to the destination location (while holding the button). C)
The user releases the button while touching on the desired destination.
CDOM interaction techniques where the required action by the user does not relate to the spatial layout of the displays in the physical space are called non-spatial techniques. The most common examples of non-spatial techniques are those that use some kind of naming of the displays. For example, in Multibrowsing [Johanson et al., 2001], the user can redirect content from one screen to another by selecting the destination from a list of available displays (see Figure 1 in page 3). In this case the interaction technique is non-spatial because the control movement (moving the mouse so that the cursor is on top of the correct item of the list) is not related to where the display is in physical space. Instead, the destination display is indicated by a symbol (the name of the display).
The same kind of referencing by name is common in cross-device file transfer techniques such as Instant Messaging (IM) [Microsoft, 2008c], Shared Folders [Microsoft, 2008b], or E- mail4. For example, in IM, instead of selecting the destination display by the name of the display (as in Multibrowsing), the user has to select the name of the buddy that is connected to that
4
Instant messaging, shared folders and e-mail were not originally intended as cross-display object movement techniques but they are often used for that purpose in co-located situations due to the lack of more specific support for CDOM. For example, it is common to pass a file through IM to a nearby co-worker because the two machines are usually connected through a LAN or the Internet, but not connected at the level of an interface that knows of their physical proximity.
display from the buddy list. E-mail can require a very similar action (selecting from an address list) or an even more symbol-based action
contrast, shared folders can be more spatial in the sense that the icons of the remote fo
be spatially distributed around the desktop. In fact, if the user takes the time to arrange them according to the physical arrangement of their destinations, this technique can be considered spatial (it becomes a map of sorts
distributed in the display randomly, the technique is equivalent to IM and E
above, with the exception that the input movement is not limited to one dimension as in selecting from a list, but it is instead two
within the plane).
Figure 10. An example of Spatially Arranged
There are a large number of potential variations of techniques based on symbolic selection of the destination. For example, symbolic links could be established between a destination display and color, shapes, ideograms, hieroglyphs, photos of the display, sounds, tactile textures, and almost anything imaginable. However, most of these do not
use. Most existing techniques use text (i.e., the name of the display). Aside from symbolic techniques, there exist other non
navigation of some data structure related to the destinations. Two examples of these are the keyboard-switch Multi-Monitor Mouse and the mouse
mail can require a very similar action (selecting from an address based action, such as typing the address of the recipient. In contrast, shared folders can be more spatial in the sense that the icons of the remote fo
be spatially distributed around the desktop. In fact, if the user takes the time to arrange them according to the physical arrangement of their destinations, this technique can be considered spatial (it becomes a map of sorts – see Figure 10). However, if the shared folder icons are distributed in the display randomly, the technique is equivalent to IM and E
hat the input movement is not limited to one dimension as in selecting from a list, but it is instead two-dimensional (the control movement can go in any direction
. An example of Spatially Arranged Folders (a spatial technique).
large number of potential variations of techniques based on symbolic selection of the destination. For example, symbolic links could be established between a destination display hieroglyphs, photos of the display, sounds, tactile textures, and However, most of these do not make sense and would be difficult to . Most existing techniques use text (i.e., the name of the display).
Aside from symbolic techniques, there exist other non-spatial techniques that are based on the navigation of some data structure related to the destinations. Two examples of these are the
Monitor Mouse and the mouse-button-switch Multi-
mail can require a very similar action (selecting from an address typing the address of the recipient. In contrast, shared folders can be more spatial in the sense that the icons of the remote folders can be spatially distributed around the desktop. In fact, if the user takes the time to arrange them according to the physical arrangement of their destinations, this technique can be considered ). However, if the shared folder icons are distributed in the display randomly, the technique is equivalent to IM and E-mail described hat the input movement is not limited to one dimension as in selecting dimensional (the control movement can go in any direction
olders (a spatial technique).
large number of potential variations of techniques based on symbolic selection of the destination. For example, symbolic links could be established between a destination display hieroglyphs, photos of the display, sounds, tactile textures, and make sense and would be difficult to
spatial techniques that are based on the navigation of some data structure related to the destinations. Two examples of these are the -Monitor Mouse
presented by Benko and Feiner [2005, 2007] (most other techniques presented in this paper can be classified as spatial). In these variants of the Multi-Monitor Mouse, the cursor switches from one screen to the next in a circular list of available displays, much as the Alt-Tab key combination switches between applications in Windows™ systems.
Unfortunately, it is difficult to further categorize techniques based on non-spatial referential domains or to study them all in detail because of the countless possibilities for assigning symbols or descriptions to displays or locations inside displays (symbolic techniques) and because of the large number of possible ways to traverse the space in navigational techniques. For the rest of the chapter I have chosen text-based symbolic techniques (e.g., naming schemes such as the one in Figure 10, or IM) as the most prominent representative of the non-spatial group of techniques. Although other non-spatial techniques can offer an unexplored and potentially fruitful direction of study (one that certainly deserves further research), I do not have any reason to believe that other symbolic ways to refer to displays (e.g., color, texture, etc.) can outperform text.