Redirecting content from a (possibly distant) external display to a user’s mobile device can have several characteristics: first, the entire display’s content can be shown on the mobile device. Second, users can select a certain region of interest within which they wish to interact in. These regions can either refer to a single information item or a sub-region of the entire display. And third, the information can be brought to users in a time-multiplexed fashion, potentially causing delays. In this section we give a brief overview of existing systems in each of these categories.
The World-in-Miniature Representation:
The simplest form of bringing content to the user is to show a miniature representation of the target display. This technique is often referred to as world in miniature [SCP95]. Within this view, users can interact with remote content on their local display. From the user’s point of view the content is being redirected. However, from the technical perspective, each input occurring in the local miniaturized view is redirected to the target display, causing it to change its visual content. These changes are then also updated on the local representation. One major drawback is the degree of miniaturization needed to allow both local and remote content being shown. Hence, the control-display (CD) ratio (i.e., the ratio between input movement and target movement) is decreasing when the local display requires higher size reductions. While an interactive tabletop in combination with a wall display is rather uncomplicated, the usage of a mobile device together with a large display will harm the interaction efficiency.
2.3 Interacting at a Distance 23
Especially if both devices differ in size (i.e., the local display is much smaller than the target screen), additional techniques have to be found to allow high precision input. One common issue is thefat finger problemwhen touch devices are used. If the screen is highly miniaturized already, the user’s finger occludes significant portions of the remote display’s content. Vogel et al. showed how shifting the content can help to increase the targeting accuracy of finger input for small tar- gets [VB07]. While the input is made more accurate, the high visual capacities of large displays cannot be reproduced on small screens. To address this, such systems use local magnifications of the content region of interest. These techniques partially distort or remove content surrounding the area of interest [LA94]. The distortion is further dependent on the factor of magnification needed which in turn correlates with the difference in screen sizes of both displays. Hence, such approaches only partly reduce the problem.
Figure 2.4: Bringing content to the user: a) Select sub-regions of desired content with
WinCuts [TMC04]. b) Bridge large distances using Frisbee [KFA+04]. c) Dragging the
entire content toward the user similar to aTablecloth[RCB+05].
Selecting Regions of Interest:
Instead of showing the entire remote screen on the local display, users can select regions of in- terest (i.e., parts of the remote screen’s content) a priori. The simplest form is a shared space holding documents and applications as shown inIMPROMPTU [BBB+08]. Tan et al. present
WinCuts, a system that allows users to cut out content of a remote screen for subsequent local
manipulation [TMC04]. If the selected content area is smaller than the local device’s screen, no miniaturization occurs (see figure 2.4a). The CD ratio remains constant allowing for high- precision manipulation. Repositioning the current cut can then be done using the Tablecloth
technique [RCB+05]. Users simply drag the entire content until the new region of interest is shown (see figure 2.4c). Consequently, the cut moves in the opposite direction of the drag oper- ation. Radar Views allow similar kinds of interaction [NASG05]. They present a sub-region of
the target display in which users can interact. The concept ofFrisbeeworks likewise [KFA+04]: in this system, users interact with a local representation of distant content (see figure 2.4b). The view of distant content can be moved both locally (i.e., the lens is repositioned, the content re- mains) or remotely (i.e., the position of the lens remains, but the shown distant content is moved). The authors also propose a linked version in which remote content movement corresponds to the local motion of the lens. In both systems the context of the selected region is not shown during the interaction. This potentially may harm the interaction or lead to zooming operations if users want to get an overview of the current region.
Figure 2.5: Spatial versus temporal content movement: a) Accessing content by dragging
towards it with Drag-and-Pick [BCC+03]. b) Time-multiplexed content movement on a large surface in predefinedInterface Currents[HCS06].
Indicating a direction can bring content of interest to the user. Baudisch et al. presentDrag-and- Popwhich allows users to drag an item toward possible target icons (i.e., icons that can handle the dragged item) [BCC+03]. Similarly, users can select items by moving the cursor in the direction of target icons using Drag-and-Pick. The icons are warped to their temporary location to give a hint of their origin (see figure 2.5a). When used from a personal display the spatial arrange- ment of displays in the environment (and especially the relationship between mobile and distant screen) needs to be known during the interaction. To avoid the need of positioning methods, Collomb et al. presentPush-and-Pop [CHBL05]. Their system combines the aforementioned
Drag-and-Popwith thePush-and-Throwmethod [Has03]. When users start to drag an item, the
system responds with potential target icons regardless of their direction. Hence, this approach works when invoked on a mobile device without knowing where it actually resides. However, as discussed in section 2.6, users need to be aware of the icon’s origin.
Time-multiplexed Content Redirection:
In contrast to the discussed approaches, content can also be brought to users in time-multiplexed ways (see figure 2.5b). In such systems, content items are constantly floating on given paths. At the same time, the content passes by each user. Hinrichs et al. presents an implementation
2.3 Interacting at a Distance 25
on a tabletop computer [HCS06]. While their system only uses one display as workspace, the proposed Interface Currents can also float across multiple displays. Users are able to reach content that would otherwise be distant when it is passing them. Spreading such interfaces across multiple screens requires the spatial arrangements of displays in the environment to be known a priori. Only this ensures the logical routing of information across displays.
Content shown on a distant display can also be brought to users if it is of interest for them. Rukzio et al. present theRotating Compass, a system that presents navigational information to users [RSK05]. A large (projected) display shows a simplified version of a compass with the four cardinal points while a compass needle is rotating. If the compass is pointing into the correct direction for a certain user, the mobile device of this person vibrates. This notifies the user that the currently shown information (i.e., direction) is the correct one. While this system does not transport information to the user’s mobile device from a technical point of view, it still conveys information to the user by linking both the external display and the mobile device.