Finally, the most related sets of previous work are those that investigated problems of physical occlusion on tabletop displays. In the recent decade, work is emerging that
Figure 3.4: Overview of occlusion management techniques presented by Javed et al. [Javed 2011]. The large gray rectangle represents the footprint of a physical object placed on a tabletop display.
partially addresses the problem space of physical occlusion. Javed et al. [Javed 2011] presented six different techniques to address occlusion in a physical-virtual setting. The techniques are depicted in figure3.4. Each of these techniques helps users deal with a single issue – such as identifying or accessing occluded digital objects. The techniques are evaluated through two experiments in a simulated hybrid setting using virtual occluders, but no physical objects. Experiments focus on guided and unguided visual search tasks with various numbers and sizes of occluded objects. Their results indicate that techniques that consume less space and provide less visual clutter are in general more promising. The study shed light on the characteristics of each individual technique and inspired the design of some of our techniques.
The goal of the SnapRail [Furumi 2012] interface was to alleviate the visibil- ity and manipulability problems of virtual objects occluded by physical objects on tabletops. The interface is basically a rail widget that appears around the phys- ical occluder. Once occlusion happens, occluded virtual items are automatically rearranged and snapped around the rail widget. Users can then browse the virtual elements along the rail widget. A preliminary user study of the SnapRail interface showed that it was easy to use and received positive feedback from participants.
(a) (b)
Figure 3.5: (a) Five different zones found in Tabard’s work [Tabard 2013] on which participants placed physical objects and (b) a user-drawn menu designed by Lei- thinger et al. [Leithinger 2007] using a stylus
Tabard et al. [Tabard 2013] investigated how users managed physical and digital objects during the 16-week deployment of a tabletop in a biology lab setting. The study was conducted on the eLabBench [Tabard 2012] tabletop system to support synthetic biologists. For the study, four participants used the eLabBench, covering 15 hours of bench work, extending over five sessions each lasting from 8 to 30 minutes. Based on analyzing video codes, their results revealed that on average, 15% of the tabletop screen was occluded at any point in time. They found five different tabletop zones where participants placed objects as illustrated in figure
3.5a:
1. long term storage: outside of tabletop screen, in which no occlusion occurred 2. storage: areas located at a far distance to users
3. observation: areas for short and transient observation 4. digital: where digital objects are mainly used
5. interaction: where small physical objects are frequently placed and used. Based on their results, they suggested that occlusion management techniques should either limit distractions as new objects are put on the table or avoid occlusion rather than focus on reacting to it.
Other researchers have proposed approaches to occlusion avoidance for hybrid tabletop settings. Cotting and Gross [Cotting 2006] introduced environment-aware
display bubbles that aim at significantly enhancing the flexibility, interactivity, and adaptivity of displays. They described a top-projection display metaphor that au- tomatically projects free-form bubbles on a tabletop. Using a warping approach the display content is represented by multiple bubble forms that can elastically be manipulated.
Leithinger et al. [Leithinger 2007] investigated digital menu representation on mixed digital-physical tabletops. In contrast to automatic placement, they pre- sented a user-driven approach in which users, using a stylus, draw menus of custom shapes around areas that may be occluded by physical objects (cf. figure 3.5b). In this way, users can simply draw an occlusion-free menu with less confusion. Their design also requires less technical computations for automatic recognition of phys- ical objects. They proposed four different metaphors derived from natural ways of spreading information. They compared these techniques with traditional pie and pop-up menus in normal and cluttered environments. Their techniques showed a significant reduction of time in cluttered settings.
Freeman et al. [Freeman 2013] presented a technique to find uncovered areas on a tabletop surface to represent occluded content. Their technique first detects the footprint of physical objects atop the tabletop using a blob detection algorithm, and constructs a binary matrix that shows occluded areas of the display. Using this matrix, it is then possible to search for the largest unoccluded rectangular area that is suitable for presenting information.
3.1.4 Summary
Table 3.1summarizes the examination of the related work discussed above. In this section, we analyzed related work in three different areas. We first revisited previous work that dealt with occlusion in 2D digital environments. Although not directly related to the challenges created by physical occlusion, these studies motivated us to use a proxy-like representation of occluded items as a means to aid ease of access. We then review prior studies that addressed challenges raised by physical body occlusion on touch- and stylus-based displays. It was found that an ample amount of display gets occluded while interacting with such displays. We moreover learned that the callout visualization is a promising way to visualize occluded content. Finally, we discussed previous works addressing problems of physical occlusion. We learned that techniques that consume less space and introduce less visual clutter are more promising for general tabletop applications. Our analysis of the literature in this field also revealed that occlusion management techniques should limit distractions
clusion management system should support interaction with and organization of digital objects.
Previous studies have not been examined in true hybrid settings with everyday objects.
None of the previous studies compared their system with the natural way of coping with occlusion (i.e., moving or lifting physical occluders).
Prior occlusion avoidance systems do not allow for intentional occlusion.
Table 3.1: Summary of the state-of-the-art analysis. s indicate important findings from the literature analysis that we considered in our interaction design, ands are lacking features that are addressed with our contributions in this chapter
caused by the system.
The present chapter significantly extends the previous work in three distinct ways. First, our contributions present an integrated interactive solution and tech- niques for hybrid tabletop settings that seamlessly supports awareness, access, and interaction with virtual objects as well as their organization. Second, we study our interaction concepts in a true hybrid setting using everyday physical objects. This enables assessing the influence of physical properties of various physical objects on the strategies users employ to cope with occlusion. Third, our evaluation compares the techniques with the natural way of coping with occlusion of everyday physical objects: moving or lifting the occluder. To our knowledge, this is the first such study. Furthermore, in contrast to the occlusion-avoidance approaches, we allow for intentionally occluding digital objects and at the same time provide occlusion-aware support.