This generic gesture has different meanings depending on the areas it is performed upon. This provides for an intuitive and flexible way of defining different link an- chors. Different kinds of association areas are contained on printed and digital doc- uments, on physical books and on physical folders. Each type of area represents another scope. Figure 5.24 provides an overview on these types of association ar- eas. It shows how the physical areas are complemented by digital representations. In the document viewer, the areas are displayed at the same positions on the docu- ments as on paper. In the ecological view, the nodes of the graph that represent the documents act as digital association areas. On these areas, the pen gesture for creat- ing hyperlinks can be performed in the same manner as on paper. We now discuss each type of association area in turn.
• Entire hybrid document: Each hybrid document contains an area on the top of its first page where gestures are made for links that apply to this entire document (Fig. 5.23 left). Since one of our key design aspects is supporting cross-media linking, these association areas are included in each printed document page, in their digital representation in the document viewer and on Web pages in
4Related research [LHGL05] compared several pen-based techniques for switching between modes, one of them being tap-and-hold. The authors found tap-and-hold to be slower and more error- prone than switching mode by varying the pressure or by using additional buttons. We never- theless opted for tap-and-hold for the following reasons. First, pressure-based mode switching is not possible in paper-only environments, as it is important to provide real-time feedback to the user about the current pressure level. Second, other techniques which require additional hardware were not acceptable in our case, as it heavily restricts deployability in real settings. Third, while tap-and-hold requires the user to wait a short moment before making the actual gesture, time is not critical in our case, since not so much links are created. Finally, the main problem of tap-and- hold reported was that users had difficulties in holding the pen at a fix position on the slippery display of a pen-sensitive screen. This was no problem with CoScribe, as paper is not slippery and the screen is large enough to allow some minimal movements. Our design decision was confirmed in the evaluation. Users had no difficulties in creating hyperlinks and reported to appreciate the clear distinction between a normal mode and the gesture for creating a hyperlink.
Physical
Digital
Ecology View Doc. browser
Documen
t sc
ope
Collection En tir e h ybrid documen t En tir e ph ysic al- only documen t Sub- passag eFigure 5.24: Types of association areas
Mozilla Firefox. CoScribe therefore includes a Firefox extension which man- ages and displays areas and pen-based hyperlinks (see Section 6.1.2 on imple- mentation details). Another digital association area is the thumbnail of the document in the ecology visualization (see Section 5.2.5).
• Entire physical-only document: In order to support interactions with Anoto pens on physical-only documents (books, journals etc.), which do not contain the Anoto pattern, users can attach a small sticker covered with the pattern onto the document cover (Fig. 5.25) and register this sticker with the document’s barcode. Depending on the pen technology, the user can directly scan the barcode with the digital pen or she manually enters the barcode. The digital metadata of this document (including an image of the cover, if available) is then automatically retrieved from a database. (Our implementation uses the Amazon.com web service.)
• Sub-document level: A margin area contained on each page of hybrid docu- ments provides for creating links from or to sub-passages within a document. It is included both in printed documents and in the CoScribe viewer for digi- tal documents. The user can define the precise extent of the linked document passage. She therefore draws a vertical line besides the passage where the as- sociation gesture is made. Such scope markings are deleted the same way as association gestures by a zig-zag gesture over the marking. If the document contains several columns, each column has an own association area. A scope may span several columns and/or pages.
5.4 Linking Documents 157
Figure 5.25: Links from and to physical books or journals are enabled by a small sticker which is covered with Anoto pattern
In a user study on how people make references within traditional paper doc- uments, we found that users often do not explicitly define the scope of a refer- ence. Instead, the scope remains fuzzy and refers to an unspecified passage or item near the reference. Prior systems [LCC07, LGHH08] do not account for that. Our interaction also supports fuzzy scope definitions. These associations are made in the association column without any further specification.
In addition to these formalized gestures that are computer-interpreted, users can make any informal markings for the scope definition, which are displayed like normal handwriting in the electronic viewer (e.g. brackets, lines and ar- rows). While users understand them, the system is agnostic of their meaning. • Document collections: For links from or to a collection of documents (one-to- many or many-to-many), CoScribe includes the concept of folders. Physically placing a document into a folder is a very intuitive traditional interaction for defining a collection of paper documents. Users can then easily reference the entire collection by referring to the folder.
Our slightly formalized interaction design is inspired by this practice. A folder (Fig. 5.26) contains one or several documents and may be the starting or end- ing point of an association gesture, as it contains an association area on its cover. Similarly, the digital representation of a folder in the ecological view acts as an association area.
In order to detect that physical documents are added to or removed from a folder, the interaction technique relies on location tracking of document and folder positions. At locations where no tracking infrastructure is available, the user can alternatively perform a pen gesture to inform the system about an added or removed physical document. The same gesture applies if a user wants to add or remove a digital document.
Figure 5.26: Physical folders provide for defining collections of documents in an intu- itive manner
Summing up, the conceptual activity of creating links combines up to three core interactions of our framework, namely inking, combining and bridging. It moreover illustrates the principle of spatial multiplexing. The generic association gesture has different meanings depending on the area it is performed upon. Finally, gestural multiplexing enables to differentiate between creating associations and defining sub- document scopes.