On the Web, tagging resources is nowadays very common. Personal tagging is sup- ported by browser bookmarks. Collaborative bookmarking tools such as del.icio.us1,
3.3 Pen-and-Paper Applications 77
digg2or cite-u-like3enable sharing own bookmarks with other users. Many Web 2.0 platforms contain their own tagging functionality, e.g. for photos (flickr4) or videos (YouTube5). However, these approaches are restricted to resources that are available on the Web. Moreover tags apply always to entire documents and it is not possible to tag sub-passages of them.
Before we review in more detail systems for tagging paper documents, we will have a look at digital notetaking and annotation systems that inspired our work. Some collaborative forum and annotation systems (e.g. [LTZ05]) enable the user to categorize his or her contributions following their functions in the discourse. This relies on the concept of speech acts [Aus62, Sea69, Win86] and is called “scripting” [Kin07]. This classification of individual annotations or individual forum posts in- spired our approach for tagging individual annotations with semantic categories.
We now review approaches for tagging contents on paper documents. Only lit- tle work has explored this issue. Some of the commercial software packages that are shipped with Anoto pens (e.g. Logitech io2 software [Log], Oxford Easybook M3 Digital Notepad [Oxf]) allow the user to mark up the notes in order to control fur- ther electronic processing steps already while taking notes on paper. This comprises typical office tasks, such as adding a new task to Microsoft Outlook, inserting a note into a new document in Microsoft Word or sending a note as an e-mail to another person. In order to invoke the electronic command, the user writes a specific char- acter or symbol in the margin of the notebook and selects the operand by drawing a vertical line beneath the appropriate note(s). However, this activity does not create tags in a sense that the notes are categorized for later retrieval.
Semantic tagging in a more precise sense is supported by PapierCraft [LGHH08], which has been already introduced in the previous section. It is a gesture-based command system that offers two methods for tagging contents on printed PDF doc- uments. On the one hand, the user can tag contents using predefined categories. This is done with a specific pen gesture for predefined categories (Fig. 3.18 upper row). The cardinal direction of the ending of the pigtail gesture decides upon the category. The current prototype supports two predefined categories. This approach guarantees readability on paper and a reliable gesture recognition. However, only a small number of tags can be supported (up to the eight cardinal and secondary directions). Moreover, the gestures are abstract, since they have no intuitive connec- tion to the category. The user must memorize the gestures and the corresponding keywords. On the other hand, in addition to predefined categories, the user can tag contents with freely-chosen keywords. In order to so, she performs a basic tagging gesture and adds a handwritten label (Fig. 3.18 lower row). In contrast to our work, the current prototype of PapierCraft requires an additional device, for instance a push button, for switching between a writing and a command mode.
In another publication, Liao et al. [LGA+07] discuss several ways for tagging in-
2http://digg.com
3http://www.citeulike.org 4http://www.flickr.com 5http://www.youtube.com
Figure 3.18: PapierCraft’s pen gestures for tagging document passages with predefined or freely-chosen keywords [LGHH08].
dividual notes with pen-based interactions. Spatial differentiation consists of pro- viding separate areas on a document for different categories. The note is tagged with the category of the area it is written in. While this is a simple, quick and in- tuitive means, it forces the user to follow a predefined spatial arrangement for his or her notes and moreover supports only a rather small number of types. A sec- ond method, pen differentiation, means that the user switches between different pens whereby each pen is associated with a specific category. All notes written with that pen are tagged with this category. This does not interfere with the spatial layout of notes but switching between pens takes more time than spatial differenti- ation and moreover this requires additional hardware. In addition, research shows that students rather use one single pen than switching among many marking tools [Mar97]. A third solution consists of classifying notes by performing specific pen gestures, like the ones used in PapierCraft. This provides for a possibly large set of categories. However, current digital pens cannot recognize gestures by themselves. Therefore, gesture recognition is performed by the back-end system. In a mobile set- ting, where pen data is temporarily buffered on the pen, this has the disadvantage that the user cannot get real-time feedback on success or failure of gesture recog- nition. Novel generations of digital pens which include gesture recognition could solve this problem in the future.
Finally, Quickies [MM08] proposes using physical stickers for taking notes on pa- per, which are automatically digitized. Stickers can also be used to tag physical objects with a handwritten label. If the sticker is attached to the object, this can be retrieved in the physical space, as an RFID tag is applied to the reverse side of each sticker. While this approach is intuitive and powerful for tagging entire objects, it is not possible to tag sub-passages of documents.