Forma Musical

By analogy to the physical world,toolsare here considered as those interfaces which are used for editing and transforming an object, and in this specific context for editing a digital object. As interactive surfaces have become progressively available, they have enabled the spatial merging of the physical input tool with the digital output in the same area in which the tool is applied. This is analogous to what actually happens in the physical world.

The examples referenced in this section conceive of the use of physical tools for editing digital information which is spatially mapped directly to the physical object. The resulting changes can affect the overall visual or audio system display (thus using tools as handles for manipulation), as well as a portion of it, in proximity of the point in which the tool is applied.

Fitzmaurice et al.’s (1995) mocked-up prototype of the GraspDraw ap- plication on the Active Desk is one of the first examples illustrating the manipulation of graphical shapes with physical handles (see Fig. 3.17, a). As sensing technologies have become more mature, such a concept has been applied in other contexts. The Urban Planner Workbench (URP) is an ap- plication which aims at supporting urban planning (Underkoffler and Ishii, 1999). It relies on a vision technique for tracking the position and orientation of physical objects on a unique pattern of colored dots on a tabletop (see Fig. 3.17, b). Users can reorient and move the physical models of buildings, so as to visualize casted shadows with a front projection system. Furthermore, they can use tools for changing system status, such as the hour on a clock tool, so as to arbitrarily change the light condition and preview its shad-

3 Related Work

Figure 3.17: Examples of objects as handles for manipulation. a) Bricks for changing graphic shapes on the Active Desk (Fitzmaurice et al., 1995). b) Tools for changing buildings position, light effects and measures on the URP system (Underkoffler and Ishii, 1999). c) Pucks and knobs for changing system as well as local conditions with the Sensetable (Patten et al., 2001). c) Pucks for the creation and modification of electronic sound on the reacTable (Jord `a et al., 2007).

owing effects. Additionally, they can use measuring tools, which track and dynamically display the distance between two buildings.

Patten et al.’s (2001) Sensetable tracks the position of wireless pucks elec- tromagnetically. The movement of the pucks on the touch-sensitive surface (consisting of two Wacom7 _{tablets placed next to each-other) dynamically} edits the graphical representation which is front projected on the tabletop. Like handles for manipulation, the physical tools can be bound to the graph- ical representation to scratch it, move it, and edit it. Furthermore, a dial is mounted in each puck so that users can as well change the state of the puck, thus applying local changes. This concept has been integrated in a diversity of domains by the authors, such as chemistry and system dynamics, business supply chain management, urban planning, interactive visual art, as well as performance and composition of electronic music. This last domain has been explored in the Audiopad project (Patten et al., 2002). This relies on an infrastructure similar to the Sensetable, although it integrates a matrix of antenna elements which track the positions of electronically tagged objects on the tabletop. As the user interacts with the tools, s/he can physically ma- nipulate the sound as well as the visual display (see Fig. 3.17, c). Each object represents either a musical track or a microphone, and they have different shapes to afford different functionalities.

Similar to that, Jord`a et al. (2007) create a semantics of objects/tools for the reacTable. The objects, consisting of plastic pucks in different shapes, have a marker attached on the side which is in contact with the table sur- face. The marker is tracked with a camera-based system mounted under the table together with a beamer for back projection of the visual display. Each

3.2. Interactive Objects

puck represents a modular synthesizer component with a specific function for the generation, modification, or control of electronic sound. By changing the number, position, spatial relationship, and local rotation of the pucks, an unlimited number of users can casually interact with the system and dy- namically edit the sound output. An interesting aspect of this system is that fingers can also be used for interacting and locally altering the function of single pucks. This is made possible by simple, small paper markers that users can stick on their finger tips, thus augmenting the interaction vocabulary of bare fingers.

DataTiles (Rekimoto et al., 2001) and TViews (Mazalek et al., 2006) are other examples in which the use of tools has an effect on the local area in which the tool is applied, rather than on the overall system display. DataTiles are physical transparent tiles which a user can slide on a grid of rails over a screen. Similar to interactive filters, these tiles allow for different visual- izations and manipulations of the information displayed underneath. Users interact with a pen directly on the tile and across multiple tiles (see Fig. 3.18, a).

Figure 3.18: Examples of objects as tools for local modification: a) Interacting with DataTiles for editing the information displayed underneath, or aside a tile (Rekimoto et al., 2001); b) Using physical pucks for moving photos on TViews (Mazalek et al., 2006).

TViews (Mazalek et al., 2006) is a general platform for tangible interaction with digital media on a tabletop for everyday life social environments, e.g. domestic living rooms. In this case, physical objects are the only input device and the sensing technology relies on both ultrasonic and infrared sensors. In PhotoSorting, one of the applications implemented on TViews, physical pucks are used as kind of magnets to “attach” photos and move them around on the table surface (see Fig. 3.18, b).

3 Related Work