– MIT Global Risk Survey

Tangible interaction systems include mechanisms that exchange data between the system components and other mechanisms that encode both the identity and position of tangible objects. A range of technologies support these mechanisms. Figure 3-9 illustrates my stack and system perspectives on the role of supportive technologies within a three-component tangible interaction system. The data exchange mechanisms, and identity and position encoding mechanisms (as shown in Figure 3-9) are discussed below.

Tangible interaction system

Figure 3-9 My stack and system perspectives on the relationships that exist in tangible interaction systems

I classify mechanisms by which system component data are exchanged as tethered and untethered and I refer to the respective tangible systems that incorporate them as tethered and untethered tangible systems. Tethered tangible systems are characterised by wires or direct physical contact to interconnect components. Examples of tethered tangible systems are Horn & Jacob’s (2007) Tern and Suzuki & Kato’s (1995a) AlgoBlock. Untethered tangible systems incorporate objects that have no wires leading to other objects and neither is it a requirement that an object be in physical contact with other objects. Examples of untethered tangible systems are Bricks (Fitzmaurice et al.

1995) and musicBottles (Ishii 2004) .

In addition to classifying the mechanisms according to how data are exchanged, I also classify mechanisms on whether or not either the identity or the position of an object is encoded. Horn et al.

(2008) proposed passive tangible interface terminology to describe physical objects that neither require a constant source of electricity nor maintain a continuous link to a digital system. Applying their terminology, I classify mechanisms by which data is exchanged between system components as either active or passive. I interpret Horn et al.’s use of the word “passive” as comprising two independent mechanisms. The first is the provision (or absence) of electricity supply to the tangible objects whereas the second considers the provision (or absence) of a link to a digital system. Figure 3-10 illustrates my interpretation of the relationships between the mechanisms.

To conclude, I refer to the respective tangible systems that incorporate active or passive mechanisms as active tangible systems and passive tangible systems. By applying my classification scheme, I can state that an active tangible system relies on embedded electronics to encode either the identity or the position of a tangible object. Examples of active tangible systems include Schiettecatte and Vanderdonckt’s (2008) AudioCubes and Reitsma’s (2011) StoryBeads.

Conversely, a passive tangible system does not rely on embedded electronics to encode either the identity or the position of a tangible object. An example of a passive tangible system is the Marble track music sequencer (Fischer & Lau 2006). These examples do not consider the

mechanisms by which data are exchanged. Date exchange mechanisms as applied to untethered passive and untethered active tangible systems are discussed in the following sections.

TetheredUntethered

Data exchange mechanism

Identity / position encoding mechanism

Active Passive

Tethered tangible systems

Untethered tangible systems Passive

tangible systems

Active tangible systems

Untethered active tangible systems Untethered passive tangible

systems

Figure 3-10 My classification matrix of mechanisms in tangible systems mapped according to data exchange, and identity encoding/position encoding

3.2.3.1 Technologies that support untethered passive tangible systems

I map technologies that simultaneously support untethered data exchange and passive identity/position encoding mechanisms of the bottom-left corner in Figure 3-10. As reported in the literature, magnet and vision based technologies are pervasive in untethered passive tangible systems. Examples of untethered passive tangible systems that include magnet and vision based technologies are described next.

The following illustrate magnet based technologies that support untethered passive tangible systems. My GameBlocks (Smith 2007b) and Dialando (Smith 2010a) systems use static magnetic fields to sense object position and orientation. Mazalek, Davenport and Ishii’s (2002) Tangible Viewpoints applies electromagnetic resonant circuits to detect objects on a horizontal interaction surface while Tangible Viewpoints uses loop antennas inside the surface to determine the position and identity of objects on top. The latter is possible by embedding a coil and capacitor resonator circuit inside each object. Actuated Workbench (Pangaro, Maynes-Aminzade & Ishii 2002; Pangaro 2003) is an interaction system in which a table top contains embedded electromagnets and these move objects in two dimensions under software control. The software also determines when the user disturbs an object. Some systems, such as those developed

by Ishii, Fletcher, Lee, Choo, Berzowska, Wisneski, Cano, Hernandez and Bulthaup (1999), Mazalek, and Lee (2001) and Ishii (2004) combine glass bottles and electromagnetic resonant circuits to detect the presence of bottle stoppers.

Examples of vision-based technologies that support untethered passive tangible systems are the following: Jorda, Kaltenbrunner, Geiger, and Bencina’s (2005) reacTable incorporates Bencina, Kaltenbrunner and Jorda’s (2005) reacTIVision vision software to detect and track optical markers in two dimensions. The system senses the position of objects placed on the table in real time and gives user feedback by means of a visual display projected onto the translucent surface.

Tangible Object Placement Codes(TopCodes) (Horn 2007, 2009) is another vision-based detect-and-track system based on optical markers that generates data similar to that of Kaltenbrunner and Bencina’s (2007) reacTIVision. Underkoffler and Ishii (1998), Underkoffler et al. (1999), and Underkoffler’s (1999b) I/O Bulb integrates image capturing and projection technology into a single object. Using I/O Bulb, objects on a flat surface are sensed and processed with the result then projected onto the surface. Coloured dots on the objects enable tracking. In the Diorama Table (Takahashi & Sasada 2005; Takahashi 2007a), a camera and projector are mounted above the table on which the user positions everyday objects. The visioning system then detects and analyses the objects and projects animated images onto the table. Another vision-based tangible system is Tseng, Bryant and Blikstein’s (2011) Mechanix that consists of magnetised tangible objects, a video camera and projector and a semi-transparent vertical screen with an embedded ferromagnetic mesh. The screen supports rear-projection and serves as a surface onto which magnetised tangibles can be attached. The camera detects optical markers on the tangibles and sends this data to a system for processing. Results are then projected onto the surface.

3.2.3.2 Technologies that support untethered active tangible systems

I map technologies that simultaneously support untethered data exchange mechanisms and active identity/position encoding mechanisms to the bottom-right quadrant in Figure 3-10. Inductive, electromagnetic, optical, and acoustic wave are examples of relevant technologies. Systems that incorporate these technologies are described next.

Topological data is exchanged between tangible cubes by means of infrared light in Schiettecatte and Vanderdonckt’s (2008) AudioCubes. In addition to supporting inter-object communication, optical technologies can also be applied to position sensing. For example, infrared beacons can be used to associate a physical object to the room in which it is located (Want, Fishkin, Gujar & Harrison 1999).

Fernaeus (2007) also applies the electromagnetic spectrum in Patcher by integrating radio frequency identification (RFID) tags with a reader to track the positions of objects on a floor mat. In addition to tracking objects, RFID-based systems can also associate physical objects with data. Want et al. (1999) offer two examples of RFID-based systems that associate physical objects with data.

First, photographs on the sides of a cube contain RFID tags with the addresses of associated web sites. Second, an RFID tag inside a wristwatch triggers the computer to display the wearer’s diary.

Keeping to the electromagnetic spectrum, albeit at a much lower frequency, SenseTable (Patten et al. 2001) incorporates inductive coupling between a sensing surface and the objects placed on the surface. A wire grid below the sensing surface conducts low-frequency alternating current while objects contain resonant circuits. These circuits absorb maximum energy at a pre-set frequency and by varying the frequency of the signal within the wire grid, the identity of the object can be determined. The object coordinates are determined by constantly alternating the row and column in which the current flows.

Systems such as TViews (Mazalek 2005) rely on pulsed acoustic waves that travel along a horizontal surface from predetermined fixed positions. When the waves reach an object the detection circuitry inside the object sends infrared signals to a common receiver. In turn, the receiver applies triangulation techniques to calculate the position of the object on the surface.

I conclude this subsection on supportive technologies with a comment by Horn et al. (2008). They comment that (when compared to an active tangible system) a passive tangible system offers the system designer a wider choice of materials and designs with which to implement a solution.

Additional benefits include improved durability, improved robustness, and reduced cost. In the research reported on in this thesis users partially assume the role of the system designer in that the users are encouraged to create their own tangible objects. Having considered the above, I base my designs in Chapter 6 on untethered passive tangible systems.

This section discussed mechanisms for connecting objects to the computer. Section 3.2.4 considers how gestures can manipulate data and associate objects with data.