SERVICIO AL CIUDADANO EN EL DISTRITO CAPITAL

There are three possibilities to track the mobile device: first, the environment tracks the exact location of all displays. Second, the mobile device knows its own position and communicates this information to all external displays in the environment. And third, the respective target display identifies a mobile device once it is placed upon its surface. In our prototypes, we decided to use the third approach to demonstrate the effects of directly superimposing a mobile display on a large screen. Having a mobile device tracked and identified by the respective external display implies the target display from the user’s point of view. In other words, the system only assumes a tight connection between two screens when a mobile display is directly superimposed on another screen. Such tight couplings exist if the personal device is placed directly on top of the secondary display. In contrast toUbiquitous Graphics, our prototypes employ an interactive tabletop [SH06]. This large display allows users to place the mobile device on top of if while avoiding the need of permanently holding the device.

Figure 3.6:Devices used in our prototype: (a) Our external display is a multi-touch tabletop

based on FTIR [Han05]. (b) The mobile device is embedded into a frame which contains infrared five LEDs.

Our tabletop system uses the principle of frustrated total internal reflection (FTIR) which was first introduced by Han [Han05]. Figure 3.6 shows our prototype devices. This method relies on infrared light traveling inside acrylic glass. The light is reflected totally within the panel when it is beyond a critical angle between its movement vector and the panel’s surface. Another (soft) material touching the surface frustrates this total internal reflection letting light escape the panel orthogonally. This emitted light can then be captured using a camera that operates in the infrared spectrum by for example using an infrared filter. The back-projected nature of the origi- nal idea can be avoided through thin form factors while still relying on infrared light [HIB+07]. SMART Technologies present a different approach which uses infrared LEDs in the display’s bezels [SMA10]. Cameras in the corners detect but not identify objects (i.e., missing infrared light at certain angles) and calculate their position using angulation similar to systems using visi- ble light [BHB07]. The aforementioned systems either still have a low resolution (i.e., thin form factors) or cannot identify objects. Furthermore, larger objects cause shadows in which other contact points cannot be detected when the system uses LEDs in the display’s bezels. In our

prototype, we chose to use FTIR for our tracking. Similar to recent approaches, we ensure that the tracking is not confused by sources emitting infrared light (e.g., lamps or the sun) by using a filter on the camera. This filter can only be passed by light with wavelengths close to the ones emitted by the LEDs attached to our surface (i.e.,±5 nm).

Figure 3.7: Different LED placements for mobile devices: Two possible patterns for LEDs

on a mobile device. Right shows a sample frame from our tabletop with both a tablet PC as well as a hand being placed on it. In this scenario, pattern B can be detected.

As mentioned before, FTIR tracks soft objects touching the surface. However, the tablet PC we have used in our prototypes cannot be detected without modifications. Instead of detecting a contact point on the acrylic, the panel may be covered with a thin flexible foil which also serves as projection canvas. Contact points occur on the surface when the foil is being pressed down [HKI08]. While contact points from solid objects are now being detected as well, another problem arises. According to the laws of physics, one plane superimposed on another (i.e., the tablet PC on the tabletop) ideally results in three contact points. These points also carry the entire weight of the upper plane. Even if four points could be detected (e.g., the four corner points), determining the rotation of the mobile device within the tabletop’s plane is ambiguous. In general, the three points would be sufficient for tracking the mobile device’s position as well as its orientation. The positions of these points are not known a priori making it hard to determine the spatial relationship between the mobile device and the tabletop. Nevertheless, three fixed (i.e., known a priori) contact points placed in a linearly independent way would allow for identifying the mobile device. To ensure the tablet PC’s visibility in the camera image, we decided to use a frame for the mobile device with built-in infrared LEDs. These LEDs are placed uniquely to allow for the identification of multiple devices at the same time. For stability reasons, we decided to use five LEDs which are placed as follows: first, four of them are located in the corners of the frame and thus serve as reference frame of the device. The fifth LED is placed at a predefined position on the frame’s border. For each device, the last one is placed differently. Figure 3.7 denotes two placements of the LEDs for two individual devices.

Besides the points introduced by the frame, other touch points (i.e., fingers) may also occur on the tabletop simultaneously. The input needs to be processed in order to detect a device’s pattern in the overall set of points. In our prototypes, we realized in multiple steps: first, the system takes two points out of the set that have the same length as the tablet’s diagonal. Second, the system tries to match other points according to the locations of other LEDs in the frame using a small threshold. And third, when all five points have been found, the algorithm refines the detected

3.3 A Proof-of-Concept Prototype 57

shape according to the detected points. By doing so, we obtain the tablet’s position on the tabletop as well as its rotation (i.e., the angle of the tablet’s coordinate system with respect to the tabletop’s coordinate system). These values are then used to construct an invertible transformation matrix between both displays. This allows for geometrical transformations of the content according to the display it is shown. Similarly, each input point coming from the tablet PC can be transformed into the tabletop’s coordinate system using this matrix and its inverse respectively. Running this algorithm for all known LED configurations, the system is also able to identify the tablet in case multiple ones are present. This method fails if a pattern similar to the tablet’s LED configuration is present on the tabletop since fingers and LEDs cannot be distinguished. One solution is to use a blinking pattern with a frequency higher than users could constantly tap on the tabletop.