CONVIVENCIA ESCOLAR:

Three types of interaction tasks are specified on such 3D tactile touchscreens with a view to ac- complishing a set of goals via tactile feedback:

Identification of objects: The operator is able to identify objects and the non-objects area via tactile feedback.

Selection of a single object from a group of objects: The operator is able to identify objects and the non-objects area via tactile feedback.

Manipulation of a selected object: The operator activates, repositions and reorients the selected object and gets a matched tactile response.

To enable these interaction tasks to be accomplished, a set of interaction techniques were devel- oped to represent the performance of user actions: touching, moving, short-pressing, long- pressing, moving onto, moving off, releasing and rotating. These interaction techniques were applied in the following detailed tasks.

8.1.2.1 Button clicking

To recognize a tactile object, it is necessary to determine the location and orientation of the ob- ject’s edges and surfaces. Accordingly, a number of interaction techniques were designed to characterize more mechanical-like behavior of a feel of button clicking. Considering the need for users in a distracting environment to feel their way to the required control before actuating it, the lift-off pointing strategy was employed as a basis for real-time feedback in this case study. The findings in Chapter 6 show that an impulse when the finger moves onto a button gives the impression that an object has an edge in a 2D tactile touchscreen, and that the finger is entering the area of a button, no matter how far the buttons are from each other. In comparison to a 2D touchscreen, a 3D force-sensitive touchscreen provides additional tactile information in the ver- tical direction. Thus, tactile data was provided in three stages in a 3D touchscreen to enable users to determine the location of a button and manipulate it. Firstly, threshold-triggered tactile feed- back was applied to facilitate detection of the button edge. Secondly, a constant vibration was provided to allow the surface area of the button to be tracked. In the third stage, pressure force and brief tactile feedback were combined to give the impression of the button clicking, the aim of this design being to simulate the feel of a real button clicking when pressed and released. There was no vibration when the finger was outside a button’s touch area. When the finger moved onto a button, a short, high-amplitude vibration occurred. When the finger touched the surface of a button but without actually pressing it, a light, constant vibration occurred. This kind of haptic location information added to buttons can improve the memorability of button place- ment.

(a)

(b)

(c)

(e)

(f)

Figure 58: Tactile simulation of button events

Based on these results obtained by investigating haptically enhanced basic UI controls in Chapter 6, the tactile impulses which are clearly sensed were selected for this experiment. Therefore, the most sensitive impulses, namely those with a combination of a frequency of 58 Hz and a de- creasing amplitude, were selected for recognizing the edge of a button. An increasing impulse as the second most sensitive tactile effects was used to differentiate from the decreasing impulse. Considering that the impulse duration when the finger moves onto or let go of the button has to be short but sensed clearly, the second most sensitive duration of 50 ms and not 150 ms was cho- sen to give the impression of the edge of a button. In addition, a constant vibration, a relatively strong signal, was used to simulate the button when the finger stays on it. Thus, four touch events were generated: 1) an impulse with a 58 Hz, 50 ms sine wave of decreasing amplitude was applied when the finger went onto the button, 2) a constant vibration with a 80 Hz, 200 ms sine wave of increasing amplitude was used as long as the finger stayed on the button, 3) an im- pulse with a 58 Hz, 50 ms sine wave of increasing amplitude was triggered by pressing the but- ton, and 4) an impulse with a 58 Hz, 50 ms sine wave of increasing amplitude was delivered on release of the button. The stimulus duration of 50 ms was selected in view of the fact that the touch events occurring when the finger moves onto, lets go of and presses a button are very short actions. As shown in Figure 58, these four events were combined into five different interactions making up the clicking of a button. Therein, Figure 58 (a) describes a variation when pushing a mechanical button.

i) event 1) + event 3) (move onto  press), see Figure 58 (b),

ii) event 1) + event 2) + event 3) + event 4) (move onto  touch  press  release), see Figure 58 (c),

iv) event 1) + event 2) + event 3) (move onto  touch  press), see Figure 58 (e)and v) event 2) + event 3) + event 4) (touch  press  release), see Figure 58 (f).

8.1.2.2 Zooming images

Tactile perception can be utilized to provide additional UI control functions by non-visual means. This makes it possible not only to reduce the size of UI controls but also to present more information on the display. In this way, a touchscreen can be used as a remote control to manipu- late an image shown on a large display.

Press-hold (or long-press), the commonly used touch gesture for one-finger zooming, as a tech- nique which provides self-explanatory and natural touch interaction, was used to zoom on-screen objects in this experiment. The tactile touchscreen system offered two different strategies for zooming an image to view it in more or less detail. 1) Firstly, when the finger pressed a button for more than 300 ms, the image on the other display was enlarged bidirectionally through three magnification levels of 125%, 150% and 200% of the original size. If the finger continued press- ing the same button on the touchscreen, the resized image would be zoomed back out to an over- view, either directly or through the magnification levels of 150% and 125% of the original size. The image could be stopped at any one of the intermediate sizes by lifting the finger off the but- ton. The resized object presented in the zoomed image was modified so as to fit the relevant in- formation into the current image size. 2) The second strategy allowed an image to be zoomed in continuously without going through distinct magnification levels, until the maximum of 200% of the original size was reached. If the finger continued pressing the same button on the touch- screen, the resized image would be zoomed back out to an overview, either directly or continu- ously without distinct magnification levels. The image could be stopped at any intermediate size by lifting the finger off the button.

8.1.2.3 Navigating objects

A navigation interaction technique was developed to process and control movement of the image shown on the large display. When the finger pressed a button on the touchscreen and moved, the object depicted on the large display moved a relative distance. In order to simplify the interac- tions, zooming and navigating could be integrated into one function: when the finger started to move from any part of the button toward one of its four corners, the image on the large display was zoomed in to show the corresponding quarter of the image. For instance, as shown in Figure 59, when the finger started moving from its current position toward the top right corner of the button control, while pressing and subsequently releasing the button, the top right quarter of the image was zoomed in and displayed in full on the large screen.

The zoom and navigation interactions could also be activated one after another. For instance, when the finger presses and moves but not out of a button, the resized image would be moved in

the large display, followed by zooming an image. The zoom and navigation functions were de- signed to be combined in five different ways: 1) zooming in continuously and navigating, 2) zooming in and out continuously and navigating 3) zooming in through distinct magnification levels and navigating, 4) zooming in and out through distinct magnification levels and navigat- ing, and 5) zooming in in four directions.

Figure 59: Zooming an image in through movement on the tactile touchscreen [@32]