Emisión de Facturas

The helping mechanisms should not stop as soon as the user starts to in- teract. Especially for longer gestures it may be necessary to remind the user of how the current gesture should be continued, to tell the user what gesture the system thinks he or she is performing, and in case there are still multiple candidates, to tell the user what gestures might still fit to the current input. In this way, the users can adapt their input during the performance as in the affordances and feedback/-forward mechanisms in surface computing presented in Section2.6.

FUBI has several methods for providing affordances, feedback and feed- forward during the interaction. Again, in the case of continuous interaction such as an avatar control or a freehand cursor control, there is usually no need for additional techniques as feedback is implicitly given by rendering the motion of the controlled object. Exceptions are cases, in which the application should try to prevent false postures during the actual interac- tion, e.g. during a rehabilitation exercise. If parts of the user’s motion are interpreted as a an input event (i.e. discrete full body interaction), more information during the interaction is needed.

For assisting the developer with this task, FUBI can return so-called correction hints along with the boolean recognition result. Correction hints contain additional information on the current state of the recognition as well as hints on how to adapt to conform to the recognizer’s requirements. Therefore, the hints provide both feedback and affordances. A correction hint always determines the main joint involved in the corresponding gesture, but the rest of the information differs depending on the type of recognizer it refers to.

Accordingly, the hint suggests to change the speed, pose, direction, form or number of displayed fingers depending on the inspected basic recognizer. Number of fingers logically refers to a finger count recognizer and reports how many more or less fingers the user should display. Form corresponds to symbolic gestures, i.e. FUBI’s template recognizers. In this case, FUBI provides the current distance measured between the user performance and the gesture template. Speed, pose and direction can either refer to (the

change of) the orientation or the position of a joint which is indicated by a boolean value. They further report how much the corresponding value needs to be changed for each axis or overall (e.g. when using the distance in a joint relation) and whether this means that it has to be increased, decreases or simply changed in general.

For example, a joint relation recognizer requires that the left hand is placed over the shoulder, but the current user has placed it 20 cm lower. In this case, the recognizer reports that the pose of the left hand has to be changed according to its position. The position does not have to be increased or decreased in general, but it has to be changed for 20 cm in direction of the positive y-axis.

In a different example, an angular movement recognizer requires that the head is pitched downwards with at least 15 °/s, but the user is moving the head in the opposite direction with 20 °/s. Here, the recognizer reports that the angular speed needs to be changed. Again, no general increase or decrease, but −35 °/s around the x-axis.

For easier reporting the hints to the users, FUBI can construct text messages out of the correction hint’s values. For the above examples, those would be “Please move left hand more up: 200 mm” and “Please turn head faster down: −35 °/s”. The information of the correction hints is also used for the corrective arrows of the automatically generated gesture symbols (see Figure 6.1c), which do not emphasize the motions performed by the virtual character, but actually visualize what the user has to do next to mimic the displayed gesture.

Combination recognizers report correction hints as well. At first, the hints include the state for which the recognizers failed. This can be the current state that has been interrupted, or the next state which cur- rently cannot be reached as its recognizers are not fulfilled, yet. The rest of the recognition hint is set to the information provided by the first failed recognizer within that state.

In addition to the correction hint, combinations also log the user track- ing data at the entering and leaving of each state. This information can be accessed to further investigate the current recognition progress. Com-

binations also have a separate method, that reports additional information about the current state of the recognition. This includes the index of the current state, the total number of states and whether the state is currently interrupted or in transition to the next state.

Last but not least, combinations support that the gesture designer in- cludes meta information for each of the combination’s states. The meta in- formation of the current state can be requested during runtime. In FUBI’s Unity integration, this is used to link each state to the frame index of the animation used as a gesture symbol for the combination. As soon as the user starts the combination gesture, the symbol exactly displays the part of the animation that has to be performed next.