FASE II: PLANIFICACIÓN DE AUDITORÍA

This section contains the conclusion and discussion of the test results that were gathered throughout the evaluation phase of this project. This conclusion and discussion is divided in three different sections, which represent the three different parts of the evaluation phase. Section 7.3.1 contains the conclusion and discussion for the functional evaluation, section 7.3.2 for the first round of user tests, and section 7.3.3 for the second round of user tests.

7.3.1 Functional Evaluation

The functional evaluation showed that all must have-requirements were met by the prototype of the skiing game. All should have-requirements were met, except for the requirement of multimodal feedback since no haptic feedback was included in the prototype. Finally, the could have-

requirements were only partially met, due to the limited time available for the development of the prototype. The could have-requirements that were not met were the ranking, triangularity in the game, and the game being an asymmetrical game. Based on the functional evaluation it was concluded that the prototype functions well enough for people to use it as a skiing game.

7.3.2 First Round of User Tests

In the first round of user tests, the participants’ perceived cyber sickness symptoms, the participants’

6 4 2 5 0 1 2 3 4 5 6 7 8

This version of the game made me more aware of my mistakes

This version of the game made me more

aware of the things I did right

In this version of the game I felt more

distracted

This version of the game was more clear

to me Sc o re Statements

Feedback

77 intrinsic motivation, the multiplayer aspect of the game, and the learning aspect of the game were investigated for the first version of the skiing game.

With regard to perceived cyber sickness it can be concluded that the augmented reality skiing game does not evoke cyber sickness symptoms. This confirms the theory by Rebenitsch and Owen [32] that inclusion of the real world in an application, which is done in augmented reality, can prevent cyber sickness. However, it must be noted that this was only tested on a small group of seventeen

participants, which means that no statistical significance can be derived from these results. Besides that, it could be the case that the exposure did not last long enough for participants to be affected by cyber sickness symptoms. According to research by Bruck and Watters [30], cyber sickness symptoms were only increased after six to ten minutes of exposure, while the experiment lasted only three minutes.

The intrinsic motivation scores that were obtained from the test were divided into the categories interest/enjoyment, perceived competence, and effort/importance. The categories

interest/enjoyment and effort/importance were rated higher than neutral. Based on these results, it can be concluded that the participants felt interested and enjoyed while playing the skiing game and that the test participants put effort into the skiing game and found it important. The category perceived competence scored lower, only slightly above neutral. However, this is not seen as a negative score since perceived competence should not be too high, as being very competent in the skiing game already might cause boredom for the players. Again, these results were obtained out of a group of seventeen test participants, which means that they are not statistically valid.

With respect to the multiplayer aspect of the game, only eight out of seventeen test participants were presented to the multiplayer version of the game since the connection failed in the other cases. Out of the eight participants who played the multiplayer version of the game, only five noticed the fellow player. The five participants who noticed the fellow player indicated that they liked the fellow player and that its presence motivated them. However, no statistical significance can be derived from these results since they were generated by such a small group.

Finally, the learning aspect of the game was evaluated very positively. The test participants indicated that they think that the skiing game is a useful tool for learning how to ski. The test participants who were moderately experienced at skiing also indicated that playing the game more regularly would help them increase their own skiing skills. The other two groups of test participants, the experienced skiers and the very experienced skiers, said that they did not think they can learn anything from the game. Some of them indicated that this was mainly due to the low pace that the game is being played at and that they might be able to increase their skiing skills by playing the game if the speed of the game would be increased. Again, these results do not have any statistical relevance since they are generated by a group of seventeen test participants.

7.3.3 Second Round of User Tests

The second round of user tests was executed with only four participants, who also participated in the first round of user tests. Therefore the conclusions drawn in this section are not statistically valid. Because the test participants had already participated in the experiment, their judgement about the skiing game could be biased. The goal of the second round of user tests was to investigate if the feedback system of the game had improved.

It can be concluded that the improved version of the game did not cause any cyber sickness

78 given in the improved version of the game made the test participants more aware of the things they did wrong in the game. All four test participants preferred the improved version of the skiing game over the first version.

79