BLOQUE2: LAINTERPRETACIÓN MUSICAL o Entender la voz como instrumento y

The thinking aloud is an approach, which has traditionally been used as a psychological research method (Ericsson and Simon, 1984; Neilsen, 1993). However, ‘usability king’ Neilson Jacobson has stated this approach is being heavily used by evaluating human computer interfaces (Neilsen, 1993). Furthermore, he describes that conducting a think-aloud experiment with real participants is one of the most fundamental evaluation approaches in software applications testing. The testing process involves participants using the system or prototype to complete a predetermined set of tasks while being observed and verbalised their thoughts and comments. The observations, thoughts and comments are then analysed to address the research question.

There is a large volume of published studies that critically discuss the think-aloud approach (Dix, et al., 2004; Hertzum and Jacobsen, 2001; Neilsen, 1993; Nielsen, Clemmensen and Yssing, 2002; Nørgaard and Hornbaek, 2006; Preece, Rogers and Sharp, 2002; Ramey, et al., 2006). Ramey, et al. (2006) describes that think-aloud is the most important approach in system evaluation, because it discovers more problems than any other existing measure. Perhaps, this is the main reason that think- aloud is more popular as frequently applied techniques among the academic researchers and industrial expertise (Nielsen, Clemmensen and Yssing, 2002).

The most significant benefit of the think-aloud approach is that it enables to design the system from actual users, who are representatives of general population. Participants provide the researcher an understanding of their views of using the system based on verbalising their thoughts. This enables the researcher to recognize participants’ major confusions (Neilsen, 1993). The think-aloud process is used for researchers to gain participants’ behaviour through an examination. This examination analysis can reveal the causes of system problems. In addition, participants’ voice and behaviour provides insight into their effective reactions such as frowns, sighs and scowls which convey dissatisfaction, frustrations or maybe happiness. Therefore, observational data gathered through a think-aloud approach is quite beneficial and powerful to design a system.

The think-aloud method is flexible so that it provides a vital benefit for system evaluators. Some researchers realise that the think-aloud approach can affect the type and amount of data collected. Therefore, they may attempt to manipulate the think- aloud session to make sure the exploration of research areas to match the objectives of the experiment (Ramey, et al., 2006). This is a significant benefit of the approach, when concerning on certain aspects of an interface such as designing a game interface for learning. In addition, this approach can help identify the suspected areas to be problematic or to uncover reasoning for problems.

Despite these advantages of the think-aloud approach there are some bottlenecks, which are worth concerning when evaluating this technique. Ramey, et al. (2006) describe the validity and reliability of the think-aloud approach has been overlooked due to its usefulness in convincing researchers that problems exist. However, it would have been important to concern the lack of soundness when investigating the effectiveness of this approach.

Dix, et al. (2004) stated that the usefulness of the think-aloud approach is influenced by the effectiveness of observation and subsequent analysis. This depends on researchers who facilitate to conduct think-aloud sessions. As suggested by Preece, Rogers and Sharp (2002), the system evaluation though users is strongly controlled by the researcher, who is in charge of the test. On the one hand, this is an advantage for practitioners to ensure goals of the systems evaluation. However, on the other hand, the think-aloud approach does not provide necessary guidelines for researchers to perform reliable analysis (Hertzum and Jacobsen, 2001). This is due to a combination of unclear goal analysis, unclear evaluation procedures and unclear problem criteria. In addition, Norgaard and Hornbaek (2006) argued that improper setting up of think- aloud sessions and incomplete analysis of results could generate research findings, which may not represent the actual user experience. These issues can enhance the possibility of inconsistency in research findings from different researchers.

The interruptive role of the researcher may cause for inconsistency in the think-aloud approach, which leads to the lack of participants’ concentration (Preece, Rogers and Sharp, 2002 and Ramey, et al., 2006). Perhaps the researcher forces participants to spend more or less time on a certain aspect of system evaluation, which could mislead

to represent participants’ natural use of advices. Norgaard and Hornbaek (2006) stated that the researcher vary the way they conduct the think aloud sessions because they desire participants to confirm known issues which they may have foreseen or already known problems.

The laboratory environment where think aloud sessions take place is unnatural which may affect participants’ behaviour. Participants interact with the system during the think aloud test while verbalising their comments in a completely unnatural environment whilst being observed. Therefore, they tempt to comment based on their experience while completing tasks in the laboratory environment. This may increase their cognitive load, which impacts their problem solving behaviour (Nielsen, Clemmensen and Yssing, 2002; Preece, Rogers and Sharp, 2002; Ramey, et al., 2006). In addition, participants are not allowed to take phone calls, check emails, talk to their friends, or occupy with any task, which they may regularly do in their normal life (Preece, Rogers and Sharp, 2002). Furthermore, observation in the laboratory environment is also an obstructive approach where participants are usually aware that they are being monitored by the researcher (Dix, et al., 2004).

The results of think aloud sessions are limited to what they truly represent in terms of the user population to test the system. Conducting think aloud experiment with a large sample size is time and resources consuming, therefore, the findings of such evaluation typically reflect on a few users (Hong and Landay, 2001). The literature revealed that ongoing disagreements around the sample population size to conduct a think aloud test. Generally between five to fifteen participants are advised. However, ‘usability king’ Nielson suggests that five participants are enough to find major problems (Neilsen, 1993).

In summary, Dix, et al. (2004) stressed the simplicity of the think-aloud approach. This technique is described as a “quick and dirty” approach gaining a huge number of results quickly via using a simple methodology (Preece, Rogers and Sharp, 2002; Ramey, et al., 2006). Nielson (1993) stated that the wealth of qualitative data which may collect from a relatively small number of participants taking part. Although, qualitative data is known valuable, this research study compares findings gained from

thesis. In addition, the idea of strengthening findings of this thesis by combining finding from methods which independently collect both quantitative (questionnaire) and qualitative (think-aloud) data will be reported.