Informe del Experto independiente sobre la cuestión de las obligaciones de derechos humanos relacionadas con el

The NASA TLX (Hart and Staveland, 1988) is a cognitive workload rating scale derived from the NASA Biploar-Rating Scale in which self-report scores are collected from participants in relation to six bi-polar subscales assumed to contribute to the total perceived workload experience. The NASA TLX is founded on the idea that cognitive workload is a hypothetical construct that represents the cost that must be incurred by the operator in order to maintain an acceptable level of performance. According to Hart and Staveland (1988) workload is the product of the interaction between the requirements of the task, the circumstances under which the task must be performed and the skills, behaviours and perceptions of the human operator. They argue that its multidimensional nature requires a broader assessment tool that is implied by the MHS scale. The workload sub-scales proposed by Hart and Staveland (1988) and their associated definitions are shown in Figure 17.

Figure 17 NASA-TLX rating scales definitions

The six subscales shown in Figure 17 can be grouped according to the three factors assumed to produce workload. The mental, physical and temporal demand sub-scales are properties believed associated with the task. The performance and effort sub-scales are assumed to be characteristics of behaviour and skill. Finally, the frustration is assumed to be a characteristic of the individual. Each subscale is presented to the rater using a 20 point scale with values running from 0-100 along the scale. The extremes of each scale (the endpoints) have verbal descriptors.

The assessment process involves two steps. In the first phase, raters are asked to score the task relative to each subscale by placing a mark at some point along the 20 point scale. If the rater places a mark within one of the boxes then that mark is awarded. If the subject places a mark against the division indicator between two boxes the higher mark is allocated to the task. Ratings on each scale are multiplied by 5 to give each scale a value in the range 0 to 100. In the second phase, raters are required to make paired comparisons using each of the possible 15 pairwise contrasts between the sub-scales. In these comparisons, raters are asked to identify which of the two factors is the more important with regard to workload in the task being evaluated. A rank order of the sub-scales, from 0 to 5 is derived from the results, by which the individual subscale scores of the rated task are weighted. By summing the weighted scores and then dividing them by the sum of the weights, a Mean Weighted Workload Score is obtained, that indicates the level of workload expressed as a percentage (see Figure 18). If two or more qualitatively different tasks are rated with regard to workload, a separate paired comparison has to be made for each task (e.g., NASA, 1986). The NASA TLX is a more sophisticated tool than the MCH and requires both the analyst and task rater to be familiar with the assessment method. It has been suggested that with sufficient training the subject needs approximately one minute to complete the event storing and around three minutes to complete the paired comparisons. If these assessments are accurate then the method should score highly on the implementation assessment criteria (e.g., O’Donnell and Eggemeier, 1986).

Figure 18 NASA-TLX scoring example (underlined scale rated most important) Despite the widespread popularity of the NASA TLX rating scale, the tool has been criticised on a number of occasions. In a study carried out by Pfendler

Score for development an objective assessment of cognitive workload. Replications of these results have questioned whether the paired comparison element of the assessment method is required. A similar conclusion was reached in an evaluation of the method performed by Nygren (1991). He described the weighting procedure as ineffective and recommended that it be ignored when using the NASA TLX scale in applied situations (as opposed to situations involving laboratory-based assessments of cognitive workload). Further practical considerations associated with use of the scale have been discussed on several occasions (e.g., Beevis, 1992; Hancock et al, 1989; Hart and Wickens, 1990) and investigators have been both supportive (e.g., Hill et al, 1992) and critical (e.g., Veltman and Widdel, 1993). On the positive side, it is suggested that the method has good face validity and assignments are well accepted by the user community. Ratings can be obtained quickly from subjects and application of the methods tends not to intrude into the normal work activities of the raters. . Finally, the method is generally thought to be superior to MCH and other global rating scale methods due to its multidimensional character, which is thought better able to capture the complexity of cognitive workload. On the negative side, doubts about the technique have been raised from a methodological standpoint. It has been suggested that the validity of certain sub-scales used by the method is questionable or irrelevant. Veltman and Galliard (1993), for example, reported that in many experimental assessments, the sub-scale “frustration” and “physical demand” only show minimal influence on total workload. They floated the idea that these constructs might be dropped from the assessment method. Pfendler (1993), on the other hand, took issue with the validity of the workload computation. He suggested that the TLX scale shows lower sensitivity (i.e., ability to respond to variations in workload) than is the case for other scales such as the Sequential Judgement Scale (SJS) or the Dutch ‘Effort’ Scale (BSMI). Less problematic, but still relevant, various authors have raised doubts about the practicality of the TLX. Because of the relatively large number of dimensions being assessed, the TLX scale is unsuitable for real-time assessments obtained concurrently during task performance in a field situation. The instructions for use were found to be too extensive – at least in the earlier versions of the scale – and contained technical terms not immediately comprehensible to all subjects. As the technique is based on the use of expert judgments, it is necessary that the user has direct recent experience of the task being assessed. This means that the method has limited applicability for use in an undeveloped system at the concept stage of design.