11.4 MATRIMONIO EN EL IUSNATURALISMO

If a metric was capable of at least discriminating the multitasking state from just driving, it was of interest to know whether the metric could be used to make additional discriminations between the set of higher-workload and lower-workload tasks. This would be done separately for visual-manual, auditory-vocal, and Just Drive tasks because of their substantially different nature. This was a second level of discrimination of high practical importance to product development efforts. Level 2 discrimination would be critical to distinguish alternative designs for a function or feature. Level 1 discrimination would also be important to comparisons with a standard other than Just Drive.

It was recognized that Level 2 might be a very difficult level of discrimination when the higher and lower workload tasks within a set or relative workload category (based on prior prediction as explained in Chapter 2) were very similar. Level 2 might also be a very easy level of discrimination when the higher and lower workload tasks within a set or relative workload category were very different. However, when this level of discrimination can be achieved, then the metric has some degree of precision to identify those tasks that are more likely to cause intrusion on driving performance, and less likely to falsely identify tasks that do not significantly intrude on driving. How much precision a metric possesses in making discriminations depends largely on the gradient of difference that exists within the task set undergoing evaluation between lower and higher workload on each metric. A metric's discrimination or sensitivity also depends on measurement error, i.e., a measure’s susceptibility to error variance or measurement variation due to unknown or extraneous causes.

It had been hoped that the surrogate metrics selected for the DWM toolkit would achieve the outcome of being able to distinguish lower- and higher-workload tasks within a set. However, it was not possible in advance to guarantee an even spread of tasks along a gradient from low to high workload within each set of tasks (visual-manual and auditory- vocal), and on every metric. It could also be limited by the sensitivity of a given measure to task differences rather than the selection of tasks per se. Therefore, the binary classification into higher-workload and lower-workload tasks, based on prior prediction, was used to frame discriminability analyses. An additional hope was to be able to evaluate those surrogate metrics using driving performance measures, including eyeglance measures, taken directly from driving (either on-the-road or, as in this case, from the test track) that also met both Level 1 and Level 2 discriminability criteria.

3.7.1.2 Application of Discriminability Analyses to Driving Performance

Metrics and Surrogates

In the analysis framework set up for this project, discriminability analysis was used twice on the driving performance measures reported in this chapter. It was used first to determine if there was observable intrusion of tasks that could be discriminated from Just Drive on measures of driving performance from the test track that were valid (by virtue of being linked to measures of real world driving performance), and repeatable (based on split-half repeatability analyses) that could be discriminated from just driving. Second, discriminability analysis was used to determine whether those measures that were valid and repeatable could discriminate higher- from lower- workload tasks (within each major type of task, visual-manual and auditory-vocal).

It would seem that in this two-tiered application of discriminability analysis, Level 1 (discrimination of concurrent performance of secondary tasks while driving from driving performance alone) is necessary, and, in some special cases, sufficient. But Level 2 discrimination of higher-workload from lower-workload tasks is of great practical value in product development. Ideally, measurement discrimination or sensitivity would produce

Chapter 3 Test Track Results

interpretable and statistically significant differences between sets of higher-workload tasks versus lower-workload tasks. Note, however, that low-workload and high-workload tasks may look similar on a metric that meets only Level 1 discriminability. Yet it may be the case that some metrics are important enough, due to the role they play in driving or in traffic situations, that discrimination from Just Drive is of interest or value even though low-workload tasks cannot be discriminated from high-workload tasks. This may occur when all tasks of a certain type similarly affect a metric, for example, an event detection metric. Alternatively, it may be that the gradient of difference between higher- and lower-workload tasks on a metric is sometimes too narrow to permit a discrimination between very tiny differences in workload (as ascertained by other measures or methods), and thus does not justify requiring a Level 2 outcome. These special conditions might warrant acceptance of a Level 1 discriminability outcome. Thus, while Level 1 may be adequate for some special metrics or purposes, Level 2 discriminability is desirable and should confer on metrics a special status as “exceptionally good” whenever it is achieved.

3.7.2 Alignment of Metric Interpretation with One-tailed Statistical Tests

To apply discriminability analysis as it has been defined on the DWM project, it is necessary to test directionally for the predicted outcome. This is necessary because the higher-workload and lower-workload prior predictions produce binary task categories that are ordinal and not simply labels (Nunnally, 1978). As indicated elsewhere, discriminability analysis was performed separately for visual-manual tasks and for auditory-vocal tasks and Just Drive combined. Mixed- mode tasks were not assessed because less is known about their properties and the performance impact of the interactive voice response systems used with both mixed-mode tasks.

Directional hypotheses can take two forms. For example, as workload increases for a visual- manual task, the mean number of glances to task-related areas would also be expected to increase. This is referred to as the “more (i.e., greater magnitude) is more (workload)” prediction alignment (meaning “as the metric scale measured more of the underlying factor, more workload was associated with it.”). For other measures, as workload increases for a visual-manual task, the measure is expected to decrease. This is referred to as a “less (i.e., lesser magnitude) is more (workload)” prediction alignment (meaning “as the metric scale measured less of the underlying factor, more workload was associated with it.”). For example, as workload increased for a visual- manual task, the mean eyes-on-road time would be expected to decrease, all else being equal). In all cases, the prediction alignments in the other direction (“more is less”) were also tested for measures whose content validity justified it.

3.7.2.1 Level 1 Discriminability Analysis

For Level 1 analyses, each task which was done concurrently while driving was tested against the Just Drive task. Thus, the Just Drive task was put on the low-workload side of the matrix, and all other tasks were put on the high-workload side of the matrix. However, low-workload tasks were summarized separately from high-workload tasks. A sample matrix is shown in Figure 3-54.

Chapter 3 Test Track Results

3-80