Rather than showing statistical evidence, we used these tools for conducting several qualitative user studies with automotive engineers in order (a) to better understand the potential domain value 3d visualizations can add, (b) to trigger new perspectives about utility as well as usability by letting engineers use the tools—again: what people do often differs from what they say or think—and (c) in doing so to learn about design criteria that are important to take into account for future developments. We did formal think-aloud user studies with three development and four analysis engineers. To better understand the tools potentials, we let our participants conduct a set of loosely defined tasks related to understanding spatial correlations, to communication between mechanical components and electronic information, and some just for exploring each tool’s fea- tures. During and after the participants conducted these tasks, we encouraged discussions about
Usability of the Tools The general feedback by using and discussing PT1 and PT2 with our participants was quite good, however, indicated as we hypothesized a strong affinity to fascination underlined by statements such as“it looks just good”or“that’s cool and fun”. By observing our users conducting the tasks and using the early prototypes, we encountered on the other hand several problems that could be set in correlation to recent findings from literature.
First, we observed our participants having several difficulties with representing messages or sig- nals as animated icons “moving through” the virtual car. While these animations in general were often evaluated to be“good looking”,“self-evident” or“well suited for presentation communi- cation processes in the car”, our participants, however, had problems in deriving sender-receiver correlations especially if there were more than one animated message/signal at a time. Further- more our participants complained that for daily work sequentially following animations signal by signal would be tedious (cf. Ware et al.’s project on visualizing underwater behavior of whales [WAPW06]). Second, we frequently observed problems in reading PT1’s labels of ECUs due to text-distortion, and for messages and signal text boxes additionally due to moving targets (cf., for instance, Grossman et al.’s findings on text readability in 3d [GWB07]). Third, for PT2 we found that our participants in several cases missed mechanical behavior animations such as moving a window which could be either a result of occlusion and/or of inattentional blindness [Mac03]. On the upside, a majority of our participants liked the possibility of filtering the in-car network components in PT1 and we encountered no major problems with navigating the 3d-models in both prototypes. The opinions about the degree of abstraction in represenating the in-car network diverged. While in general all engineers agreed in trying to be realistic in positioning ECUs and bus systems is invaluable, some argued for having just abstracted boxes representing ECUs and orthogonal lines for bus systems due to clarity reasons, others, however, argued for an exact positioning and layout of all hardware and wiring components to get valuable information from the visualization. All participants again agreed that filtering irrelevant mechanical components such as screws is definitively valuable and necessary, as a fully equipped car with all its“nearly 10,000 components [...] makes it impossible to distinguish between relevant and irrelevant infor- mation”. However, which mechanical components are relevant and which not heavily relies on the task at hand.
Estimated Utility of the Tools For all participants it was clear that both tools as they stand are not applicable for their daily working practice. The obvious reasons are the usability short- comings of PT1 and the fact that PT2 does not rely on any real data. However, beyond that we discussed with engineers much about potentials and prospective use cases which such or simi- lar solutions could support. These discussions revealed that most of our engineers estimated 3d
be a valuable helper for their own daily tasks we got twofold answers. The first half argued that they would rather stay with abstract representations and that they do not see any obvious benefits beyond the use cases named above. The other group argued that carefully adapting and extending the designs presented indeed would add value to their daily work. For using such tools beyond the use cases mentioned above, however, several aspects have to be reconsidered:
Scalability to real data: Both tools show extremely slowed down animations of one or sev- eral messages/signals through the system. While for education and presentation purposes this extremely downscaled version might be useful, for engineering purposes 3d visualizations must scale to real datasets and show them task-centered in an efficient way. This raises the question of how much information reasonably can be visualized in a 3d model view. Obviously, showing all messages in real-time, i. e., up to 15,000 per second is useless as it will not be perceptible, the same holds true for showing entire specification documents (e. g., all specified dependencies).
Meaningful ex- and abstractions of the data: Due to the restricted scalability, meaningful ex- or abstractions of the information have to be found. These ex-/abstractions one the one hand must be insusceptible against the restriction posed by a 3d representation but on the other hand provide enough spatial correlation to be worth representing in 3d. We collaboratively with our participants identified and discussed three scenarios where a 3d model view might add (more or less) additional value to engineering work:
1. Signal and message path:In line with the two early prototypes, several of our participants argued that it might be interesting for their work to see how information “moves” through the network. Scenarios mentioned were, for instance, detecting external error sources for analysts, e. g., a screw driven through a cable leading to communication breakdown, or to optimize system layout and improving reliability by additionally considering spatiality aspects, e. g., for deciding about alternative communication paths for crash safety commu- nication.
2. Electronic communication and mechanical behavior: In line with PT2, our participants argued that for several tasks it is definitely important to understand cross-correlations be- tween mechanical behavior of the car and the electronic communication. For trace analysts, for instance, it can be beneficial to see the real context of the test drive. For this purpose, to- day there are some tools available that allow for closely integrating video-taping with trace analysis. However, these recordings are usually restricted to one or several specific camera positions and it is also not possible to “look into” the vehicle as it would be possible with a virtual model. On the other hand, development engineers frequently mentioned simulation and virtual prototyping of vehicle electronics. The vision behind this idea is to start much earlier in the process of developing vehicle electronic to test its interactivity with mechan- ical components. Similar to mechanical simulations such as CFD (cf. Section 3.4.1) a 3d model could potentially serve as virtual platform for conducting test drives and to analyze the results subsequently.
the same way [i. e., color coding ECUs as in PT2]”.
Missing correlation to other tools/views: As still large parts of the electronic data is abstract (correlations, timings, etc.) nearly all our participants argued that a prospective 3d tool work definitely must be combined with other text-based or abstract, traditional or novel representation techniques. Two of our trace analysts could well imagine to integrate and combine such a view, for instance, with Carmen (assumed all technical restrictions have been overcome before, see below).