Ensuring that the driver is focused on the road and not distracted from the driving task is the main goal of various systems for Eyes-on-Road (EoR) detection known from literature [86] and implementations of series vehicles [87]. All studies mentioned in Section 4.1 performed a manual EoR detection or used an offline method with special markers positioned in the scene (e.g. quadratic black-white markers beside the steering wheel as seen in Figure 4.1). Automated EoR systems usually combine an absolute gaze direction1with AoIs of known vehicle coordinates. If the estimated gaze hits the defined AoI, e.g. the windshield, it is concluded that the driver’s visual attention is directed at this area. This method is often described as the geometric method since it makes use of a 3-D model of the vehicle. However, an accurate absolute gaze direction usually requires an end-of-line calibration of the camera in the vehicle, an expensive and time-consuming step. Moreover, a re-calibration becomes necessary after a certain amount of time. This effect is described, for example, in an extensive field study by Kircher et al. [88]. To avoid calibration while still applying the described approach, Vicente et al. proposed a vision-based system relying on robust facial feature tracking, head pose estimation, and a model-based gaze estimation [89]. As mentioned in Section2.3.3, such model-based eye-tracking approaches cannot exceed an accuracy of 5° due to individual differences of the fovea centralis. Thus, this approach will most likely show limitations for small and densely packed AoIs. Note that the results shown in [89] were achieved in a stationary vehicle.
Besides the challenge of a calibration-free system, a high detection rate of EoR gazes for the classification of take-over readiness is a difficult task on account of several factors. Estimated gaze direction is inaccurate due to varying lighting conditions, face and eye structures of different ethnic groups, or optical aids. To compensate for poor gaze estima- tion, Tawari et al. introduced a novel framework to estimate a coarse but more robust gaze direction of the driver [86]. The authors argue that such a coarse gaze direction is sufficient for an EoR system. The authors focused on increasing robustness of the EoR detection by incorporating head pose, eyelid, and iris features over a Support Vector Machine (SVM) to a gaze-surrogate estimation and showed a significant improvement over the detection rate using only a head pose based on the data of a field study. In literature, such approaches are
described as learning-based methods. However, learning-based methods are limited to a maximum accuracy of 5° similar to the model-based approach above. Vasli et al. extended the approach proposed in [86] by incorporating a multi-plane geometric model of the gaze zones resulting in a hybrid EoR detection method [90].
In addition to the above mentioned challenges, the driver’s eyes may not be visible to the camera system, e.g., due to large head rotations. Moreover, a first generation of driver camera systems may not include eye-tracking functionalities. These scenarios require a fallback strategy to compensate for missing gaze direction. The typical fallback strategy for EoR in case of missing eye gaze signals is to use an estimated head pose. Smith et al. [91] presented a system for determining the drowsiness level and visual attention of the driver based on eye features and a gaze direction computed by means of a mono camera. The actual classification of the visual attention level was done by means of three finite state machines. If drivers rotate their head up, down, to the left, or to the right or have their eyes closed for more than a defined number of frames, a low visual attention is classified. Trefflich [92] classified a driver as attentive if the vector describing the driver’s head pose intersects with a defined AoI on the windshield for at least 0.5 ms. This region on the windshield moves dynamically inside a larger static AoI and may change its size with regard to the current traffic situation. That way the AoI may increase its size towards the corresponding side when driving in a curve. If the head direction of the driver is outside of the defined AoI for more than 1.5 s, the driver will be classified as distracted. However, the absolute head pose may also be insufficient for the EoR detection in some cases since EoR gazes are usually divided into two components: eye movement and head movement. Head movement is often small and may result in an ab- solute head pose not necessarily facing the actual AoI when eye gaze direction is neglected.
There are already systems on the market which try to detect the driver’s visual attention. In 2009, Lexus introduced the Advanced Pre-Crash Safety System using a driver camera to detect the facial direction of the driver [87]. If a driver is not looking at the road for longer than a certain threshold, an audiovisual warning is given, followed by cautionary braking. The in-vehicle camera system contained a CCD2 imager with six near-infrared LEDs mounted on the top of the steering wheel column. The algorithm of this system actu- ally extracts facial features of the driver’s face and estimates the corresponding center line of the face. If the driver is not facing towards the road, the system will detect an unsym- metrical face in the recorded image. In case of drivers wearing glasses, features extracted from the edges of the glasses are used instead of the features of the eyes and eyebrows. Although it is an interesting approach, the described system does not calculate an accurate head direction or even a gaze direction for the EoR detection and is therefore prone to errors and false detections. For example, the Advanced Pre-Crash Safety System tends to unnecessary warnings in curvy road sections, since drivers usually focus on the apex of a curve while driving. Although these drivers focus on the road, the system detects a facial direction which is not pointing straight ahead and, therefore, is interpreted as "Driver is
distracted" by the system.