In this section, different approaches for a metric are discussed. As falsification without the required data is not possible for advanced metrics, the theory behind the different concepts is analyzed in order to define a suitable approach.
5.3.1.1 Object Prediction
To assess the possible outcome of a scene, the future trajectories of the objects are required. This can be done without using information of the data recordings by different approaches. This is called a priori evaluation. In trajectory planning, probabilistic approaches are the most common that predict the movement of objects by underlying assumptions (e.g. by a statistic model such as Bayesian networks). When evaluating recorded data, it is also possi- ble to use the true trajectories of the objects because the future of the scene in question is known. The possible ego-vehicle trajectories are assessed depending on the known outcome. However, this is only valid for a limited timespan as the objects movement might be influ- enced by a changing trajectory of the ego-vehicle. A disadvantage of an a posteriori assess- ment is that behaviors that are subjectively critical might be assessed differently because it did not turn out as critical in the future as it could have. One example is driving with very small time-gap on a motorway. From daily driving, it is well known that small time-gaps at high speeds are critical because the necessary reaction time on a deceleration of the front object is too small.
In an a posteriori assessment, the scene is not critical as long as the front-object does not decelerate. Nevertheless, a behavior like this would be punished by the metric in the long- run because deceleration will occur when enough data is analyzed. In other words, more data are required to assess this behavior with a lesser safety extrapolation.
assess driving with small time-gap with a lesser safety extrapolation. Nevertheless, it re- quires data to be parameterized, so the advantage is questionable. As there is only limited data available and further parametrization is necessary for an a priori approach, it is not favorable. Another aspect is that an a posteriori approach is extendable when new data is available. With existing data, a model could be built and the results of both approaches com- pared. However, this might results in over-assessment of the criticality in scenes with small lateral distance. Without sufficient data, it is assumed that the driver’s mental model of the surrounding traffic is more accurate than a model that is just based on the existing data. To summarize it is stated that the advantage of an a posteriori assessment exceed their dis- advantages. The fact that data about the future is available should be used with the benefit of relinquishing behavior models that are a source for errors is parameterization. This con- cept is obviously not found in trajectory planning because the trajectory is decided upon in real-time. A similar concept is found in silent testing or VAAFO188,189, where incorrect envi- ronment representation is corrected a posteriori. With insufficient data, the safety extrapola- tion could underestimate the true safety compared to an a priori approach. However, the fact that additional data improves the extrapolation using a posteriori assessment, while all data must be reevaluate with an updated model in a priori assessment. This consideration also suggests using a posteriori assessment, as it is uncertain if enough data is available.
5.3.1.2 Criticality Assessment
From section 5.2.3.6 it is known that in order to fulfil RM 1, the criticality assessment shall consider different trajectories and assess the driving requirements in order to find their min- imum. Different approaches known from literature that could be used with a posteriori as- sessment of the scenario are discussed in the following. In section 2.2.2.2.1, multi-object metrics where introduced in the categories geometry-based, sample-based, potential-field and optimization. Here, their potential for the fulfillment of the requirements, especially RM
1 is discussed.
For simple evasion maneuvers, geometry-based methods find suitable trajectories. Though, there is no information whether the final solution is the least critical or not. The combination of different maneuvers is also not covered. Some scenarios might require parallel or serial combination of steering and braking maneuvers so the methods cannot be applied on every scenario. Hence, they are not eligible for EVT.
Sampling-based methods generate trajectories based on random inputs, usually following a certain probability distribution resulting in a high number of possible outcomes. In the liter- ature, either the number of accident free trajectories defines the criticality190 or the available
188 Junietz, P. et al.: Gaining Knowledge on Automated Driving’s Safety‐-The Risk-Free VAAFO Tool (2019). 189 Wachenfeld, W.; Winner, H.: Virtual Assessment of Automation in Field Operation (2015).
area around the trajectory191. These definition might fulfil RM 1 in some scenarios but not in every as the following example shows: When straight driving is possible but the area or the number of trajectories sampled in the available lane are small, the criticality would be as- sessed as high. So only similar scenarios are comparable. Single trajectories could also be assessed according to their driving requirements but there might be a relatively uncritical trajectory that is not part of the sample.
Potential-field and optimization methods on the other hand, both result in a single trajectory. Especially, optimization methods could find the trajectory with the least criticality according to a defined cost function. In trajectory planning, the driving mission is always included in the cost function. If the cost function is modified in order to suit the definition of criticality in section 1.2.4, RM 1 might be fulfilled. Hence, the cost function should contain the neces- sary acceleration, the reaction time and the necessary precision of the maneuver.
To conclude, out of all discussed approaches, optimization methods are likely most suitable for criticality metrics. However, the definition of the cost function that contains all criticality information into one equation without other information such as the driving mission ins un- usual for trajectory optimization. Even though the approach seems promising, the fulfillment of the requirements cannot be proven but is falsifiable after being applied on data.
A remaining challenge is the combination of the different aspects of criticality. In order to define the cost function, they shall be combined into one function with output costs that are minimized in an optimization. This requires the definition of at least two parameters (shape and weighting) per component resulting in six parameters. As derived above, this parametri- zation and especially arbitrariness in the parametrization needs careful evaluation and justi- fication. Ideally, the parameters are connected to the driving capabilities in a way that deri- vation of accident likelihood is possible. However, modelling the driving capabilities requires data that might not be available.
According to RM 2, the maximum value for the final metric shall represent a scene, in which the accident cannot be prevented. This could be used either by designing the cost function in a way that at each time step, the value range is below or equal to the maximum value, or by using part of the cost function as criticality assessment.
As the range of value for the final metric is arbitrary, it is hereby defined between zero and one so scenes without increased criticality are described by the value zero and scenes, in which the collision is not preventable are defined by the value one.
5.3.1.3 Summary
As summary, these design guidelines can be concluded:
The metric shall consider the reaction time, precision of course angle and driving dynamic reserve.
The three components shall be combined to a value that describes proximity to an accident using the estimated driving skills.
Parameterization and weighting of the different factors shall be analyzed and justi- fied.
If arbitrary parametrization is chosen, the influence on the risk extrapolation shall be analyzed.
Object trajectories should be assessed a posteriori, if there is no calibrated object prediction model available.
All possible accident free trajectories should be considered resulting in the trajectory with the least accident likelihood. Trajectory optimization is one feasible approach. In the following, a metric is developed according to these guidelines and the requirements derived above.