Failure can be defined as the "omission of occurrence or performance; specifically: a failing to perform a duty or expected action" and as "a cessation of normal func- tioning" [87]. The previous Section 2.1 presented four examples of state-of-the-art autonomous ground vehicle systems, which despite their implementation differences, all share a common underlying system architecture where perception plays a vital
role for mission success. In this section, the main cause of failure in UGV perception systems is discussed along with examples of perception systems contributing to a fail- ure of the UGV. These examples motivate the methods for mitigation and recovery considered in this thesis.
The main cause source of failure in perception systems are from abnormal environmen- tal conditions. When a sensor is employed outside of specified operating conditions, it will produce unexpected data that does not necessarily match the requirements of the perception system. Subsequently, the perception system is unable to interpret the sensor data to produce the required accurate and useful representation of the en- vironment for the application. Challenging environmental conditions are those that tend to cause failure in state-of-the-art UGV perception systems because the sensors do not provide appropriate data for a useful representation. Recognised challenging conditions include; darkness, bright lights, fog and smoke with visual cameras; dust, smoke and rain with lasers, and fire with IR sensors. For example, a perception sys- tem using only visual cameras will fail to function at night without artificial lighting because the sensor is incapable of providing any useful information. Similarly, it will also fail if the camera is pointed directly at the sun causing the resulting image data to saturate. A laser sensor will also typically fail when pointed at the sun as the light saturates the signal. These types of challenging conditions are recognised as having a significant effect on the performance of state-of-the-art UGVs [65, 130], and gen- eral perception problems caused by sensor data interpretation errors are still largely unsolved for robotic perception systems [71, 139, 141].
The Boss UGV perception system (see Section 2.1.3.2) was found to fail in the pres- ence of dust clouds [141]. Figure 2.9 shows Boss attempting to perform a U-turn manoeuvre after detecting a blocked road. In the figure, a number of different colours have been overlaid to represent different aspects of the environment as interpreted by the perception system. These include roads edges as blue lines, blue squares showing regions considered high cost (difficult but not impossible to traverse), unobserved regions in green and obstacles in red. The white lines show the limits of the vehicle turning circle and the red lines show some potential paths of the vehicle as estimated
by the path planning module. In this example, obstacles have been detected close to the front and back of the vehicle even though there is no real obstacle on the road in front of the UGV. The result is that no suitable path can be planned and the UGV fails to move for a number of minutes.
Figure 2.9 – Example of perceptual failure of the Boss UGV. Figure from [141]. The laser scanners observe a dust cloud and a false obstacle (in red in front of the vehicle) is created in the RotE. Subsequently, the vehicle was unable to move.
This failure was due to the onboard laser sensors detecting dust clouds in the en- vironment. These laser returns were indistinguishable from those of a solid object (such as a tree, rock or pole) and were included as an input to the perception system. Although a dust cloud is not an actual obstacle for a UGV, the dust-affected laser data was above the surrounding terrain and this rapid elevation change was inter- preted by the perception system as an obstacle. Subsequently, the RotE included false obstacles generated by dust and the path planning application failed to obtain a valid path through the environment causing the vehicle to stop.
Perceptual failures were also observed in the MER system (see Section 2.1.3.1). The authors of [82] identified a number of situations in which the perception system failed to provide a sufficient representation for an accurate localisation. First, at times when
the rover was physically tilted upwards due to the slope of the terrain, the onboard cameras pointed towards the sky. In this case, the limited observability of the sur- rounding terrain meant that there were an insufficient number of features to provide an accurate localisation estimate. Second, at times the ground appeared homoge- neous in images, such as in darkness or shadow or when the ground was covered in fine dust. In these situations, the image data did not provide enough unique features that could be distinguished and then tracked effectively for an accurate localisation. Third, when the vehicle moved too quickly (particularly when turning), there was little overlap between consecutive images and not enough features were available for successful tracking. Finally, although not mentioned explicitly in the report, the RotE for localisation assumes a static world environment meaning that features that are tracked by the perception system are stationary over time. This is a fair assumption in most situations on Mars, but, despite the Martian atmosphere being only 1% the density of Earth, the wind and dust storms can be severe, leading to a highly dynamic environment [30]. If a significant portion of the image contains airborne dust, there will not be enough stationary features available in the image data to provide model the motion of the vehicle for an accurate localisation.
Challenging environmental conditions are recognised as a common source of failure in perception systems. In a UGV system, the perception system is linked to high level decision making and so a failure can propagate and cause the UGV to fail to meet objectives. The following section presents related work that has been performed to improve the resilience of perception systems subject to recognised challenging condi- tions.