8.1
Experiments
In this section we present the experiments performed to evaluate the accuracy of the 3D map segmentation and the generated path feasibility, on the 3D labeled map. The segmentation has been evaluated with respect to the time required to both segment and label the point cloud, and with respect to its accuracy. The segmentation accuracy has been measured by manually assigning a label to each cluster and by comparing these labels with the labels returned by the segmentation process. The percentage of the clusters correctly classified states the accuracy of the segmentation. The feasibility of the 3D path is measured in terms of the time required to build the graph structure representing the traversable areas of the environment, with respect to the value of the path smoothing algorithm, and in terms of the time needed to the robot to complete the path, namely the journey time. This time has been taken by hand with a stopwatch. To test the effectiveness of the autonomy capabilities of the overall system three different scenarios have been considered.
8.1.1 The Italian Fire Fighters rescue training area in Prato (IT) First experiment has been performed at the Italian Fire Fighters rescue training area in Prato (IT), during the final review meeting of the EU project NIFTi (247870). In this experiment the robot traversed the harsh terrain of the rescue area, though not climbing any ramp or stairs, overcoming small obstacles, following different paths toward several target poses, manually posted by an operator on the 3D map. Figure 8.1 illustrates the segmented map with the inflated region in red, the generated path toward the goal (the gray cube) and screen shots of the robot following the path. The dialog window accommodates automatic goal generation, which is not treated here. Table 8.1 reports the accuracy of the segmentation of the 3D map of the environment and the feasibility of the path, generated on the 3D segmented and labeled map. Here the values are averaged over eleven trials to obtain the same goal position. On average, the map had about 52 thousand points.
8.1.2 Fire Escape stairs
The second experiment has been performed on the fire escape stairs of the Department. The goal is located on the landing at the end of the second stairway, and when the robot is up the goal is moved to the ground. The robot has simply to climb up the stairs to reach the goal and then turning on its self and step down back to reach the new selected goal. In some trials the robot is actually not able to get down autonomously, because localization problems might arise when the robot is turning around itself. Figure 7.4 shows the robot climbing the fire escape stairs, visualizing only the graph where the inflated regions are in red. Figure 8.2, left panel, shows
Figure 8.1. The Italian Fire Fighters rescue training area in Prato (IT)
Figure 8.2. Fire Escape stairs
the segmentation of the courtyard with the fire escape stairs and, in the window up right, a detail of the robot climbing the stairs. Table 8.1 reports the average segmentation time, first table; the percentage of the clusters correctly classified, middle table; and in the last table the average time needed to generate the 3D path on the stairs, with respect to the value of the path smoothing and the journey time. On average the map has 41 thousand points.
8.1.3 Full 3D designed scenario
. The third experiment has been designed purposefully to test the effective 3D autonomy, within an environment whose structure is composed of multiple levels. A gallery, surmounted by a ramp, extended with a bridge, is built and it lies between a step on the floor and a wall of bricks (see Figure 8.3). The robot has to first traverse the gallery and then climb the ramp passing over the gallery and continue up to
8.1 Experiments 65
Table 8.1. Data from the three experiments
Point Cloud Path
Segmentation Graph Smoothing Journey
Time (s) Accuracy Time (s) k Time (s)
Prato experiment 0.27 0.86 1.19 0.2 357
Fire escape stairs 0.32 0.91 1.45 0.2 210
3D designed 0.12 0.97 0.92 0.2 143
the end of the bridge. The goal is located at the end of the bridge. The robot is constrained to pass under the gallery by obstacles. It is interesting to note that in this experiment the space is fully 3D since the robot has to face both the levels: under and over the construction (see Figure 8.2, right panel). Table 8.1 reports, for three trials, the accuracy of the segmentation of the scenario, as well as the feasibility of the complex 3D path. On average the map has 15 thousand points.
Figure 8.3. On the left segmentation of the third experiment map. On the right a screen shot of the generated graph.
8.1.4 Computational time performance
In addition to the previous experiments, several tests have been performed to evaluate the computational performance of both the segmentation and the path planning algorithm, with respect to the size of the point cloud. During these tests the robot was teleoperated, so as to explore a wide area and to acquire, at real-time, the point cloud. The goal was fixed and the robot computed a new path every new point cloud. The results, in terms of time of computation, of the different algorithms are reported in Figure 8.4. Note that the robot was able to elaborate a point cloud composed of 55000 points and to generate a path in less than 3.5s.
Figure 8.4. Time of computation for both the segmentation and the path planning algorithm with respect to the size of the point cloud.