To calculate the amount of coverage C, an estimate of the area covered by the robot is required. It is important not to calculate this area from robot’s positions estimated from dead reckoning. This is because dead reckoning attempts to estimate the robot’s position based on the distance it travelled in its current direction from its previous position. Consequently, accumulation of errors, especially in the orientation, can lead to a larger and larger discrepancy between the estimated and the actual poses.
Therefore, an external, independent perspective of the experiment process is required. A simple and cheap solution is to use a wall mounted camera to capture a movie of the robot’s progress. The experiment setup used in this thesis is shown in Figure 5.3 and a sample image from the camera is shown in Figure 5.4. Computer vision techniques can then be employed on the images captured to estimate the area covered by the robot. Figure 5.5 shows a flowchart explaining the process. First, the original frames captured by the camera are combined to form a single composite image. This composite image illustrates all the surfaces the robot covered. Since the original images contain perspective distortion, this artefact is removed using a deskewing operation. The percentage coverage is then estimated from the resulting deskewed composite image.
In this thesis, a composite image of an experiment refers to a single image that shows the trail or path of the robot during the experiment. To create such an image, the position of the robot is first extracted from each frame captured by the camera. These positions are then superpositioned to form a single image.
Two methods have been devised for creating composite images [105]. The first method, image
subtraction, requires very little computation. However, it suffers from two major disadvantages.
Firstly, it is not particularly reliable as artefacts frequently appear in the background subtraction process. This is because of environmental factors such as lighting. Additionally, in the case of a tethered robot (such as the Khepera) the cable motion is computed as part of the final composite image.
5.3 In real robot experiments 105
Figure 5.3: Mounted camera capturing frames of robot’s movements.
Figure 5.4: An image captured by the experimental setup shown in Figure 5.3.
image
deskewedimage
Area=?
coverage
frames
original
composite
106 Performance metrics
of the robot using a Hough accumulator [79]. The major disadvantage of evidence gathering is the Hough transform used is very computation intensive. More details on the computer vision techniques employed in evidence gathering can be found in Appendix B.
Image subtraction
The first step in this method is to subtract a reference image r from each of the original images
fiin the sequence. Thus
si = fi−r (5.6)
As the images employed are full colour (RGB) images, this subtraction is computed for each
colour channel separately. The difference si highlights the movement of the robot in the image
sequence. This is because the robot is the only moving object in a static environment.
It is desirable that the reference image r be an accurate representation of the environment with-
out the robot in it. Any error in the reference image will result in artefacts in si. If the ex-
periments are very short, it is possible to use an image taken immediately before or after an experiment as the reference image r, as long as the robot is not in the scene.
However, if there are illumination changes within the scene, then a temporal average of the entire sequence should be used as the reference image r. A na¨ıve approach would suggest using the average of the images. However, this will result in a ghostlike trail that corresponds to the motion of the moving object. Instead the median operator was employed. This computes the reference image as follows
r=medianf0,· · · , fn−1
(5.7) for a sequence with n images f0to fn−1.
In implementation, the median operation is computed via a histogram of image intensities at each point and finding the value corresponding to the 50th percentile.
After the subtraction in (5.6) is computed, the difference images si are thresholded pointwise
with a step response. If the value of a pixel is greater than 127, the value is changed to one;
otherwise if the value is smaller than 127, 0 is assigned instead. These thresholded difference
images are then combined by performing a pointwiseOR of the all the images. This method-
ology works as the main point of difference between successive frames is the robot. As the
thresholded images are effectively a binary representation of this difference, then a logical OR
5.3 In real robot experiments 107 Evidence Gathering - Hough Transform
The first step of evidence gathering is image subtraction using (5.6). The resulting difference
images si are then edge detected to find the edge information for each colour channel. The
edge map is computed using a Canny edge detector [25, 79], for two main reasons. Firstly, it is considered to have an optimal response for step edge responses, and secondly it can serve to reduce noise in the image. This noise reduction is useful as it can help remove small artefacts
in si due to estimation problems in the reference image. The final edge map, ei, is computed
via weighted summation and thresholding of the individual colour channels. The weighting in the summation allows the emphasis of features in particular colour bands. The value of the threshold is tuned empirically.
The edge map ei is then presented as evidence to a model of the robot. The specific model
used depends on two factors — the shape of the robot; and how this shape changes because of
perspective effects as the robot moves within the environment. For example, the Khepera robot
was modelled as a circle. This is possible as at the extreme end of the enclosure, where the robot is at the maximum distance from the camera, the Khepera is still circular. In this case the model fitting must find the diameter that best fits the Khepera in each frame. A Hough transform is used to fit the model to the edge data. The specific model fitting algorithm is shown below:
Algorithm 5.1 Hough Evidence Gathering for d = Dminto Dmaxdo
for all edge pixels (x,y) do forθ= 0 to 2πdo
xc ← x− d2cosθ
yc ← y− d2sinθ
if (xc,yc) is in image and (xc,yc) is an edge point then
increment A(x,y,d) end if
end for end for end for
For a circular fit, the image is initially examined to find all the edge points. For each edge point, all the points are computed that are in a circle of diameter d and centred on this edge point. Diameter was employed instead of radius as it is easier to compute this from an image sequence. If the computed point is another edge point, then an array is incremented with a point indexed by the original edge point and the diameter.
After evidence is gathered from the entire image, the array A(x,y,d) will contain peaks which
correspond to likely circle centre points and diameters. Within this array the peak corresponds to the circle centre and diameter which occurs most often. This is the best candidate for the robot’s
108 Performance metrics u 3 2 0 1 3 2 1 0 y x v
Figure 5.6: Quadrilateral to square mapping.
location within the frame. It can be computationally expensive to search all possible diameters so the possible search space is reduced by examination of the original image sequence. The purpose of this is to find estimates of the diameters of the robot at the maximum and minimum
distance from the camera. This sets upper and lower bounds to the diameter parameter, Dmin
and Dmax.
The output of the model fitting stage will be a series of numbers which describe the central coordinates of the robot and the model chosen to describe the robot. By examining the results from all successive frames an accurate representation of the robot’s path can be found. This, along with the model of the robot, can be used to produce a composite image for estimating the surface area covered by the robot.