PARA USO DEL CENSO SOLAMENTE

Figure 3.7: Metal structure of the visual system. Limits the field of view of the camera to 30 x 50 cm.

Figure 3.8: Representation of the circuit at INSIA installations took with Google Earth tool.

3.3.

Visual Localization Algorithms

The implementation of the image processing related to this work has two parts: The line detection and the marks detection and decoding. The first is used in both approaches, and the mark detection is used, only in the second one, in which the system has information about its location inside the circuit.

3.3.1.

Lateral Estimation Error

For the line’s detection, a custom real-time computer vision algorithm was designed. The algorithm is able to detect the line’s centroid and orientation under harsh conditions, such as partially occluded and poorly painted line on a rough terrain, coping with non-compact line shapes.

On the front-end of the visual system, the camera captures the scene which is lit with UV light at 30 fps. First, a color-based segmentation is performed on

Chapter 3. Fuzzy Logic Control for Ground Vehicles 23

YUV space. Despite some other color spaces were tested, YUV was found to be the best performer under different light conditions. A rectangular prism inside the YUV colour space is defined, so that only the pixel values inside this volume are considered to be part of the line. The result is a binary image where only the line pixels are set. This method proved to be robust detecting lines of different blue tones and brightness.

In the binary image, every 8-connected pixel group is marked as a blob. At the first step, to reduce the noise, blobs having an area outside a defined range are discarded. Then, for every survivor, centroid, dominant direction and maximal length are calculated, and those being too short are ignored. The remaining blobs are clustered according to proximity and parallelism, so each cluster becomes a candidate line. The centroid and dominant direction of each candidate line is calculated from the weighted sum of the features of its component blobs, being the weight of each blob proportional to its relative area. In this way, the algorithm is able to accurately detect lines that are fragmented because of ageing.

Finally, from the whole set of candidate lines, a detected line must be selected for the current frame. In order to do that, the distance between the centroids of every candidate line in the current frame and the detected line in the previous frame is measured. If the smallest distance is higher than a certain threshold, the detected line will be the leftmost or rightmost candidate line, depending on the user-defined criterion. Otherwise, the closest candidate line is taken as detected line. This mechanism avoids switching to ”fake” lines when there are traces of old paintings along the circuit, even when it is deteriorated.

The algorithm outputs whether the line is detected or not, if it is, it also outputs the error of the line in the x-axis from to the center of the image and the direction of the line, expressed as an angle.

3.3.2.

Global Localization by Visual Marks

The algorithm is able to detect and recognize visual marks painted next to the line to follow, even when they are rotated in the image because of vehi- cle turns. Each visual mark represents a binary-encoded number, which is the mark’s unique identifier. Mark bits are drawn as bars that are parallel to the line. Because of the reduced visual space, instead of painting a bar for each bit like in common barcodes, wider or narrower depending on the bit’s value, a more com- pact encoding scheme was chosen. All bits are bars with the same width, with no spacing between them. When a bit is 1, the bar is painted; when it is 0, the bar space is left unpainted. In the image, a mark appears as a set of yellow rectan- gles. Every rectangle is a bit with value 1. A start bar is added at the furthest bit slot from the line to designate the beginning of the mark. The mark specification also defines the number of bits per mark, a fixed bit width, a minimum bit length, and valid ranges for line-mark angle and line-to-start-bit distance. According to the specification, the algorithm will only spot marks that lie to the right of the line in the direction of movement of the vehicle.

24 3.3. Visual Localization Algorithms

As in the line detection phase, the potential mark pixels are segmented by color in YUV space. The color space boundaries are set so that the segmentation admits several yellow tones, ranging from tangerine yellow to bright green, as seen in tests with multiple paints. This makes the color-based filter less restric- tive, avoiding false negatives. On the other hand, the probability that any yellow noise has a valid mark structure is low. The next step of the algorithm seeks this valid structure in the color-segmented pixels, thus reducing the false positives.

The 8-connected pixels from the color segmentation are grouped in blobs. The blobs not meeting the following criteria are considered noise and are dis- carded: the blobs must lie to the right of the line, the blob area must be in a valid range (computed from the visual mark specification), the angular distance between the dominant blob direction and the line must be in the specified range, and the blob length in the dominant direction must be larger than the minimum bit length.

The blobs that pass the filters correspond to a set of bits with value 1. The specific number of bits in each blob is determined by pattern matching. Assum- ing each mark has a total of N bits (including the start bit), N pattern tests are carried out for each blob, one test for each number of bits in [1, N]. For i bits, the pattern Pi,B is a rectangle with the direction and length of the blob B and a width equal to i times the bit width in the specification (in pixels). The number of bits in blob B will be the value i that minimizes the cost function present in the Equation 3.1.

c(i, B) = f (i, B) + ˆg(i, B) (3.1)

Where

f(i, B) = 1 −a(B ∩ Pi,B) a(B) ˆ

g(i, B) =g(i, B) g_max g(i, B) =| a(B) − a(Pi,B) | and a(S) is the area in pixels inside the shape S.

Function f indicates how much the pattern covers the blob. Function g eval- uates the similarity between the blob and pattern areas. Patterns whose f or g are above a threshold are discarded. This forces the best solution to have a minimum quality in both indicators. Then, the minimization process favors patterns that cover the blob while having a similar size. The threshold for g is gmax and ˆgis a normalized version that stays in [0, 1], like f does.

After assigning a number of 1s to all blobs, the rightmost 1 is interpreted as the start bit. If its distance to the line is in the range allowed by the specifica- tion, the mark is decoded. The decoding starts by finding the mark’s dominant direction as the average of all its blobs. The orthogonal vector to this direction defines a baseline that is divided into N consecutive fixed-width bit slots, being the start bit on the first slot. All bit slots are assigned a default value of 0. Then, blob centroids are projected on the baseline and each projection falls into one of

Chapter 3. Fuzzy Logic Control for Ground Vehicles 25

the slots, which is filled with a 1. Adjacent slots are also filled with 1s according to the blob’s number of bits. The slot values define the binary-encoded mark identifier, whose least significant bit is the closest to the start bit.

The final detected mark identifier that is passed to the control system is elected with a voting process among the last M frames. Large values of M pro- duce higher detection delays but increase detection robustness, as more image samples are taken into account. In our experiments, M = 3 has given good re- sults. Besides the detected mark identifier, the algorithm provides an estimation of the number of frames and time lapse since the mark was last seen. It is espe- cially useful at high speeds and high values of M, when the decision is deferred until M frames have been captured, but the mark was only seen on the first few frames. Moreover, an estimation of the mark quality is given based on its aver- age luminance. Figure 3.9 shows the detection of the line and the mark which represent the number 19.

Figure 3.9: Line and mark detection by the comnputer vision algorithm.