El dominio espacial del dispositivo de seguridad

This section has provided an overview on the importance of depth information for effective scene recognition. A robust and efficient method of ordinal depth recovery, inspired from the TBL motion of bees and wasps is also presented. How the proposed SRS exploits TBL motion to enhance the salient ROIs with ordinal depth information is detailed in section 5.2.2. The contribution of ordinal depth to the recognition accuracy of the SRS is discussed in section 7.5.5.

3.6

Final remarks

This chapter has introduced several important concepts related to the proposed SRS. It is helpful to link these core concepts, summarised below, to the parts of the thesis for easy reference. All the references link to the specific sections where the concepts are first introduced.

• Visual saliency for reliable landmark detection (section 4.1)

• Encoding of the salient ROIs using SURF descriptors (section 5.1.4) • Ordinal spatial configuration in the Scene matrix (section 5.3)

• Determining scene similarity using rank correlations (section 6.1) • Illumination invariance using HSV colour space (section 5.1.1) • Recovering ordinal depth from TBL motion (section 5.2.2)

The rest of the thesis will focus on presenting the design of the proposed SRS using the concepts introduced. The next chapter begins this description by detailing how the proposed SRS selects the initial salient ROIs using visual saliency.

82

Chapter

4

Visual saliency for landmark extraction

This chapter describes the first stage of the proposed SRS with the selection of visually salient initial landmarks, or salient ROIs from a computed saliency map. The computational model of visual saliency adopted in this work is described in section 4.1. This computational model includes several enhancements such as the inclusion of new composite feature maps described in sections 4.2 and 4.3. The final output is a depth-weighted saliency map (section 4.4) where the regions in the foreground are made more salient than regions in the background using an estimated dense ordinal proximity map. The depth-weighted saliency map indicates regions of high salience as potential salient ROIs which are then extracted using simple image morphological operations (section 4.5).

4.1 Modified Itti’s computational model of visual saliency 83

4.1

Modified Itti’s computational model of visual

saliency

The usefulness of visual saliency in selecting initial salient ROIs for scene recogni- tion had been explained in sections 3.1.2 and 3.1.3. In this section, an overview of the modified computational model of visual saliency by Itti is detailed.

The computational model of visual saliency proposed by Itti et al. [51] is one the most popular models of the human attention system. The original work made predictions of how human attention shifts from one salient region to another, by applying a Winner-Takes-All (WTA) algorithm on the resulting saliency map that is produced. The original model is shown in Fig. 4.1 (left). In this work, a modified version of this model that uses several new composite features, Cf, suitable for scene

recognition in outdoor environments is proposed (Fig. 4.1 (right)).

Definition 4.6. Composite features Composite features, denoted as Cj_f for the jth _{composite feature, are image regions that are deemed to be potentially salient.}

They are used by the saliency algorithm to determine if salient ROIs exist in these regions.

Apart from the original intensity and orientation composite features, a modified opponent-colour composite feature using the concept of colour constancy for an illumination invariant representation (section 3.4.2) proposed in [111] is used. This

4.1 Modified Itti’s computational model of visual saliency 84

Figure 4.1: Left: Itti’s original computational model of visual saliency [51]. Right: The modified computational model of visual saliency with several new composite features.

representation was shown in [111] to be able to robustly extract salient regions under varying illumination. Furthermore, two new composite features, namely long edges and skyline, are extracted from the image if they exist. The salience of each composite feature is computed and represented as a conspicuity map. Unlike the equal weights that are used in [51] to combine the conspicuity maps together, the entropy of the individual conspicuity map is used instead to weigh each map before combination to form the saliency map. A final additional step involves modulating the saliency map with an estimated dense ordinal proximity map so as to finally obtain a depth-weighted saliency map.

The three composite features - intensity, orientation and opponent-colour, are extracted by various algorithms detailed in [51, 111]. These features are well-known in the psychophysics literature as potentially salient regions that model well the

4.1 Modified Itti’s computational model of visual saliency 85

human attention system. The intensity features are extracted by simply converting the original RGB image to grayscale while the orientation features are obtained by convolving the intensity (grayscale) image with a set of Gabor filter banks [85]. The four opponent-colour channels R0, G0, B0, Y0 are computed directly from the normalised RGB channels, r, g, b. See [51] for details of the computations. The various composite features extracted are shown in Fig. 4.2. The next two sections focus on the remaining two novel composite features introduced in this thesis.

Figure 4.2: Various composite features extracted from the input RGB image: intensity (grayscale), orientation (gabor) and opponent colours.