Contrast stretching is not applied to the entire image, but rather has to be applied to individual sub-windows. An update of all the SATs is required for every sub-window
Figure 7.4: Merging positive hits with Moments Invariants over the same object demand a more sophisticate approach than the one adopted when using Haar-like features.
and the method becomes too slow. Ideally, the contrast stretching should be part of the feature extraction. An approximate method based on what Lienhart and Maydt (2002) did for Haar-like features is proposed. One needs extra SATs for a rapid implementation of the contrast stretching using moments due to the fact that there are different orders for the moments invariants. However, these extra SATs can be pre-computed as they are independent of the images. The basic equation used for the lighting contrast stretching in this chapter differs slightly from that used by Lienhart and Maydt (2002).
Lienhart and Maydt (2002) created an extra SAT using the squares of the pixel values (I2(x, y)), so they could obtain the sum of the squares of any rectangular area of the frame. This SAT is used to compute the variance in rectangular areas of the image. With the mean and variance of a particular sub-windows computed straight from two SATs, they implemented a fast contrast stretching method. They used equation 2.8, adequate to the properties of the Haar-like features. Rather than using that equation, preliminary experiments showed that equation 7.39 is more convenient, as its form makes it easier to generalise the fast contrast stretching calculation:
¯i(x, y) = 255(i(x, y)−µ+cσ) 2cσ , c∈ ℜ
+, and 0≤I¯(x, y)≤255 (7.39)
Where ¯i(x, y) is the image after contrast stretching,µ is the mean,σ is the variance, and c is a constant. This equation is similar to the one commonly used for contrast stretch (e.g. see Gonzalez and Woods (2002)). Rather than explicitly choose a value for the transform’s constants (the slope and the crossing point of the straight line), these are based on the local mean and variance.
7.2. Geometric Moment Invariants 113
Each componentxpyq of a momentm
pq can be written as:
mpq =
X
n
i(x, y)Cn (7.40)
Where: Cn isxpyq.
The resulting moment after the contrast stretching is: ¯ mpq = X n 255(i(x, y) +cσ−µ) 2cσ Cn= X n 255i(x, y)Cn 2cσ + X n 255Cn(cσ−µ) 2cσ (7.41) ¯ mpq = 255 2cσ X n i(x, y)Cn+ X n Cn(cσ−µ) (7.42) ¯ mpq = 255 2cσ(mpq+ (cσ−µ) X n Cn) (7.43)
As Cn is a function of the position (x, y) only, the expression PnCn can be pre-
computed at the beginning using SATs with unit images (all pixels values set to 1) and do not need to be repeated for the duration of the sequence of images. Values between 1.5 to 2.0 are typical for the constantc. In our experiments,c= 1.8. The approach is not linear due to the cut off that needs to be done in order to limit the pixel values between 0 and 255. In practice, however, there are few pixels to be cut, so the final values for ¯mpq
given by equation 7.43 and 7.39 are equivalent.
Experiment 3: contrast stretching
Stretching Contrast Before After Contrast Stretching B C D E F G A
Figure 7.5: An example of contrast stretching. The images after applying the contrast stretching are actually slightly different to each other, yielding different moments.
The results of the computation of moments, with and without contrast, demonstrate that equation 7.43 works for a reasonably large variation in lighting conditions. Figure 7.5 shows some face images with different contrasts. The moments computed from these raw
images have large variances. Each image was pre-processed by equation 7.39, shown in the second row of images in figure 7.5. The moments computed from these stretched images are compared to the approximation given by equation 7.43 (computed directly from the raw images). Table 7.2 shows the mean and variance of the moment invariants obtained from the images of figure 7.5. The results show that the variances among the moments of the raw images are large, while the variances for both contrast stretching methods are much smaller. Moreover, the values for the moments computed from the stretched images from equation 7.39 are very similar to those computed directly from the raw images using equation 7.43.
Table 7.2: Variances for contrast stretching.
raw images equation 7.39 equation 7.43 (slow method) (fast method)
µ σ µ σ µ σ ψ1 6.56 0.7273 6.76 0.0048 6.73 0.0040 ψ2 28.40 4.2471 26.28 0.3421 26.13 0.4193 ψ3 57.02 7.0751 52.81 0.1389 52.33 0.1973 ψ4 40.96 5.5388 39.43 1.7405 37.46 0.6608 ψ5 56.03 8.5864 51.54 0.5968 51.28 0.7381 ψ6 39.83 5.9100 36.61 0.4489 36.25 0.4529 ψ7 12.82 1.4578 13.21 0.0118 13.15 0.0045 ψ8 46.57 6.7813 43.91 0.9761 43.70 1.4094 ψ9 44.98 6.5260 42.05 0.3995 41.66 0.3631 ψ10 73.45 9.2288 70.60 1.5961 69.35 1.1996 ψ11 73.13 10.4276 68.50 0.7519 68.14 0.8600
The results of this simple experiment showed that it is possible to overcome the prob- lem of lighting variation when computing moments from SATs. The advantage of using equation 7.43 is the fact that the contrast stretching was incorporated into the moments computation, making it still possible to extract feature in real-time. The cost of the im- provement is limited to additional lookups into theCn SATs (fifteen SATs if computing
the eleven moment invariants used before), which can be created at the start to be shared by any frame.