ESTUDIO DEL CASO DE LA EMPRESA INDUSTRIALES DE TINTAS LTDA.

One of the limitations of the Mean-shift tracking algorithm together with the skin detection is occlusion handling. Because there is no significant information about the

boundaries of the hand and face i n the skin blobs, using an extra cue to distinguish the boundaries of the hand and face may be useful. In this section we describe the technique of using the depth information together with the Mean-shift algorithm. The depth information is extractable using a stereo-vision system.

A stereo-vision system can provide the depth information of the scene in terms of

distance to the camera. This information can be used for enhancing the algorithms

which we introduced in Chapters 3 and 4.

The simplest configuration of a stereo-vision system is hi-camera. However, three or more cameras also may be used in a stereo-vision system. Stereo-vision can produce a dense disparity map which can be translated to the depth information map. The resultant disparity map should be smooth, detailed and continuous. Moreover, surfaces should produce a region of smooth disparity values with their boundaries precisely delineated while small surface elements should be detected as separately distinguished regions. Unfortunately, satisfying all of these requirements simultaneously is not achievable. Algorithms that can produce a smooth disparity map tend to miss the details and those that can produce a detailed map tend to be noisy. The depth maps obtained by hi-camera stereo systems are not very accurate and reliable. A higher number of cameras may gain better quality depth information

[ 1 50] .

While the luxury of stereo-machines with more than two cameras is not yet commonly available, the normal hi-camera stereo-vision systems are the most available choice. However, the depth estimation ability is somewhat limited. We used

Chapter 5. A novel approach for robust tracking .. .

a Bumblebee19 bi-camera stereo-vision system for applying the techniques proposed

in this section. Connected to the fire-wire port, this camera is able to record two 1 024x768 calor or gray scale images in 25 fps. The test platform was Windows XP and the programming platform was C++ in Visual Studio.Net 2003. Although the

depth map obtained using this camera is not highly accurate (Figure 5 . 15), it can

provide an estimation of the distance of different objects to the camera which was sufficient for evaluation of our ideas.

Figure 5.15. a) A sample image, b) the depth information of the sample image. The light area is the object closer to the camera, the black patches are of unknown depth.

5.5.3.1 The depth information and the adaptive skin detection algorithm

Let' s take a step back and review the concept of motion detection for our adaptive

skin detection algorithm. In fact, one of the reasons for the motion detection is

separating the user and the background. Considering the fact that in an HCI environment the user is closer to the camera than the background, the depth information can significantly improve the accuracy of background elimination from

the image. This idea can be implemented using depth thresholding. Figure 5 . 1 6b represents the eliminated background and the remaining foreground after applying the depth thresholding technique.

Figure 5.16. Background elimination using the depth thresholding technique.

The detected skin pixels within the separated foreground are more likely to be the actual skin color and therefore the adaptive skin detection algorithm will provide more accurate detection. Some of the unwanted areas like the surface of the table behind the user will also automatically be eliminated.

5.5.3.2 The depth information and the Fuzzy Mean-shift blob tracker

The depth information can be used for occlusion resolution in some scenarios. Figure

5 . 1 7 presents two cases in which the distance of the hand and face to the camera are

different. Obviously, without the depth information the hand and face blobs in these

images are being considered as one single blob. In this section, we describe how the

depth information can be applied for enhancing the Fuzzy Mean-shift blob tracker algorithm for occlusion prevention.

Chapter 5. A novel approach for robust tracking . . .

Figure 5.17. Some of the scenarios in which depth information can be useful for occlusion resolution.

Based on the idea of applying the depth information as an extra cue for blob tracking

within the Mean-shift algorithm, we implemented another variant of the Fuzzy Mean- shift algorithm . The basics of this variant in Kernel shifting is similar to its ancestor. However the kernel management is slightly different.

The kernel in this version of the algorithm in addition to the vertical and horizontal boundaries of the tracking blob also carries the minimum and the maximum depth of the points within. The shrinking and enlarging procedure on the boundaries of the

kernel are based on the pixels with depths within ± 1 0% of the kernel ' s min-max

depth. The pixels which do not satisfy this constraint or their depth is unknown will be ignored. Therefore, blob trackers which are tracking blobs in different depth levels

will not interfere with each other. Figure 5 . 1 8 represents the result of this algorithm

on an image. We should note that without the depth information, the hand and the face blobs are considered as one connected blob. Using the depth information, the

Mean-shift algorithm is able to stay around the blob which has the homogeneous

Figure 5.18. Occlusion resolution using the depth information and the Mean-shift algorithm. The tracked blobs are displayed on the disparity image (right).

5.6 Chapter summary

In this chapter, we presented a new approach for boundary detection in blob tracking based on the Mean-shift algorithm. Our approach was based on continuous sampling of the boundaries of the kernel and changing the size of the kernel using our novel

algorithm. We also showed that the proposed method is superior in terms of

robustness and stability compared to the density-based tracking method known as the

CAM-Shift algorithm.

The robustness of our method against noise makes it a good candidate for use with cheap cameras and real-world vision-based HCI applications. This method is to be

applied in conjunction with a fast pixel-based skin color segmentation algorithm as