The computation of optical flow is important to understand the dynamics of a scene. This section evaluates the optical flow estimation in different scenarios under different illumination conditions (night, day, shadow). The images are taken from a moving vehicle where the camera monitors the road ahead.
Figure 4.19: Optical flow computation with illumination changes. Due to illumination changes, the optical flow constraint in the two input images (upper images) is violated and flow computation using plain gray value inten- sities fails.
The first experiment in Figure 4.19 shows the optical flow computation on an image se- quence with a person running from the right
into the driving corridor. Due to illumina-
tion changes in the image (compare the sky re- gion for example) and severe vignetting arti- facts (images intensity decreases circular from the image middle), flow computation using plain
gray value intensities fails. Using the pro-
posed structure-texture decomposition, a valid
flow estimation is still possible. See Fig-
ure 4.20 for the same two frames where opti- cal flow is computed on the structure-texture
decomposed images. Note the reflection of
the moving person on the engine hood which
is only visible in the structure-texture de-
composed images. What is more impor-
tant, artifacts due to vignetting and illumina- tion change are not visible in the structure-
texture decomposed images. This demonstrates
the increase in robustness for the optical flow computation under illumination changes using the proposed decomposition of the input im- ages.
Figure 4.21 shows the same scene a few frames later using the additional fundamental matrix data term. Most of the image, except for the running person, is static and the expanding flow field should follow the epipolar rays. The results show that, except for the Mercedes star and the running person, this assumption holds. It is not in the scope of this chapter to segment moving objects, the results however are well suited to detect independently moving regions of the image. There are more constraints available than just the distance to epipolar rays constraint to detect moving objects (see [2]); segmentation approaches will be discussed in Chapter 6.
Figure 4.20: Optical flow computation with illumination changes. Using the structure-texture decom- posed images, a valid flow estimation is now possible (compare with Figure 4.19).
4.4 Experimental Results (Optical Flow) 54
Figure 4.21: Optical flow for a scene with a running person and a moving camera, installed in a vehicle. The distance to the epipolar rays encodes independently moving objects.
original frames
structure-texture decomposed frames
Figure 4.22: Optical flow result for a night scene.
Another example to illustrate the robustness of the structure- texture decomposition is shown in Figure 4.22. Here a scene at night with reflections on the ground plane is used. In the intensity images the scene is very dark and not much structure is visible. The structure-texture decomposed images reveal much more about the scene. Note, that this information is also included in the intensity image but most structure in the original images is visible in the cloud region. The figure shows the optical flow using the decomposed images. Note the correct flow estimation of the street light on the left side.
The next examples in Figure 4.23 demonstrates the accurate optical flow computation for large displacements. It shows a scene with shadows on the road. Clearly, the structure-texture decom- posed images reveal the structure on the road surface better than the original intensity images (the shadows are still noticeable be- cause a blended version of structure and texture is used as pre- sented in Section 4.3). Different scales for the optical flow color scheme are used to demonstrate the accuracy of the optical flow algorithm. Although nothing about epipolar geometry is used in the flow algorithm, the effect of expansion (and hence depth) corresponding to flow length becomes visible. Note, that opti- cal flow for the reflection posts is correctly estimated even for flow length above 8px. Optical flow is correctly estimated for the road surface up to 30px. The shadows in the scene have no neg- ative impact on the flow calculation. The engine hood acts like a mirror and optical flow on the engine hood is perturbed due to reflections. Although the optical flow for the engine hood is very much different from flow vectors on the road surface, this has no negative impact on the estimation of the optical flow for the road surface. Note the accurate flow discontinuity boundary along the engine hood and in the tree regions.
In Figure 4.24 the image is taken while driving below a bridge on a country road. Note, that the shadow edge of the bridge is visible in the original images but not in the decomposed images. The large flow vectors on the reflector post are correctly matched. The figure also illustrates the limits of the presented Refine- ment Optical Flow approach. In the vicinity of the car optical flow is perturbed due to missing texture on the road surface. Due to the dependency on the linearized optical flow constraint, op- tical flow in texture-less regions and in regions with very large displacements is still not satisfactory, highlighting the need of further research.
Figure 4.23: Optical flow field for the scene depicted in the upper left with the original and structure- texture image. The flow is saturated for flow vector length above 1, 2, 3, 4, 5, 6, 8, 10, 15, 20, 25, 30 pixels from left to right.