A typical Downhill Simplex minimization requires distance measure evaluations in the order of 100 to 200. Using the C based distance calculation and OpenGL rendering, pose estimation in synthetic image of 800 by 600 pixels took around 1 minute for models with more than 30,000 vertices. Recent work done in [65] on pose estimation using point correspondences, takes more than 3 minutes (200 seconds) for a synthetic image of a model with only 80 points. Hence, despite being a pixel based method, the performance of our approach is very encouraging. Further improvements in speed may be obtained by using the graphics hardware (GPU) for computing the distance measure.
method to obtain this 3D pose by optimizing an illumination invariant distance measure. As with the pose estimation method proposed earlier Chapter 3, the distance measure is optimized with respect to the pose parameters in order to obtain the optimal matching pose. We present pose estimation results on real car images. When comparing the sensitivity of the pose estima- tion methods to the initial pose used to seed the optimizer, we show in Section 4.6.1 that the method presented in this chapter is a lot more robust than the method presented previously in Section 3.2 in Chapter 3. Therefore, the pose estimation method presented in this chapter is better way to assist the model based segmentation work presented in Section 3.2.
An advantage of our method is that it does not rely on extracting feature points from the photograph. Therefore, it is useful when working with photographs of cars, which have large reflective surfaces where feature extraction gives noisy and unreliable results. Another advan- tage is that our distance measure is independent of the scene illumination. Hence, the method can be used under different illumination conditions which are not known a priori.
Experimental results suggest that our methods does not work well when the vehicle in the photograph has very large perspective distortion effects. Also, the pose estimation method is sensitive to the GrabCut margin used to initialize the background removal process.
In theory part of the vehicle which are not in the 3D CAD model can be expected to be damage. However, vehicles have very reflective body panels which result in a lot of inter object reflections which can also be confused as damage. We present next in Chapter 5, a method to obtain reliable point correspondences between photographs of highly reflective objects such as vehicles, with large homogeneous regions. Based on the work described in Chapter 5, we also discuss methods for detecting image edges in vehicle photographs caused by inter object reflections in Chapter 6.
5.1
Introduction
Structure from Motion (SfM) techniques that recover geometric scene information from a set of images obtained from different views and most other tasks in multi view geometry, typically require reliable point correspondences across the images (or tracks in the case of more than two images) as a prerequisite. The applications range from complete 3D scene reconstruction to stereo matching performed on uncalibrated images. Typically, various features in the images are detected and matched in order to obtain such correspondences.
Much research has been done in this area and popular applications which use feature cor- respondences include the work done by Snavely et al. [78] on aligning tourist photographs obtained from the Internet. However, most conventional techniques for obtaining such corre- spondences were not intended to be used with images of highly reflective surfaces with large homogeneous regions such as
The method proposed in this chapter is capable of obtaining reliable point correspondences from such imagery. The point correspondences obtained thus can be used for various multi view geometry and SfM tasks.
We show in Chapter 6 the utility of using point correspondences obtained from our method to estimate a homography transform between photographs of vehicle panels with mild dam- age (Section 6.3), which can be used to detect inter object reflection in the photographs. We demonstrate in this chapter, the reliability of the obtained point correspondences by estimat- ing the epipolar geometry between two photographs and by performing an uncalibrated image rectification on the photographs. We motivate our approach as follows.
5.1.1 Motivation
Most conventional techniques for obtaining such correspondences were not intended to be used with images of highly reflective surfaces with large homogeneous regions. We are particularly
Figure 5.1: Best point correspondences obtained from naive SIFT [53] matching do not give a sufficient spatial spread to recover the epipolar geometry. The reliable matches are concen-
trated around relatively non-reflective areas. Best viewed in color.
interested in images of cars, which tend to have a lot of reflections and regions which are oth- erwise poorly textured, due to the shiny metallic body of the car. To illustrate the nature of the problem, we show in Figure 5.1 results of naively applying the SIFT [53] feature detection and matching algorithm on a pair of very reflective images of a car. The best SIFT matches are concentrated towards a corner of the image and are hence unsuitable to recover the epipolar geometry of the scene. Even methods developed to work with comparatively feature impover- ished scenes like the work by Linet al. [51] were not intended for noisy images with a lot of reflections, as discussed in Section 2.3. Hence, our novel approach proposed in this chapter.
The main contribution of this chapter is as follows.
5.1.2 Main contribution
We propose a novel method which uses feature descriptors evaluated along edge points in the images to obtain point correspondences across two images. The point correspondences are picked such that they have a good spatial distribution across the images. Our method is able to obtain a sufficient amount of representative matches (inliers) which can be used to recover the epipolar geometry of the scene from images where baseline methods fail (Section 5.3). Unlike existing methods, our method does not place any restrictions on the camera (e.g., affine camera, small motions) and works with highly specular reflective surfaces such as the body of a car.
Relevant background material for this chapter, along with a review of related work was presented in Section 2.2 and Section 2.3. The rest of this chapter is organized as follows. We