As previously explained, the main loop of DCS is the harmony search improvise-and- update step but without any pitch adjustment. This means that there is no need for the PAR or FW parameters. However, DCS introduces a new parameter that controls the frequency of the swap operation called the swap rate (SR). The SR is defined as the maximum number of idle iterations that can be performed before DCS switches from the normal main loop to the swap operation. Full pseudo code for the DCS algorithm is given in Algorithm .5 to .7.
Notice also that no convergence detection is done and that optimisation only ends when the total number of iterations have been reached. I found that, due to the introduction of the swap operation, traditional convergence tests fail to accurately detect convergence and that their addition does not speed up the algorithm. One could test for convergence by monitoring the motion covariance (see Algorithm .7 line 15) and this has been experimented with. However, this also did not significantly improve the speed of the algorithm and sometimes caused premature convergence when most of the features are unmatched.
Another convergence test that was considered was idle iteration testing. This was especially convenient since the number of idle iterations are already monitored to control the swap operation. However, this approach was found to be even more prone to converge prematurely. In the end the best results came from simply terminating after a maximum number of iterations which also keeps an upper bound on the running time.
4.4 Results
4.4
Results
In this section I evaluate the DCS algorithm using an example captured using the Bumblebee 2 camera from Point Grey Research1. I use a stereo camera to simplify the
calculation of 3D feature coordinates but do not use the DCS algorithm for calculating stereo correspondences between the sensors of the Bumblebee 2. A simpler algorithm specific to this type of stereo camera is used for this. In this example DCS was used to find correspondences in real time between new features from the current image from the camera, and features stored in a digital map. The application was a visual odometry system that uses multiple view odometry to calculate 3D positions from SURF image features captured from a mobile calibrated stereo camera. Some of these features are then stored in a digital map that represents knowledge of the camera’s surroundings. By matching new features from the mobile camera with those in the map using DCS, the camera ego-motion can be calculated.
The example that is considered is shown in Figure 4.6. In the first image 112 image features are extracted and saved in the digital map. The camera is then moved forward one meter and another image is extracted. From this second image 128 features are extracted. DCS is then used to build a correspondence set between the features from the first image and those of the second.
DCS is initialised with a prior correspondence set generated by matching features using only the feature descriptors and choosing randomly where this method is ambigu- ous as explained in previous sections. The prior solution identified 39 correspondences from the two images but many of these were not accurate and the motion vector vari- ance over all correspondences was large for this set indicating inconsistent geometry.
Once initialised with this prior the DCS algorithm optimised the correspondence set in 600 iterations producing a much more accurate set of 38 correspondences. The optimised set has a 3D motion vector variance of [0.125325, 0.0113381, 0.281385]. Compared to the prior’s variance of [2.51285, 0.133806, 8.65715] the optimised set’s variance is significantly smaller.
A trace of the optimisation process illustrating the evolution of the best and worst solution in the HM is shown in Figure 4.7. Only the first 200 iterations of a total of 600 is shown as this is where most of the optimisation occurs. Notice that after an initial few
Figure 4.6: Two images of the same scene taken from different positions. The top image is taken first. Then the camera is moved one meter forward where the second image is taken.
iterations most of the progress is made through swap operations since normal memory consideration fails to make further improvements. Notice also the rapid progress made during the first 50 iterations. After 50 iterations the quality of the best solution in the HM is already more than 15 times better than the prior it was initialised with.
By carefully investigating the position of the crosses in the graph (that represent swap operations), and comparing this with the positions where progress is made to a better correspondence set, we find a pattern forming. This pattern shows several memory consideration steps followed by one or two swap operations that often result in an improved solution. It is very interesting to notice that, in the DCS improvisation
4.4 Results
process, memory consideration is mainly responsible for maintaining diversity in the initial search for improved solutions and usually does not play a large role in the later stages of optimisation. However, if we were to argue that memory consideration is not necessary after the first few iterations and that further optimisation should only be done using swap operations, all the solutions in the HM would quickly degrade to simply being slight variations of the best solution considerably slowing down convergence. DCS uses memory consideration to maintain diversity in the HM without resorting to randomly perturbing solutions in the hope that the optimiser will stumble across better solutions.
Figure 4.7: The optimisation process is shown through the evolution of the best and worst solution in the HM. Most of the optimisation is done in the first 200 iterations shown here. The crosses indicate iterations in which swap operations occurred and show its importance to successful optimisation.
Unfortunately, memory consideration is not able to maintain a diverse solution set in the harmony memory for long. Notice from the graph that as the worst solution and the best solution become more and more equal, the optimiser’s ability to improve significantly on the best solution diminishes. Normally, when the worst solution in the
HM becomes close to equal the best one it is an indication of convergence and that an optimum has been reached. I found that this was rarely the case with DCS. While the best candidate solution reached after 200 iterations is much better than the prior initialisation, the solution is still suboptimal and still has some features incorrectly matched. This means that the HM now consists of several slightly modified copies of a local optimum. Due to the loss of diversity in the HM, further progress and possible escape from this local optimum is unlikely.
However, DCS performs very well over this first few iterations and compares well with other popular algorithms like RANSAC and graph cut. In a real world setting it is very difficult to find the optimal correspondence set mainly due to the imperfect modelling of image features. These limitations in feature extraction and modelling was explored in Section 4.2.1.