experiment reported later the thresholds dτ and ςτ are set to 0.5pels and 1.0pels2 respectively, unless specified otherwise.
Given that bothDn,m and Rn,m are high, this result indicates that the general consensus of
the trajectories is that the Euclidean distances between bundle n and m stay roughly constant over a sequence. We define thresholds for the metricsDn,mandRn,masDτ andRτ respectively.
We make these thresholdsDτ and Rτ be 80% and 70% respectively for all experiments reported later, unless specified otherwise.
We always estimate the means and standard deviations of the changes in Euclidean distances for the bundle with the most trajectories. Recall that the number of trajectories in bundles n
andmareRandSrespectively. We define the bundle with the most trajectories as thereference bundle. Hence bundle n is thereference bundle ifR > S, otherwise bundle m is thereference bundle. Using these means and standard deviations for the trajectories in the reference bundle provide more robustness in estimating the merging metrics Dn,m and Rn,m considering that
there might be outlier trajectories in the bundles. Here an outlier trajectory is defined as a trajectory that does not always follow the average motion of the bundle it belongs to. Since the merging metrics are estimated through a trajectory voting process, outliers will have a smaller effect on the these metrics when there are a lot more reliable trajectories in thereference bundle. In the previous discussion we assumed to bundle n had more trajectories than bundle m. However, if bundle m is the reference bundle then we estimate the merging metrics Dn,m and
Rn,m using the estimated means ¯ds and standard deviations ςsfor the trajectories Xs in bundle
m. The procedure for estimating ¯ds and ςs is the same as previously discussed for ¯dr and ςr,
however note that in this case we are estimating the changes in Euclidean distances for the trajectoriesXs in bundle m with respect to the trajectories in bundlen.
5.4
Graphcut Solution
The α-expansion Graphcut algorithm [135] is used to solve for the MAP estimate of the trajec- tory bundle labelsLt. The posterior distribution for the trajectory labelsLtdiscussed previously
is repeated below.
p(Lt|X,L∼t)∝px(Xt|Lt)ps(Lt|L∼t) (5.21)
Details of the adaptation of the Bayesian problem in eq. 5.21 above to this Graphcut solution are exposed in appendix A. There are no major issues to raise as far as the Graphcut techniques are concerned.
5.5
Summary
This chapter discusses therefinementstage in oursparse trajectory segmentationtechnique. This stage follows the initialization stage discussed in chapter 4. In therefinement stage a Bayesian
framework is used to enforce spatial and temporal smoothness on the trajectory bundle obtained from the initialization stage.
In the refinement stage we utilize an algorithm we define as the local region constraint algorithm to ensure that each trajectory bundle represent a single coherent image region. Also in this stage we include a trajectory merging strategy that is effective for merging trajectory bundles corresponding to non-rigid objects.
6
Sparse Trajectory Segmentation Performance
Evaluation
In this chapter the performance of the proposed algorithm is compared to previous methods. Recall from the review in chapter 2 that sparse trajectory segmentation algorithms may be categorized based on whether they provide a 2D or 3D segmentation.
A 3D segmentation approach aims to group all the trajectories belonging to a particular object in 3D space. Each group of trajectories provided by a 3D segmentation is defined as a 3D trajectory bundle.
A 2D segmentation approach like our proposed technique, groups trajectories of coherent 2D image motion. To distinguish between the results of a 2D and 3D segmentation we define a group of trajectories labelled as having similar 2D image motion as a 2D trajectory bundle.
The illustration in fig. 6.1 will be used to demonstrate the difference between the segmen- tation results for both the 2D and 3D approaches. An important distinction between both approaches is the level of segmentation produced. Understanding the nature of both segmenta- tion results is necessary for establishing some grounds for comparing them.
The top row of fig. 6.1 shows a truck at two different frames in a motion sequence. Some approximately planar surfaces on this truck are uniquely highlighted for illustration purposes in red, green, cyan, purple, and yellow. Here matching surfaces across both frames are coloured in a similar manner.
A 3D segmentation algorithm is required to identify all the trajectories generated by the truck as belonging to a single object. The spatial locations of these trajectories are shown as
Figure 6.1: Top row: A truck shown at frames 1 (left) and 24 (right) in a motion sequence from the Hopkins dataset [117]. Some roughly planar surfaces on the truck are highlighted in red, green, yellow, purple and cyan. The matching surfaces across both frames are coloured in a similar manner. Bottom left: The desired labels for a 3D segmentation algorithm. The spatial locations of the trajectories for the background and truck at the current frame are indicated with red and yellow ‘dots’ respectively. Bottom right: The trajectory segmentation result produced by our proposed technique. Each trajectory bundle shown in a different colour roughly represents a single surface of the truck.
yellow ‘dots’ in thebottom left illustration of fig. 6.1. Note here that the 3D segmentation task becomes more challenging when the object of interest is subjected to significant perspective distortions.
The 2D motion in the image plane of each surface of the truck is dependent on the relative perspective of the surface with respect to the camera. The red (side) and cyan (front) surfaces for example will have relatively different 2D image motions due to their orientations and distances from the camera.