Tendencias del Branding:

steps, an explicit Euler forward scheme is performed. Let

OXi ^Vi

where the derivatives are computed by discrete approximizations and S* is computed via equation

(6.25). This leads to two independent Euler equations which can be minimized using a step size 7-

f i = - 7 (x<"> -

fÿ (6.29)

At equilibrium, the time derivative vanishes and equation (6.29) is solved, which can be achieved by iteratively solving

. '■ ...d ,1 -. |d,do,

6.3. Optimization 141

Parameter name Parameter value

^ela sticity 0.00001 ^bending 1 (^gradient 1 ^rid g e 0.1 Number o f iterations up to 30 Energy threshold 0.001

Number of points moved 1%

Greedy search space up to 7 X 7 pixel units

ICM normal search space 20 pixel units

Euler forward step size 7 = 1

Table 6.5: Energy function parameters.

This technique is also called simultaneous over relaxation. Numerical details of this technique

can be found in [Press et a l, 1992]. This purely explicit method allows for incorporation of hard

constraints into the energy function, in particular the curvature matching process as a combina­ tion of internal and external energy terms. Moreover, it can be continuously evaluated along the

contour, rather than at discrete snaxels only, whereas the classic scheme only allowed for contin­

uous evaluation of the external energy terms. Termination criteria are chosen similar to those of the greedy and ICM algorithms. The main disadvantage of this techniques is that only small step sizes 7 can be taken, as otherwise the technique becomes numerically unstable. This implies that

although each iteration is actually faster than for the greedy and ICM algorithms (as for each con­

trol point only the two derivative steps need to be calculated, rather than evaluating an M x M neighbourhood, or the normal search space), more iterations are necessary.

Table 6.5 lists the fixed, general parameters of the energy terms for all three optimization tech­ niques along with the associated optimization parameters. The energy parameters were selected empirically via a set of experiments, and then fixed for all results in this dissertation. The choice of the energy weighting parameters reflects the emphasis on the curvature matching process and the edge terms, while the weighting parameter for the elasticity terms is chosen rather low, as the

adaptive sampling strategy enforces uniform snaxel spacing.

This section has formulated three suitable extensions for scale-space integration into existing lo­ cal optimization techniques, where the locality of the respective solutions is given by the image and contour scale. Though all three techniques have been found suitable and yielding similar

6.4. Summary 142 speed, flexible search space, numerical stability, and good convergence behaviour in practice.

6.4 Summary

This chapter has presented the theoretical framework for a novel multi-scale active contour model. In particular, a suitable continuous spline representation has been chosen to overcome problems encountered in the classic discrete active contour model. This representation has been formulated in a multi-scale setting by adapting the internal contour scale in terms of the control point spac­ ing to the image scale using several newly developed multi-scale sampling strategies. A multi- scale energy function has been formulated to include differential invariants in scale-space, as well as normalized edge potentials in terms of the scale-space gradient and the distance transformed ridges of its magnitude. A curvature matching process of the contour curvature to the underly­ ing isophote image curvature has been developed and evaluated in terms of its dependence on the image contrast, robustness, performance and scale-dependence, and has been found to perform better in extracting shapes of high curvature parts than using the classic energy function which minimizes the contour curvature. It was shown that the concept of a spline-based multi-scale ac­ tive contour can be formulated in a local optimization framework. In the following chapter, the application o f this model as an implicit segmentation tool for shape description will be developed, and the concept of the resulting multi-scale shape stack will be presented.

Chapter 7

Multi-Scale Shape Stack

- P e t i t b o n h o m m e, n’e s t-c e p a s q u e c’e s t u n m a u v a i s r ê v e c e t t e h i s t o i r e d e S E R P E N T ET D E R E N D E Z -V O U S ET D ’ É T O IL E ...

’’Li t t l e m a n, t e l l m e t h a t i t i s o n l y a b a d d r e a m - t h i s a f f a i r o f t h e s n a k e, A N D THE M E E T IN G -P L A C E , A N D THE S T A R ...”

Le Petit Prince, Antoine de Saint-Exupéry.

The multi-scale active contour model presented in the previous chapter provides a tool for shape regularization and description in scale-space. The idea behind this is that a shape is represented and tracked by an active contour model through an image feature scale-space, consisting of the scale-space isophote image curvature, scale-space gradient magnitude and direction, and the dis­ tance transformed ridges of the scale-space gradient magnitude. At each level of this scale-space, the shape is quantified with respect to its size, curvature and other shape measurements. Further­

more, the whole set of regularized shapes is tested with respect to shape changes across scales.

This is achieved by formulating the set of shapes in a hierarchical manner as a multi-scale shape

stack, where each level of the stack represents the level of contour and image scale at which the

shape has been regularized. The shape stack can be obtained via two different processes: active

shape evolution, and active shape focusing. The analysis of the shape stack is referred to as active shape description.

This chapter first introduces some scale-space notations for the different possible dimensionalities of the image scale-space, multi-scale segmentation, and multi-scale shape stack. The techniques for active shape evolution, focusing, and description based on the concept of a multi-scale shape stack will be presented, and an example section will illustrate the interaction of these techniques.

7.1 Scale-Space Notations

Applying an active contour model to 2D or 3D image data and corresponding scale-spaces re­

quires the introduction of a suitable notation for image scale-spaces, multi-scale segmentation, and the resulting multi-scale shape stack. Table 7.1 presents an overview of the different scale-

7.1. Scale-Spaœ Notations 144

Properties 2 |D stack 3D stack 3 |D stack 3 |D stack

Original image Dimension 2D 3D Notation L{x, y) L { x , y , z ) Slice L{x, y, Zk) Image scale-space Dimension 3D 3 |D 4D

Notation L { x , y , a ) {Lzk(x,y', a)} L{x, y , z ' , a)

Sample L{ x, y , ai ) {Lzf^{x,y]ai)} L { x , y , z \ a i ) Slice Lzk(x, y]ai ) L( x, y , Zk \ ai ) Segmentation Dimension 2D 2 |D 2 |D 3D Notation v(5;<7) v (s,r;o -)

Sample v(s;o-i) v(5,r;o-i)

Table 7.1: Overview of scale-space dimensionalities of the multi-scale shape stack based on the different image scale-space dimensionalities and segmentation methods.

space dimensionalities which will be further discussed below. Let L(x, y) denote a 2D image

and L { x , y , z ) a ZD or volumetric image. Individual slices of a volumetric image are denoted by L{x, y,Zk). Furthermore, one can distinguish between image scale-spaces of three different

dimensionalities (note that the more general writing of <7i rather than for the individual scale

samples is used):

• 3D. A 3D image scale-space L{x, y\ a) can be computed of a 2D image, where the image

scale (J is treated as an extra degree of freedom. Individual scale samples or slices o f the

scale-space are denoted by L{x, y\ai).

• 4D. A 4D image scale-space L { x , y , z \ a ) can be analogously computed of a volumetric

image. A scale sample of this image scale-space is denoted by L{x, y, z; cr%), and a sample

with respect to an image slice is denoted by L{x,y,Zk;(Ti).

• 3 |D . A S |D o r slice-by-slice image scale-space is obtained by computing for each image

slice of a volumetric image a separate ZD image scale-space, yielding a set {Lz^ (x, y\cr)}.

A scale sample for a certain image slice is denoted by (x, y; (7^), and for the whole set

by {Lzj^{x^y\<7*)}. The scale-space structure is either organized as a 4D scale-space, or,

more commonly, as a list of ZD scale-spaces with one entry for each image slice.

The first two approaches can be generally formulated as an (N+l)-D scale-space L (x; a) of an

N-D image L(x). Scale samples of this scale-space are consequently denoted by L(x; ai). In

general, slice-by-slice approaches for the scale-space computation of N-D images are less mem­

ory exhaustive and computationally more efficient than their true (N+l)-D counterparts, but they

7.1. Scale-Space Notations 145

An active contour model is by its nature a 2D technique. If applied to a volumetric image, how­

ever, it can be optimized on the individual slices of the 3D data set. This is denoted by changing

the notation of a 2D active contour model \ { s ) to a set (s)} of 2D active contour models,

where the subscript denotes the image slice. In combination with image scale-space techniques

for 2D and 3D images, one can generally distinguish between four segmentation dimensionalities

[Vincken, 1995]:

• 2D. The full 3D scale-space information of a 2D image is used for the segmentation of a planar, 2D shape.

• 2 |D . A slice-by-slice, 3^D scale-space of a 3D image reformatted in order to obtain for

each image slice Zk an associated 3D scale-space Lz^{x, y\(j) which is then segmented

individually. The result can be formulated as a set o f planar shapes (one for each image slice), or as a volumetric shape using a suitable concatenation of the individual results, e.g. using triangulation.

• 2 |D . This segmentation is based on a true 4D scale-space of a 3D image, which is re­

formatted in order to obtain a 3D scale-space L{x,y,Zk]cr) for each image slice z*- Seg­

mentation is then carried out in analogy to the 2 |D approach. The higher dimensionality is motivated by the fact that the correlation between neighbouring image slices due to the higher order image scale-space influences the segmentation result.

• 3D. A true 4D image scale-space of a 3D image is used to segment a volumetric shape. A true 3D segmentation requires a volumetric technique, e.g. an active surface model v (s, r ) . This approach is not followed in this dissertation due to the high computational complexity and memory demands involved, but necessary extensions will be discussed in chapter 10.

All segmentation dimensionalities except for the last one can be achieved using active contour models. It is important to note that the aim of this dissertation is not multi-scale segmentation, but shape regularization with respect to scale. This is in contrast to other multi-scale segmenta­ tion techniques like edge focusing or the hyperstack (chapter 3, section 3.4.2), which are either only interested in the lowest scale result only, or in a final downward projection after establishing suitable links between the different scale levels.

This dissertation proposes to investigate each scale level individually, and regards only the full set of shapes at all scales as a complete shape representation. This set is obtained using the multi­ scale active model in scale-space, where it gains an extra scale dimension, and optimizing it on

the individual scale-space slices in a slice-by-slice fashion. Additionally, if the model is applied

7.1. Scale-Space Notations 146

the individual scale slices. Extending now the representation from v(5) to v(s; a), with v(a;

(for a natural scale-space) or v(s; Oi) (in the general form) at a particular scale level, allows to

incorporate the extra scale dimension. Moreover, at each individual scale level the model has also an individual contour scale % or %, respectively. When applying this model to volumetric images

and associated or 4D scale-spaces, each slice Zk can be optimized with an individual multi­

scale model, which is then denoted by (s; a), with analogous notation for the individual scale

levels. The resulting set of shapes v(s; a) (for 2D images) and (s; a)} (for 3D images) are

formulated as multi-scale shape stacks which are of one of the following dimensionalities:

• 2 |D .A 2 ^ £ > multi-scale shape stack is based on instances o f a planar shape in a 3D image

scale-space, consisting of all intermediate results of a 2D multi-scale segmentation process.

It is obtained by optimizing a model v(s; a) in each slice of the scale-space of a 2D image

separately. The organization of the stack is as a set of 2D planar shapes, or as a 3D structure

obtained by concatenating the individual scale results.

• 3D. A 3D shape stack is based on instances of a series of planar shapes (one for each im­ age slice) in a 3^D image scale-space. A multi-scale active contour model v^^(s; cr) is optimized separately in a 2^D fashion in each 3D scale-space associated with each image slice. The resulting shapes for each scale can be transformed back into a set of volumetric shapes using standard concatenation techniques. The 3D stack can therefore be structured in two different ways:

— as a set of 2^D multi-scale shape stacks, one for each image slice, or

— as a 3 |D multi-scale shape stack (see below), where each layer represents a single volumetric scale result.

3 JD . a 3^D shape stack is similar to a 3D stack, but it is based on a true 4D image scale-

space and a 2 |D multi-scale segmentation of a model (s; a) which contains spatial and

scale information about the shapes in the neighbouring slices. The individual results can again be concatenated to a volumetric shape. This stack is of higher dimensionality than 3D due to the higher order image scale-space.

• 3 |D .A 3 ^ D stack would be based on instances of a volumetric shape in a 4D image scale space. Such a stack can only be constructed using a multi-scale volumetric segmentation

technique, e.g. by a multi-scale active surface model v (s, r; a), which will be discussed in

chapter 10.

The multi-scale shape stacks developed in this dissertation assume that the shapes under inves­ tigation are planar, or can be formulated as a set of planar shapes. They add an extra scale di­ mension, and extra dimensions for instances of shapes in neighbouring slices. A true multi-scale

7.2. Active Shape Evolution 147

H For ail image slices

for zjfc = 1 to iV do

// Initialize with true (ground truth) model

setv ^ J s;o -o ) =

// Optimize for increasing scale levels

fo r i = 1 to n do

set {ai) = O p tim ize (s; a i - i ) )

end for

end for