Surface based algorithms actually perform all the registration in 3D. To do this, either a surface representation must be deduced from the video image or some other device such as a laser range nder. If a laser range nder is used it must be calibrated with respect to the video images. i.e. the transformation between 3D laser coordinates and video image coordinates must be known. Once a surface representing that present in the video image has been reconstructed, it must be registered to the surface extracted from the 3D image. Three such algorithms are now described in detail.
3.5.3.1 Grimson
et al.The work done by Grimson et al. [Grimson et al., 1995; Grimson et al., 1996] uses a laser range nder in conjunction with a single video image to deduce the surface visible in the video image. The laser range nder can reconstruct a surface accurate to 0.08 mm [Grimson et al., 1996]. This is registered to a surface derived from MR/CT. The reconstructed surface is rst manually edited and the two surfaces manually aligned. If
3.5 Medical Image 2D-3D Registration
76
with
T
, a matrix representing the transformation, then the evaluated function is:E
1(T
) =, X i X j exp ,(jT l i ,m j j 2=22) (3.3) which due to its inverse exponential nature gives a smooth cost function. The search strategy was the Davidon-Fletcher-Powell quasi Newton method (DFP) described in [Press et al., 1992]. This is a gradient based search requiring calculations of derivatives. The Gaussian function enables a multiresolution approach, by changing the variance. The resultant pose is rened using a least squares measure of the form:E
2(T
) = " 1n
X imin
nd
2max;min
jjT l
i,m
jj2 o # 1 2 (3.4) wheren
is the number of points andd
2max is a maximum distance threshold. This is more accurate but prone to local minima. The nal pose is perturbed randomly, and re-registered to further avoid local minima.Using a 0.9375 0.9375 1.4mm resolution MR image, results in an RMS error of
the order of 1.5mm. The algorithm takes 2-4 minutes for the laser scanning and the alignment, and the capture range is of the order of 5mm or degrees, with 100% success, reducing to 70% success at 10mm or degrees perturbation. This measure of success was obtained by randomly perturbing the start point from a xed point, and seeing how many times the algorithm reconverged to the start point. The threshold for successful was an RMS value of 2.5mm. The experiment ran 10 tests, at each of 1 - 10 mm or degrees misregistrations. More recent clinical work tests this system in 70 patient cases [Grimson et al., 1998].
3.5.3.2 Betting And Feldmar
et al.Betting and Feldmar also propose a surface based video - MR registration algorithm [Feldmar et al., 1997]. Two video images are taken, and a surface reconstruction leads to a dense set of points and normals. A second surface is extracted from an MR scan. The algorithm starts by computing pairs of bitangent points (see gure 3.8). Two points are bitangent if the plane dened by each point and its normal are the same. The initial estimate is performed as follows. Two points on one surface are taken, and the distance between them calculated. Then all pairs of similarly separated points in the second surface are tested. The two transformations to align the four points are calculated, and this process repeated until a suitable transformation found. Obviously, in practice,
3.5 Medical Image 2D-3D Registration
77
n 1 M 1 n2 M 2 (a)Figure 3.8: Two points are bitangent if the plane dened by each point and its surface normal is the same.
the algorithm takes into account that pairs of points will not be exactly superimposed due to discretisation noise, and also points in one surface may not correspond to points in another. The iterative match is a modied ICP algorithm where, instead of a 3D distance function, a 6D distance function computes the Euclidean distance between each corresponding point and their surface normals. The algorithm nds 5000 bitangent point pairs on the MRI surface, and 598 pairs on the stereo surface, in about 30 seconds. The initial estimate of the registration takes 30 seconds. At this point 80% of the points on the stereo surface have a closest point within 8mm, compared with the voxel dimensions of 4mm 4mm 8mm. The modied ICP is then applied using 15000 points on the
MRI and 10000 on the stereo surface. This takes 20 seconds, whereupon 85% of the stereo points have a closest point less than 3mm away. The registration is visually accurate, allowing pre-operative data to be overlayed on the video.
3.5.3.3 Colchester
et al.Colchester et al. [Colchester et al., 1994; Henri et al., 1995; Colchester et al., 1996] also use a video based surface reconstruction which is matched to a pre-operative surface extracted from an MRI/CT scan. The video surface is reconstructed by projecting a series of stripes onto the object. Two video cameras capture an image of the patient, and detect the stripes. Corresponding points in each view are matched and then surface points reconstructed using triangulation. The surfaces are registered using a cost function of the form
C
=XN3.5 Medical Image 2D-3D Registration
78
where
i
is a point number,N
the total number of points andd
i is the distance betweena point on the reconstructed surface and the corresponding closest point on the MR surface. The distance is found with a distance map, and if a distance is
>
10mm the point is ignored. This technique coupled with the log function deals well with outliers, and incorrect matches. A simple optimisation procedure, incorporating multiple start points and multi-resolution step sizes is employed to minimise the cost function and hence register the surfaces. The surface reconstruction takes 30 seconds, on a SUN SPARC IPX, and has an accuracy of 0.5mm in 3D. The MR data set was 256 256 80slices with 0.940.942.0mm voxel dimensions. The algorithm registers reliably with
real video data, coping with misregistrations of10mm and20, resulting in a mean
surface separation of 0.4 (0.5)mm, and a maximum surface separation of 2.2mm, which
was visually inspected, and `no deviations were apparent between the two surfaces'. If the surfaces were fully overlapped, then the