Enfoque de las necesidades

Several cross sectional studies have confirmed a high level of precision in lesion volume

quantification with newer techniques (Wicks et al, 1992; Filippi et al, 1995d; Grimaud et al,

1996). However, it is not the absolute lesion volume, but the difference between serial estimates that is measured to provide an endpoint in definitive treatment trials; no previous work has

defined the precision and reliability of such techniques in measuring this change. Clearly, measurement of any change requires a technique with a high level of precision, since random errors in measuring lesion volume at each time point will have a cumulative effect in compromising the identification of differences over serial MRI investigations. This is particularly important given that changes in Tj lesion volume measured on annual MRI are fi-equently small (typically 5-10% per annum).

In this study, the efficacy of two quantitative techniques for measuring lesion volume was examined. Lesion segmentation is only one of many potential sources of measurement error; the the overall accuracy and precision of measurement is affected by errors at each stage. These results therefore ignore the impact of variable scanner performance. Furthermore, the effects of

sub optimal repositioning (Gawne-Cain et al, 1996; Simon et al, 1997) and inconsistency in

lesion identification have also not been considered, because the aim of this study was to define and compare the precision and reliability of the quantitative techniques themselves.

The values for intra and inter rater CoV obtained in this study confirm a higher level of precision with the contour technique than for manual outlining. A high level of agreement was also found between the lesion volumes obtained with the two segmentation techniques. The manual outline technique is currently regarded as a gold standard measure, but this study has shown that the contour method produces very similar lesion volumes to those derived with manual outlining, with the higher precision afforded by computer-assisted lesion delineation. The results also confirm that the contour method has a higher level of precision than the manual outlining technique in identifying differences in lesion volume between serial studies. This implies that, being less subject to random error, it represents a more powerful technique for identifying any effect of treatment on change in lesion volume.

Reliability provides an index of the ability of a measurement technique to discriminate between the different members of a sample population (Fleiss 1985; Streiner & Norman 1995). Even a technique that is highly precise may not be able to distinguish between patients if the population range of the measured value is narrow. The very high values of reliability for measurements of baseline lesion volumes are perhaps not surprising, given the wide range of lesion volumes on these scans. More significantly, however, the reliability for measuring the relatively small changes in lesion volume over two years was excellent with both techniques. This suggests that the variance due to random measurement error is small, compared with that due to the wide biological variability in lesion volume changes across the patient population. To exclude the possibility that sample variability had been increased by including eight patients treated with interferon beta-lb, the variance between patients for the change in lesion volume in the placebo group and for the group as a whole was subsequently analysed. The variance between patients was actually reduced by inclusion of the treated group; the values obtained for reliability were not therefore elevated by the sample choice.

The impact of less than perfect reliability on sample size estimations for treatment trials is illustrated by the following equation (Fleiss 1985);

n = n*/R

where n * is the sample size per group based on a perfect measurement technique, R is the reliability defined as the ICC and n is the sample size per group after incorporating the effects of measurement error. With values for reliability of greater than 0.9, the effect of measurement error on sample size requirements is clearly small for both segmentation techniques (measure-

ment error would necessitate an increase sample size in each arm of less than 11%). This reflects the wide distribution within the sample for the change in lesion volume, and might suggest that optimal precision may not be an imperative. However, in a more homogenous population, or with a shorter interval between serial studies, the higher precision of the contour method might be reflected in a more substantial difference in reliability between the two techniques, and it is clearly appropriate to use the more reliable segmentation method.

A major disadvantage of both quantitative techniques used in this study is the high level of human interaction that they require. Phase III treatment trials may require analysis of thousands of images; lesion identification and segmentation can therefore take months or even years to perform. Several automated quantitative techniques have recently been developed using multi-

parametric approaches to perform lesion segmentation (Mitchell et al, 1994; Simmons et al,

1994; Udupae^a/., 1996; Udupaera/., 1997; Evans e/a/., 1997; Simone/a/., 1997). These offer the potential for considerably greater efficiency. However, the significant presence of motion artefacts, field inhomogeneity within images and partial volume effects can potentially lead to errors in lesion classification. Any such inconsistency over serial MR images will result in the inaccurate assessment of any change in lesion volume. These techniques should therefore be validated by demonstrating that they can remain responsive to real changes in lesion volume over time, in the presence of such artefacts. Despite the considerable time requirements of the contour technique, human intervention in lesion identification reduces the risk of mis-classification. Furthermore, if serial images are assessed together, consistent decisions can be made on whether or not to classify equivocal areas of high intensity as lesions. The contour method utilises both the ability of an experienced observer to discriminate between lesion, artefact and normal anatomy, and the higher degree of precision in lesion delineation than is possible with the fully manual technique.

Although the contour technique is more precise than manual tracing of the lesion boundary, the algorithm still requires an observer to place the cursor at a point on the lesion edge. Lesions may have poorly defined edges due to the effects of volume averaging. Several possible boundaries can be produced by the contour algorithm for less discrete lesions, depending on the exact position of cursor placement, and this substantially contributes to the residual inconsistency in derived lesion volumes. Two approaches may further improve precision in serial studies. The first is to optimise the lesion-to-background contrast and therefore reduce the amount of manual editing required. The fast FLAIR sequence suppresses CSF signal intensity and is reported in

some (Filippi et al, 1996d; Bastienello et al, 1997; Filippi et al, 1998d) but not all (Gavme-

Cain et al, 1998) cross sectional studies to improve precision with the contour method. The

second approach is to use a smaller slice thickness in order to reduce partial volume effects. One effect of finite slice thickness is to cause tissue mixing at the perimeter of lesions and produce loss of edge definition. As slice thickness is reduced, volume averaging effects should be less apparent and this may improve precision in quantification of lesion volume. This issue will be addressed in Chapter 5.

In summary, this study has demonstrated that the contour technique represents a major improvement over manual outlining for lesion volume quantification in terms of precision and reliability; these results support the further use of the contour method of Tj lesion volume quantification on serial MRI. Indeed, based on the data in this chapter, it was decided to use the

contour technique in assessment of the T2 lesion volume in the therapeutic trial of interferon beta-

Chapter 4. The performance of global thresholding with a histogram