Through the exclusion criteria imposed, the original cohort of 281 sagittal slices (52 patients) has ultimately reduced to a cohort of 140 slices that were processed and analysed. This number was still large enough not to hinder statistical inference of the study. If agreement between the sheargram and radiologist were by chance (i.e. the probability of agreement was 0.5), three standard deviations about the mean of a binomial distribution with n=140 would be placed at 88/140 slices or 62.8% agreement. Any agreement above 62.8% may therefore be considered statistically significant to a p-value of 0.01.
137
5.3 Results
To reiterate, the purpose of this pilot study was to assess the extent to which the sheargram (visceral slide quantification technique) agreed with clinical opinion as a means of determining its potential as a diagnostic aid for cine-MRI abdominal wall adhesion detection. Consideration of the three broad agreement classifications in Table 5.1 provides a coarse overview of the level of agreement, as displayed in Figure 5.4. It was the radiologist’s sheargram judgement (not the tech expert’s) that was compared to the original report. This was because the radiologist in this pilot study was the same radiologist who had previously reported the scans (0.5–3 years earlier) and therefore it was most appropriate that his opinion should have greater authority.
Figure 5.4: Overview of the level of agreement between the sheargram and clinical decision/original report for both reporters in the pilot study (some information duplicated from [4])
The distribution in Figure 5.4 indicates that, in general, the sheargram matched the original report and the radiologist’s judgement in the large majority of cases. The percentage of the number of sheargrams which agreed with the radiologist’s final decision was 82% for the radiologist and 78% tech expert. 79% of the radiologist’s sheargram interpretations agreed with the original report. The percentage of cases where the sheargram failed to correlate with the clinical judgement was 11% (15/140) for the radiologist and 10% (14/140) for the tech expert.
0 20 40 60 80 100 120 140
Agreement Partial Agreement Disagreement
N u m b e r o f sag itt al sl ic e s
Number of sheargrams which agree/disagree with presence of
adhesions (broad classifcation)
Radiologist Sheargram compared to Original Report
Radiologist Sheargram compared to Radiologist cMRI decision Tech Expert Sheargram compared to Radiologist cMRI decision
138 The technical expert had a higher proportion of partial agreements. There were more disagreements of the sheargram to the original report than the radiologist’s final adjudication. This indicates that after observing the sheargram and making a final decision on the presence of an adhesion, in some cases the consultant radiologist had altered his opinion to align with what the sheargram was displaying.
Deeper interrogation of the underlying constitution of the broad categories for the agreement between the sheargram and the radiologist’s final decision is presented in Figure 5.5.
Radiologist Technical Expert
Figure 5.5: Correlation between the sheargram and clinically inferred adhesions on the cine-MRI for each of the two reporters represented as pie charts. The numbers on the charts are the number of sagittal slices (total
sagittal slices = 140) (taken from [4])
Only considering the definitive conclusions, i.e. excluding any equivocal cases (i.e. excluding the yellow portions in Figure 5.5), allows the production of three 2x2 contingency tables shown in (Table 5.2 to 5.4) summarising the definitive agreement for each of the three comparisons.
Table 5.2: True and false positives/negatives resulting from the Radiologist’s sheargram interpretation compared to the Original Report
Orig. Report = Adhesion Orig. Report = No Adhesion Radiologist Sheargram Positive 20 17 Radiologist Sheargram Negative 2 90
139 Table 5.3: True and false positives/negatives resulting from the Radiologist’s
sheargram interpretation compared to the Radiologist’s final clinical decision Final decision = Adhesion Final decision = No Adhesion Radiologist Sheargram Positive 23 14 Radiologist Sheargram Negative 1 92
Table 5.4: True and false positives/negatives resulting from the Technical
Expert’s sheargram interpretation compared to the Radiologist’s final clinical decision Final decision = Adhesion Final decision = No Adhesion Tech Expert Sheargram Positive 18 12 Tech Expert Sheargram Negative 2 92
The green highlighted cells of Table 5.2 to 5.4 show true positive and true negative figures while the red cells show the number of false positive and false negative cases. The total number of cases differs between the two reporters due to the increased number of equivocal sheargrams (omitted from Table 5.2 to 5.4) made by the technical expert, as seen in the distribution in Figure 5.4. The vast majority of sagittal slices deemed to contain no adhesions were correctly identified on the sheargram, leading to specificities of 87% for the radiologist and 88% for the tech expert, when excluding all equivocal results. However, these percentages become 81% (92/113) for both reporters when considering equivocal sheargrams as positive identification of adhesions. Sensitivities of 96% and 90% were recorded for the radiologist and tech expert respectively (again, without considering any equivocal findings). This corresponded to 1 adhesion which was not identified on the sheargram for the radiologist and 3 that were missed by the tech expert. If considering equivocal sheargrams as positive adhesions, the percentages of correctly identified adhesions remains at 96% (26/27) for the radiologist and increases to 93% (25/27) for the technical expert.
There were also 12 cases where the radiologist’s final decision on the presence of an adhesion changed relative to the original report. The constitution of the 12 changes are shown in Figure 5.6 below.
140 Figure 5.6: The breakdown of 12 changes made between the report and the radiologist's final decision in the
pilot study [4].
In 2 cases a region previously reported as having an adhesion was adjudicated to not contain one after review of the sheargram. The sheargram potentially raised awareness of adhesions in 7 regions where they had previously not been reported.
5.4 Discussion
This study is the first clinical proof of principle relating to the visceral slide measurement technique and builds an understanding of its potential as a diagnostic aid. The fundamental question related to this exercise is: “is the technique an effective aid to diagnosis?”. This discussion considers several important factors to characterise its effectiveness as a diagnostic aid and is arranged under these headings:
1. Accuracy: First and foremost, the sheargram must correlate with the correct answer. In this pilot study, what proportion of cases did the sheargram agree with clinical opinion? 2. Robustness: How often does the sheargram fail, where did it fail and what were the
reasons for any discrepancies?
3. Influence: Does it have an effect on clinical diagnosis and is this evidenced in the pilot study?
4. Limitations: What are the problems with the current sheargram implementation and what drawbacks have been revealed in this pilot study?
5. Reporting Efficiency: A decrease in reporting time is desirable. Is there evidence to support increased efficiency with the use of the sheargram?
0 1 2 3 4 5 6 7 8
Report: Adhesion -> No Adhesion Report: Adhesion -> Equivocal Adhesion Report: Equivocal -> No Adhesion Report: Equivocal -> Yes Adhesion Report: No Adhesion -> Adhesion Report: No Adhesion -> Equivocal Adhesion
The breakdown of 12 changes made between the report and the radiologist's final decision in the pilot study
141 A discussion on these core points is followed by a critical appraisal of the methodology undertaken in this pilot study.
The radiologist’s clinical judgement made in this pilot study was considered to supersede that in the original report. The original report incorporated information from dynamic transaxial images (routinely included in the clinical scanning protocol, see Section 1.4) but these were not considered in this pilot study. This study intended to see how well the sheargram matched what was observed in the clinical images. The appearance of the sheargram is based solely on information present within the sagittal slice and it is therefore appropriate that it be compared to a clinical judgement of the sagittal slice alone. The bulk of the analysis therefore focuses on comparison of the sheargram to the radiologist’s expert opinion on the cine-MRI (not the original report).