Politización y Conciencia

A. Risk of Bias

Patient Sampling

Study design: - Case control Data collection:

- Retrospective image selection / Prospective interpretation

Period of data collection 1999-2003 Country Italy

Test set derived For the 3 linear classifiers: lesions from each centre (Rome and Siena) were randomly allocated to training and test sets. Linear classifier 1 was constructed from the Rome tarining set and tested on all lesions from Siena; Linear classifier 2 was constructed on the Siena training set and tested on all lesions from Rome; Linear classifier 3 was constructed on training sets from both centres and tested on test sets from both centers. For the K- nearest-neighbour (K-nn) classifiers, a separate trianing set of lesions were selected from the image databases of several institutions that iused the same ELM instrumentation, i.e., IDI-Rome; Siena University Dermatology Clinic; IDI-Capranica; and the Italian Cancer League Clinics of Grosseto, Livorno, Arezzo, Trento, and Siena. It was then tested on all lesions from both centers as described above.

Was a consecutive or random sample of patients enrolled? Yes

Was a case-control design avoided? No

Did the study avoid inappropriate exclusions? Unclear Could the selection of patients have introduced bias? High risk

B. Concerns regarding applicability

Patient characteristics and setting

Inclusion criteria: Study inclusion criteria All melanomas undergoing ELM and excision at the two centers (1999-2003) and random sample of surgically removed benign melanocytic lesions, inlcuding 85 histologically atypical nevi.

Setting:

- Secondary (general dermatology) Dept Dermatology, University of Siena

- Specialist unit (skin cancer/pigmented lesions clinic) Instituto Dermatopatico dell'Immacolata (IDI), a research hospital for skin diseases in Rome

Prior testing:

- Selected for excision (no further detail) Setting for prior testing:

- Secondary (general dermatology) Siena

- Specialist unit (skin cancer/pigmented lesions clinic) Rome

Exclusion criteria: Study exclusion criteria: None reported

Sample size (patients): NR Sample size (lesions):

- No. included: 821 (475 from Siena; 346 from Rome) Participant characteristics: Participant characteristics Thickness/depth:

- Other: 178 (48% of 372 MM) <=0.75mm Are the included patients and chosen study setting appropriate? No

Did the study avoid including participants with multiple lesions? Unclear Was an adequate spectrum of cases used to train the algorithm? Unclear Are there concerns that the included patients and setting do not

match the review question? High

Index tests

3. Computer Assisted Diagnosis - Dermoscopy based

Derm-CAD system: DB-Mips system (KNN and Linear discrimination classifiers) Classifier selected at random for analysis in review: KNN

System details:

dermoscopy unit, internal steromicroscope, internal DB, pattern analysis system Derivation study (internal validation)

Linear classifier 3 was constructed on training sets from both centres and tested on test sets from both centers. For the K-nearest-neighbour (K-nn) classifiers, a separate trianing set of lesions were

selected from the image databases of several institutions that iused the same ELM instrumentation, i.e., IDI-Rome; Siena University Dermatology Clinic; IDI-Capranica; and the Italian Cancer League Clinics of Grosseto, Livorno, Arezzo, Trento, and Siena. It was then tested on all lesions from both centers as described above.

Discriminant analysis used to identify features for which there was a significant (t test) difference between melanomas and non-melanomas and, within these diagnostic classes, no significant difference between centers. Details provided. Selected variables were: geometric variables - area*, variance of contour symmetry*, fractality of borders*color variables - mean skin-lesion gradient*, variance of border gradient*, and border interruptions*. texture variables - mean contrast and entropy* of lesionislands of color variables - dark area*, blue-gray area*, transition region imbalance*

Lesion characteristics assessed:

38 parameters belonged to four categories (referenced to Soyer 2000): geometries; colors; textures; and islands of color (i.e., color clusters inside the lesion). These were all described in detail.

Additional predictors included: - No further information used Method of diagnosis: - In person diagnosis - CAD-based diagnosis Prior/other test data: - None reported CAD output:

- Diagnosis suggested (e.g. melanoma, benign melanocytic nevus) Diagnostic threshold:

- Threshold not reported

The method of receiver operating characteristic curves was used to identify the threshold value for a fixed sensitivity of 95%

The prevalence of melanomas among the first 100 closest neighbors was determined, and the lesion was assigned to the melanoma group if the prevalence was higher than a threshold value T 100 . The method of receiver operating characteristic curves was used to identify the T 100 value necessary for a sensitivity of 98%"

Computer-assisted diagnosis

B. Concerns regarding applicability

Dermoscopy

A. Risk of Bias

B. Concerns regarding applicability

Reference Standard

A. Risk of Bias

Target condition and reference standard(s)

Reference standard

- Histological diagnosis alone Histology (excision)

- No. patient/lesions: 821 (475 Siena; 346 Rome) - Disease positive: 372 (217 Siena; 155 Rome) - Disease negative: 449 (258 Siena; 191 Rome) TARGET CONDITION (Final diagnoses) - Melanoma (invasive): 302

- Melanoma (in situ): 70 (higher % at Siena than in Rome)

- Severe dysplasia: 85 (architectural disorder and melanocytic atypia)

- Benign naevus: 364 Is the reference standards likely to correctly classify the target

condition? Yes

Were the reference standard results interpreted without knowledge of

the results of the index tests? Yes

Were the reference standard results likely to correctly classify the target

condition (disease negative)? Yes

Could the reference standard, its conduct, or its interpretation have

introduced bias? Low risk

B. Concerns regarding applicability

Was the use of expert opinion (with no histological confirmation) avoided as the reference standard? Yes Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist? Unclear Are there concerns that the target condition as defined by the reference standard does not match the question? Unclear

Flow and Timing

A. Risk of Bias

Flow and timing Participant exclusions: none_{Intervals between tests: not reported} Was there an appropriate interval between index test and reference standard? Unclear

Did all patients receive the same reference standard? Yes Were all patients included in the analysis? Yes Could the patient flow have introduced bias? Unclear risk

Notes

Notes  

Cascinelli 1992