Cadena de Valor

A. Risk of Bias

Patient Sampling

Study design: Randomised controlled trial, only experimental group included

Data collection: Prospective

Period of data collection March 2008 to May 2010

Country UK Was a consecutive or random sample of patients enrolled? Yes

Was a case-control design avoided? Yes

Did the study avoid inappropriate exclusions? Yes Could the selection of patients have introduced bias? Low risk B. Concerns regarding applicability

Patient characteristics and setting

Inclusion criteria: Adults with any suspicious

pigmented skin lesion, i.e. any lesion presented by a patient, or opportunistically seen by a family doctor or practice nurse, that could not immediately be

diagnosed as benign and about which the patient could not be reassured.

Setting: Primary 15 general practices in eastern England

Prior testing: Clinical suspicion of malignancy without dermatoscopic suspicion

Setting for prior testing: Primary

Exclusion criteria: Those unable to give informed consent or considered inappropriate to include by their family doctor.

Sample size (patients): No. eligible: 1297; No. included: 1293 in RCT, 643 in experimental group Sample size (lesions): No. eligible: 1580; No. included: 1583 in RCT, 788 in experimental group Participant characteristics (whole population): Mean age: 44.6y (SD 16.8). Male: 465 (36%). Ethnicity: White 1214 (93.9%); Mixed 45 (3.5%); Missing: 34 (2.6%)

Lesion characteristics. Lesion thickness ≤1mm: in 'more than half' of MM

Are the included patients and chosen study setting appropriate? Yes Did the study avoid including participants with multiple lesions? No Was an adequate spectrum of cases used to train the algorithm?

Are there concerns that the included patients and setting do not

Index tests

Computer Assisted Diagnosis - Spectroscopy based MSI-CAD system: SIAscope + MoleMate (classifier NR) System details:

SIAscopy with MoleMate (software image management system) viewing platform and integrated primary care scoring algorithm

No derivation aspect (external validation study) Lesion characteristics assessed:

– Lesion characteristics not described Additional predictors included: - None reported

Method of diagnosis:

- In person diagnosis, spectroscopic images (SIAgraphs) - CAD-aided diagnosis

Prior/other test data:

- Clinical history and naked eye examination Operators: 28 clinicians

Operator qualifications: - GP

- Other (describe) 2 nurse practitioners Experience in practice:

- Mixed experience (low and high experience combined) as previously recorded Experience with index test:

- Low experience / novice users CAD output:

- SIAgraphs and Lesion score using Primary Care Scoring Algorithm Diagnostic threshold:

Primary Care Scoring Algorithm (6 or more points regarded as suspicious)

Computer-assisted diagnosis

A. Risk of Bias

Were the index test results interpreted without knowledge of the results of the reference standard? Yes

If a threshold was used, was it pre-specified? Yes

Was the CAD model evaluated in an independent study population? Yes Was model overfitting accounted for during model development?

Could the conduct or interpretation of the index test have introduced bias? Low risk B. Concerns regarding applicability

Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication? No Was the test interpretation carried out by an experienced examiner? Unclear Was the diagnostic threshold to determine presence or absence of disease established in a previously published

study? Yes

Are there concerns that the index test, its conduct, or interpretation differ from the review question? High

Visual inspection

A. Risk of Bias

Were the index test results interpreted without knowledge of the results of the reference standard? Yes

If a threshold was used, was it pre-specified? Yes

Was the CAD model evaluated in an independent study population? Was model overfitting accounted for during model development?

B. Concerns regarding applicability

Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication? Unclear Was the test interpretation carried out by an experienced examiner? Yes Was the diagnostic threshold to determine presence or absence of disease established in a previously published study?

Are there concerns that the index test, its conduct, or interpretation differ from the review question? Unclear

Dermoscopy

A. Risk of Bias

B. Concerns regarding applicability

Reference Standard

A. Risk of Bias

Target condition and reference standard(s)

Reference standard Histological diagnosis plus FU and Epxert opinion

Histology (not further described) 215 (histology result missing in further 4) Disease positive: 35; Disease negative: 180 Clinical FU plus histology of suspicious lesions: 22 of the 411 referred patients were monitored (not further described); 566 of the 1162 not referred underwent expert review and were then re-assessed at 3-6 months Disease positive: 1; Disease negative: 588 Expert opinion. Reviewed by two dermatology experts using the recorded clinical history and examination, a digital photograph, and MoleMate image where available.

Disease positive: 0; Disease negative: 725 Target condition (Final diagnoses)

Melanoma (invasive): 30; Melanoma (in situ): 6; BCC: 10

'Benign' diagnoses: 1306 Is the reference standards likely to correctly classify the target condition? Yes

Were the reference standard results interpreted without knowledge of the

results of the index tests? No

Were the reference standard results likely to correctly classify the target

condition (disease negative)? No

Could the reference standard, its conduct, or its interpretation have

introduced bias? High risk

B. Concerns regarding applicability

Was the use of expert opinion (with no histological confirmation) avoided as the reference standard? No Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist? Unclear Are there concerns that the target condition as defined by the reference standard does not match the question? High

Notes  

Wells 2012

Patient Selection