From a legal standpoint, it is necessary to provide an estimate of the reliability of the nearest neighbor associations to satisfy the requirements of the Daubert Guidelines of evidence admissibility. Both a quantified assessment of each individual’s uniqueness and the likelihood of misidentification using a method are prerequisites to the determination of its reliability (Robertson and Vignaux, 1995). As such, any statement of individual uniqueness is best
presented as the relative probabilities of a match to the correct individual, and to an individual in the general population (Christensen, 2003; Steadman, 2006). I calculated likelihood ratios as a means to make these comparisons.
Likelihood is an estimate of the probability that a hypothesis is true, and in this case, that an association made using this method is correct. The likelihood ratio is the probability of a correct match divided by the probability that it is false (Robertson and Vignaux, 1995). For the sake of continuity, the formulae,
methods, and results of the likelihood ratio calculations are presented in the following paragraphs.
Calculation of likelihood ratios
I calculated likelihood ratios for both the biological and PCFA models to use as measures of the relative reliability of nearest neighbor associations made using each of them. The likelihood ratios used in this research were calculated according to the following equation:
likelihood ratio = positive predictive value .
(1-negative predictive value) where:
positive predictive value = positive matches .
(positive matches + false non-matches) and:
negative predictive value = non-matches .
(false matches + non-matches) Figures 7.9 and 7.10 graphically illustrate the results of the nearest neighbor comparison that are relevant to the calculation of the likelihood ratios for this research for the biological and PCFA models respectively. There are four categories of data presented in Figures 7.9 and 7.10 as represented by boxes a-d. Since there are 115 total matches, and 12,996 total non-matches: box a represents the number of correct nearest neighbor matches, box b contains the number of nearest neighbor matches that were not correct, box c represents the number of times a non-match should have been a match, and box d the number of correct non-matches. The values within boxes a through d
can be inserted into the equations above to calculate the positive and negative predictive values.
Correct Incorrect Totals
Match 112 3 115
Non match 3 12,993 = 12,996
Totals = 115 = 12,996 13,111
Figure 7.9. Values used to calculate the likelihood ratio for the biological model
Correct Incorrect Totals
Match
110 5 = 115
Non match
5 12,991 = 12,996
Totals = 115 = 12996 = 13,111
Figure 7.10. Values used to calculate the likelihood ratio for the PCFA model
Using the values presented in Figures 7.9 and 7.10, the positive
predictive values and negative predictive values used are calculated as follows and the results are presented in Table 7.7.
positive predictive value = a/(a+c) negative predictive value = 1-(d/(b+d))
The likelihood ratios for the biological and PCFA models were then generated from the positive and negative predictive values according to the formulae:
positive predictive value negative predictive value
The positive and negative predictive values and the likelihood ratios for both models are presented in Table 7.7. The likelihood ratios for both models are extremely high, suggesting a very high level of probability that a match made using either is correct. Any likelihood ratio greater than 1 favors the correctness of a match, while a ratio less than one represents evidence against a match, with a likelihood of exactly 1 being neutral. Thus any likelihood ratio greater than 1 favors a match, and the further from 1 the ratio is, the greater the probative value of the evidence (Robertson and Vignaux, 1995). For example a likelihood ratio of 10, means that the two images that constitute a match are 10 times (10:1) more likely to belong to the same individual than two any other person in the general population. There is no necessary upper limit for likelihood ratios, and in the case of forensic identification, it is not unusual to attain ratios of several thousand, meaning that two images are several thousand times more likely to belong to the same individual than to any two other individuals.
TABLE 7.7. Likelihood ratios for the biological and PCFA models
Negative predictive value Positive predictive value Likelihood ratio
Biological model 0.0002308403 0.973913 4218.99
PCFA model 0.0003847331 0.956522 2486.19
Posterior probabilities were calculated for both the biological and PCFA models by dividing the likelihood ratios for each by the likelihood ratio plus one. These values represent the probability that an identification is correct assuming that it is as likely to be correct as incorrect. Thus, posterior probabilities are indications of the reliability of a method. For example a posterior probability of .999 means that the probability of a correct identification given a match would be 99%. The posterior probabilities generated from the present data are presented in Table 7.8.
TABLE 7.8. Posterior probabilities for the biological and PCFA models
Model Posterior probability
Biological model 0.99976803 (99%)
PCFA model 0.99956617 (99%)
Nomograms are useful tools for visualizing the interaction between the likelihood ratio and posterior probability for a particular relationship. Nomograms show pretest probabilities and likelihood ratios along sliding scales that facilitate estimation of the posterior probability that a particular relationship is statistically
meaningful. Their results are generally illustrative rather than exact, but are nevertheless useful in understanding the dynamics of the relationship between likelihood and probability in this research.
Nomograms are presented in Figures 7.11 and 7.12 to illustrate the relationship between the likelihood ratios generated in this research and the probability that nearest neighbor matches made using this method are correct.
The calculator used to generate the nomograms presented in Figures 7.11 and 7.12 (Schwartz, 2006) assumes a prior probability of 1, meaning that
there is no corroborative evidence with regard to the correctness of matches made using this method (this idea is discussed in detail in Chapter VIII). Thus for the biological model, a line drawn from a prior probability value of 1 along the left column, through a likelihood ratio of 4218.99 generates a posterior
probability above 95%. For the PCFA model, a line drawn from a prior
probability of 1 through a likelihood ratio of 2486.19 also generates a posterior probability higher than 95%.
This chapter illustrates the accuracy of the models generated in this research, as well as the potential accuracy with which identifications made be made using this method in the forensic context. These results also illustrate the extent to which the petrous portion of the temporal bone varies between
individuals. The size and regularity of the distinction between repeats and non- repeats, as well as the high likelihood ratios and posterior probabilities indicate a very high level of variation between individuals. These results support the
broader notion suggested by biometrics researchers that individual level variability can be extracted from most segments of human anatomy for which there are a sufficient number reliable landmarks available to quantify it (Maltoni et al., 2003).
Figure 7.11. Nomogram illustrating relationship between likelihood ratio and posterior probability for the biological model
Figure 7.12. Nomogram illustrating relationship between likelihood ratio and posterior probability for the PCFA model
CHAPTER VIII