Multimodal biometrics are intended to minimize the weaknesses of individual biometrics by providing more information upon which biometric decisions can be based. They are versatile tools since there are several possible aspects from which their combination can be considered. Some possible methods for combining biometrics are as follows [57]:
1. Measuring the same trait with multiple sensors:
(a) Single trait, multiple sensors (i.e., one finger, multiple fingerprint scanners used in succession). This combination method gathers the same biometric pattern with a series of scanners. The two options at this point are to combine the features of the individual scans into a single feature vector, which is then presented to the pattern classifier. This type of feature fusion technique is often characterized by too many features, making the pattern computationally difficult to classify. Another method is to process the raw data from each scanner into separate feature vectors and present each to a pattern classifier. Then, the scores from each are combined into a single score; this is an example of score-level fusion.
(b) Single trait, multiple classifiers (i.e., present the same fingerprint to more than one pattern classifier and aggregate the results of each into a single decision) This differs from the above method in that only one biometric scan is taken. The raw data from the scan is converted into a feature vector that is then presented to a series of pattern classifiers, each of which outputs a score for the input biomet- ric. These individual results are then combined into a single score upon which a decision can be made.
(c) Single trait, multiple versions (i.e., more than one finger, both irises or both reti- nas, etc). In this combination method, two or more scans of the same type of biometric are taken, but the source of the biometric is different. For instance, a fingerprint of the subject’s index finger and thumb may be taken in two succes- sive scans, and presented separately to pattern classifiers. Then, the scores from the pattern classifiers can be combined into a single score upon which a decision can be made.
In each of these cases, an alternative to combining scores is to combine decisions, as described below.
2.6. Biometrics 28 2. Measuring more than one distinct biometric identifier and combining the results of
individual pattern classification [122, 123]:
(a) Feature Fusion: combines biometric features from two or more different bio- metric patterns by concatenating extracted features into a single larger feature set, which is then presented to the pattern classifier. The two biometrics should be independent of one another; that is, varying one should not result in variations in the other. The new feature vector has a higher dimensionality and should result in a more reliable biometric decision, particularly if a pre–processing step is used to select the most distinctive features from each biometric.
(b) Match Score Fusion: two or more biometrics are presented to pattern classifiers and are assigned a score (but not a decision) that identifies how close the gathered feature vector is to the template vector. The scores are then combined and a decision is made based on the combined scores.
(c) Decision Score Fusion: multiple biometric feature vectors are presented to pat- tern classifiers, and are placed into one of two groups: accept or reject based on the output of the pattern classifier. The individual accept and reject scores are then combined, often with a weighting factor, to output a single accept or re- ject decision. The decision combination can be made based on a majority voting scheme, such as the one described by Zuev and Ivanov [124].
Each of these combination methods has pros and cons, and selection should be made based on the needs and user base of the system under development. One consideration when mak- ing a selection is whether the system will perform identification or verification, as defined in Section 2.6. In general, if a claim of identity is made previously, verification is the cor- rect mode. For instance, a person using a mobile device may be considered to inherently claim the identity of the device owner; in this case, verification rather than identification is performed.
Research into multimodal biometrics was undertaken as part of this work because behavioral biometrics are not expected to provide sufficiently low error rates for authentication [59]. As the research highlighted in this section shows, combining two or more biometrics may improve the error rates when compared to a single biometric. There is a large body of mul- timodal biometric research that uses variations of common biometrics, such as fingerprints, facial recognition and ear shape. This section focuses on research that uses biometric com- bination methods to reduce error rates.
Iwano et al. [125] use a combination of speech patterns and ear shape to authenticate mobile device owners. They selected these two biometrics because voice patterns have many issues such as noisy environments that make this biometric error-prone, but ear shape is relatively
2.6. Biometrics 29 static and thus can be used to increase robustness. Iwano et al. contaminated their audio samples with white noise in order to test the improvements multimodal biometrics provided. They found that combining the two biometrics reduced the error rate from approximately 38% for the individual biometrics to just over 10% for their combination. Iwano et al. did not experiment with the possible transparency of ear shape by taking the ear image while a call was made from the device, likely because devices available at the time of their experiment did not routinely have a camera on the side of the phone that is held to the ear.
In similar work to Iwano et al., Rokita et al. [126] used a mobile device camera to take photos of users’ hand and face. They extracted similar features from each photo to create a single feature vector. They did not compare the error rates of individual and combined biometrics, but found that there is a point where adding more features resulted in a higher error rate. One issue in Rokita et al.’s work is that it is unlikely that the photo pre-processing they perform prior to pattern classification, as well as the classification itself, will be performed on a mobile device due to processor speed and memory limitations. Furthermore, even if the device were capable of this, the amount of time that passes between taking the photos and an authentication decision is likely prohibitive.
In 2005, Fierrez-Aguilar et al. [74] published a comparison of fusion techniques for multi- modal biometrics that was based on the quality of the metric at the time it was gathered. Their study used fingerprints and signatures as the biometrics. Their proposed fusion techniques, while not specific to mobile devices, may be used in a mobile environment. Their technique determined the quality of the biometric at the time it was gathered, and used it to influence the result of the pattern classification by adding a quality factor to the classification formulae. One of the major issues in Fierrez-Aguilar et al.’s work is that the quality of the fingerprints is determined by a human expert as part of their experiment. This was not of concern during their work as they used a corpus of fingerprints that had already been examined for quality. This, however, would not be viable in a production environment, particularly on a mobile device.
Multimodal biometric authentication methods based on facial recognition and voice biomet- rics are common choices [127–132]. Poh and Korczak [132] developed a text-dependent voice biometric, which differs from the text-independent method used for this thesis work. The use of facial recognition implies that the authentication method is explicit, much like the work of Rokita et al. and Iwano et al. Poh and Korczak attempted to simplify the head positioning requirements typical to facial recognition systems by only using features around the eyes of the image, and accept that they will lose important features for the sake of sim- plicity. In addition, the voice analysis uses a spoken password for comparison purposes; this supports the determination that this was an explicit authentication method. Such text- dependency represents a limitation in their work since speaking a password gives attackers a strong advantage, as does the fact that the password is quite short (3 seconds of record-
2.6. Biometrics 30 ing time) since then it may be easier to spoof the intended owner. Poh and Korczak’s work shows that the fusion of these two biometrics improves the error rates seen when using one biometric, which supports the use of multiple biometrics in authentication.
An important consideration when deploying an authentication mechanism is whether the user base for which it is intended will accept and trust the method. To justify the use of biometrics as an authenticator, research has been performed to determine user acceptance of it.