A methodology for separating sheep from goats for controlled enrollment and multimodal fusion

Biometric performance assessment is made difficult by virtue of the fact that each user in the database introduces variability that cannot be controlled even with a well designed acquisition procedure and experimental protocol. As a result, the system performance is inevitably user-dependent. We propose explicitly to rank the users according to their performance using criteria such as the F-ratio, the Fisher ratio and the d-prime metric. These criteria are demonstrated to be able to partition the users in such a way that the performance of each partition differs by as much as a factor of 2. Thanks to these criteria, it is possible to assess the performance of the best case or, more importantly, the worst case scenario. While the experiments have been conducted only on face, fingerprint and iris biometrics, we conjecture that such performance discrepancy among the population of users in the same database is exhibited by all biometrics. We also explore various research avenues in this direction, including group-specific score normalization, model adequacy at enrollment and multimodal fusion controlled by a user-ranking criterion.