Human Vs. Automatic Measurement of Biometric Sample Quality

Biometric systems are designed to identify a person based on physiological or behavioral characteristics. In order to predict the utility of a particular image for identification, there is an interest in measures to calculate the biometric image quality. Such measures often assume (implicitly or explicitly) that human image quality evaluations are a gold standard. In order to test this assumption, we measured biometric image quality for face and iris recognition by 8 human volunteers and from 6 face recognition and 1 iris recognition algorithm. Algorithm quality measures were based on a log-linear fit of quality to genuine score values. Results indicate that human quality scores correlate strongly with each other (r=0.723 (iris), r=0.613 (face), p<0.001). Algorithm scores also correlate strongly with each other (r=0.534, p<0.001 (face)). However, human quality scores do not correlate with those from algorithms (r=0.234 (face), r=0.175 (iris))