A baseline for assessing biometrics performance robustness: A case study across seven iris datasets

We examine the robustness of algorithm performance over multiple datasets collected with different sensors. This study provides insight as to whether an algorithm performance derived from traditional controlled environment studies will robustly extrapolate to more challenging stand-off/real-world environments. We argue that a systematic methodology is critical in assuring the validity of algorithmic conclusions over the broader arena of applications. We present a structured evaluation protocol and demonstrate its utility by comparing the performance of an open-source algorithm over seven diverse datasets, spanning six different sensors (three stationary, one handheld, and two stand-off). We also provide baseline results for the ranking of the seven datasets as measured by four performance metrics. Finally, we compare our protocol-based ranking with a parallel ranking based on an independent survey of biometrics experts, with high correlation between the two rankings being demonstrated.