A face biometric benchmarking review and characterisation

In order to advance face recognition research, algorithm performance has to be measured and compared using a range of metrics and operating characteristics. While public challenges such as the NIST-sponsored FERET, FRVT, FRGC, and MBGC are helpful to gauge comparative performance and improvement for a particular scenario, they typically are not sufficient to fully characterise the strengths and weaknesses of the face recognition algorithm, thus researchers need to do additionally benchmarking independently. This paper provides: (1) a detailed review and categorisation of publicly available face biometrics benchmarks; (2) a discussion of metrics and performance factors to consider; (3) a proposal for a meta-face biometric benchmarking regime which suggests guidelines for benchmarking across multiple datasets to more fully characterise and quantify face recognition performance across various operating characteristics; and (4) a sample demonstration which compare the performance of a face recognition algorithm before and after inclusion of a face quality metric.