Performance improvement of face recognition using multispectral images and stereo images

Score fusion combines several scores from multiple matchers and/or multiple modalities, which can increase the accuracy of face recognition. In a real system, high accuracy rate and low false accept rates (FAR) are equally important. In this paper, we propose to improve the accuracy using score fusion of multispectral images and reduce the FAR using decision fusion of stereo images. The stereo face images are taken with two identical cameras aiming at a subject, which include two bands, visible and thermal. Specifically, the score fusion combines the face scores from three selected matchers, face pattern byte, linear discriminant analysis, and elastic bunch graph matching, and from two-band images (visible and thermal). The decision fusion combines the results (genuine or impostor) from left face and right face in stereo imaging. We present the score-fusion results using k-nearest neighbor fusion, and hidden Markov model fusion, and the decision-fusion results using logical rules, OR and AND. Our experiments are conducted with the ASUMSS face dataset that currently consists of the stereo face images of two spectral bands from 55 subjects. The experimental results show that score fusion can significantly improve the accuracy, and the decision fusion (with AND rule) can reduce the FAR with a slight decrease of the accuracy.