Fusion of Multiple Facial Regions for Expression-Invariant Face Recognition

In this paper, we describe a fusion-based face recognition method that is able to compensate for facial expressions even when training samples contain only neutral expression. The similarity metric between two facial images are calculated by combining the similarity scores of the corresponding facial regions, e.g. the similarity between two mouths, the similarity between two noses, etc. In contrast with other approaches where equal weights are assigned on each region, a novel fusion method based on linear discriminant analysis (LDA) is developed to maximize the verification performance. We also conduct a comparative study on various face recognition schemes, including the FRGC baseline algorithm, the fusion of multiple regions by sum rule, and the fusion of multiple regions by LDA. Experiments on the FRGC (Face Recognition Grand Challenge) V2.0 dataset, containing 4007 face images recorded from 266 subjects, show that the proposed method significantly improves the verification performance in the presence of facial expressions.