Multi-feature subspace analysis for audio-vidoe based multi-modal person recognition

Biometric person recognition has received a lot of attention in recent years due to the growing security demands in commercial and law enforcement applications. However, using a single biometric has several problems. In order to alleviate these problems, multi-modal biometric systems are proposed by combining various biometric modalities to improve the robustness of person authentication. A typical application is to combine both audio and face for multimodal person recognition, since either face or voice is among the most natural biometrics that people use to identify each other. In this paper, a novel approach called multi-feature subspace analysis (MFSA) is proposed for audio-video based biometric person recognition. In the MFSA framework, each face sequence or utterance is represented with a fix-length feature vector, and then subspace analysis method is performed on a collection of random subspaces to construct an ensemble of classifiers for robust recognition. Experiments on the XM2VTSDB corpus sufficiently validate the feasibility and effectiveness of our new approach.