Audio-visual information fusion for SVM-based biometric verification

The goal of this paper is to develop a reliable multi-modal biometric verification system based on speech and face information. In this paper, we propose two SVM-based multi-modal biometric verification systems. The first system is based on the concatenation fusion with SVM classifier. This method concatenates the features provided by each expert to form a new feature vector. Then the SVM classifier is trained from the concatenated feature vectors for each person and later used for verification. To determine the final verification result from several SVM classification results for many possible paired audio-visual concatenated feature vectors, we present a new scheme for computing confidence weight based on the distance between feature vector and the hyper-plane of the associated SVM model. The final verification is determined from the weighted sum of all the SVM classification results. Experimental results on the same audio-visual database for the three biometric verification systems are given to compare their performance. We show that the proposed SVM-based fusion systems outperform the traditional GMM-based opinion fusion system.