Audio-Video Person Authentication Based on 3D Facial Feature Warping

In this paper we propose a novel feature warping technique based on thin-plate-spline (TPS) analysis for 3D audio-video person authentication systems. The TPS warp features model information related to non-rigid variations on speaking faces, such as expression lines, gestures, and wrinkles, enhancing the performance of the system against impostor and spoof attacks. Experiments with multimodal fusion of acoustic and TPS shape features for two different speaking face data corpus, VidTIMIT and AVOZES, allowed equal error rates (EERs) of less than 0.5 % for imposter attacks, less than 1 % for type-1 replay attacks (still photo and pre-recorded audio) and less than 2% for more complex type-2 replay attacks (pre-recorded video or fake CG animated video).