Adaptive feature selection and data pruning for 3D facial expression recognition using the Kinect

This paper is concerned with the automatic recognition of pose-varying facial expressions from a low-resolution three-dimensional data sensor, the Kinect. We introduce a novel, comprehensive framework that employs Delaunay triangulation and a pool of Distance Metrics (DM) for feature extraction. Our recognition approach utilizes binary Radial Basis Function (RBF) Support Vector Machines (SVMs) on DM-based feature matrices. Optimal class models are then chosen automatically, and then utilized in the testing stage. We have trained and tested our system using two Kinect-based datasets, with results reaching an average accuracy of 95.1% for non-frontal poses, and more than 98% for frontal poses. Our experimental results show that automatically tuned DMs for each class outperform a fixed DM approach for all classes, especially with non-frontal poses. In addition to the design of the overall framework, this paper describes the effect of training data pruning, providing insights that could contribute to the reduction of training times for very large datasets.