Lip contour tracking using multiple dynamic models on a manifold

This paper presents a method for tracking non-rigid lip contours on video sequences, when the subject exhibits different emotions. The method is based on multiple dynamic models, which are well suited for the multiple-emotion case. Nonlinear dimensionality reduction is performed using the Gaussian Process Multiple Local Models manifold learning method, taking advantage of the time-order of the samples which provides valuable time information for tracking purposes. The method uses multiple charts which allows arbitrary manifold topology. This is accomplished by decomposing the manifold into multiple local models that are combined in a probabilistic fashion using Gaussian process regression. Furthermore, a multiple filter bank architecture is applied in the reduced-dimensionality manifold domain, based on standard filtering methods (e.g., Kalman and particle filtering). The performance of this approach is illustrated in the extended Cohn-Kanade (CK+) database, where the method achieves remarkable accuracy in lip contour tracking with a wide range of emotions.