Multi features hybrid Active Shape Model for automated lip contours tracking in video sequence

We propose and evaluate a novel method for enhancing performance of lips contour tracking, which is based on the concept of active shape models (ASM) and multi features. On the first image of the video sequence, lip region is detected using the Bayesian´s rule in which lip color information is modeled by a Gaussian mixture model (GMM) which is trained by expectation-maximization (EM) algorithm. The lip region is then used to initialize the lip shape model. A single feature-based ASM presents good performance only in particular conditions but gets stuck in local minima for noisy conditions (like beard, wrinkle, poor texture, low contrast between lip and skin, etc). To enhance the convergence, we propose to use 2 features: normal profile and grey level patches, and combine them with a voting approach. The standard ASM is not able to take into account temporal information from previous frames therefore the lip contours are tracked by replacing the standard ASM with a hybrid active shape model (HASM) which is capable to take advantage of the temporal information. Initial experimental results on video sequences show that MF-HASM is more robust to local minimum problem and gives a higher accuracy than traditional single feature-based method in lip tracking problem.