Lip Detection and Tracking Using Variance Based Haar-Like Features and Kalman filter

Speaker lip motion stands out as the most linguistically relevant visual feature for speech recognition. Lip reading is an active field that receives much attention from computer scientists. Its applications take part not only in science, such as a speech recognition system, but also in social activities, such as teaching pronunciation for deaf children in order to recover their speaking ability, and combining visual features and audio features to increase the accuracy in noisy environments. In this paper, we aim to solve a narrower problem, the lip detection and tracking, which is an essential step to provide visual lip data for the lip-reading system. A new approach is proposed to perform the problem of real-time lip detection. The proposed method combines primitive Haar-Like feature and variance value to construct a new feature, so-called Variance based Haar-Like feature. Human face and lip can be represented with a small quantity of features using the new features. We used SVM for training and classification, combining kalman filter can realize lip real-time detection and tracking. Experiments showed that face and lip detection system using Variance based Haar-Like feature and SVM can be much more efficient than face and detection system by using primitive Haar-Like features.