Facial feature tracking and expression recognition for sign language

Facial expressions play a vital role in sign language by changing the meaning of the performed sign. In this work, we propose a system composed of a facial feature tracker based on active shape models and a classifier based on hidden Markov models to recognize common facial expressions used in sign language. Face tracker works in multi-resolution and multi-view to track faces in different poses fast and effectively. Detailed tests are prepared to report the accuracy of the tracker and classifier, and it is seen that the system works in high accuracy and robustly.