Real-time lip reading system for fixed phrase and its combination

This paper presents a high-level real-time lip reading system that can recognize both fixed phrase and its combination. Lip reading provides an important means for realizing a communication support interface for speech handicaps. The approach is based on the Viola-Jones face detector, lip extraction based on active appearance model, automatic utterance section detector, and phrase classifier using DP matching. Our system has two unique ideas; combined phrase recognition, and remove false recognition phrase from a target list. These ideas are simple but efficient in practical use. We demonstrated our system with five subjects and confirmed the usefulness.