Profile Lip Reading for Vowel and Word Recognition

This paper focuses on the profile view, which is the second most typical angle after the frontal face, and proposes a profile view lip reading method. We applied the normalized cost method to detect profile contour. Five feature points, the tip of the nose, upper lip, lip corner, lower lip, and chin, were detected from the contour, and eight features obtained from the five feature points were defined. We gathered two types of utterance scenes, five Japanese vowels and 20 Japanese words. We selected 20 combinations based on the eight features and carried out recognition experiments. Recognition rates of 99% for vowel recognition and 86% for word recognition were obtained with five features: two lip heights, two protrusion lengths, and one lip angle.