Title :
A lip reading application on MS Kinect camera
Author :
Yargic, Alper ; Dogan, M.
Author_Institution :
Comput. Eng. Dept., Anadolu Univ., Eskisehir, Turkey
Abstract :
Hearing-impaired people can read lips and lip reading applications may help them to improve their lip imitation skills. Speech of normal people can be recognized by even cellular phones but lip reading systems using only visual features remain important for hearing-impaired people. This paper aims to develop an application using MS Kinect camera to recognize Turkish color names to be used in the education of hearing-impaired children. Predefined lip points are located with depth information by the MS Kinect Face Tracking SDK. Words are segmented from the speech and the angles between the lip points are used as features to classify the words. Angles are computed using the 3D coordinates of the lip points. The KNN classifier is used to classify the words with Manhattan and Euclidean distances and the best feature vectors are tried to be found. As a result, the isolated words are classified with the success rate of 78.22%.
Keywords :
face recognition; feature extraction; handicapped aids; image classification; image colour analysis; object tracking; speech recognition; 3D coordinates; Euclidean distances; KNN classifier; MS Kinect Face Tracking SDK; MS Kinect camera; Manhattan distances; Turkish color name recognition; cellular phones; depth information; feature vectors; hearing-impaired children education; hearing-impaired people; lip imitation skills improvement; lip reading application; lip reading systems; normal people speech recognition; predefined lip point location; visual features; word classification; word segmentation; Cameras; Face; Mouth; Speech; Speech recognition; Tracking; Visualization; 3D face tracking; K-nearest neighbor classifier; Lip reading; MS Kinect camera; lip activation; lip motion detection;
Conference_Titel :
Innovations in Intelligent Systems and Applications (INISTA), 2013 IEEE International Symposium on
Conference_Location :
Albena
Print_ISBN :
978-1-4799-0659-8
DOI :
10.1109/INISTA.2013.6577656
