Lipreading from color video

Author

Chiou, Greg I. ; Hwang, Jenq-Neng

Author_Institution

Dept. of Electr. Eng., Washington Univ., Seattle, WA, USA

Volume

6

Issue

8

fYear

1997

fDate

8/1/1997 12:00:00 AM

Firstpage

1192

Lastpage

1195

Abstract

We have designed and implemented a lipreading system that recognizes isolated words using only color video of human lips (without acoustic data). The system performs video recognition using “snakes” to extract visual features of geometric space, Karhunen-Loeve transform (KLT) to extract principal components in the color eigenspace, and hidden Markov models (HMM´s) to recognize the combined visual features sequences. With the visual information alone, we were able to achieve 94% accuracy for ten isolated words

Keywords

eigenvalues and eigenfunctions; feature extraction; handicapped aids; hidden Markov models; image colour analysis; image sequences; motion estimation; speech recognition; transforms; Karhunen-Loeve transform; active contour model; color eigenspace; color video; geometric space; hidden Markov models; human lips; isolated words recognition; lipreading; snakes; video recognition; visual features extraction; visual features sequences; visual phoneme; visual speech recognition; Active contours; Data mining; Feature extraction; Hidden Markov models; Humans; Image motion analysis; Karhunen-Loeve transforms; Lips; Mouth; Speech recognition;

fLanguage

English

Journal_Title

Image Processing, IEEE Transactions on

Publisher

ieee

ISSN

1057-7149

Type

jour

DOI

10.1109/83.605417

Filename

605417