DocumentCode :
2150690
Title :
MikeTalk: a talking facial display based on morphing visemes
Author :
Ezzat, Tony ; Poggio, Tomaso
Author_Institution :
Center for Biol. & Comput. Learning, MIT, Cambridge, MA, USA
fYear :
1998
fDate :
8-10 Jun 1998
Firstpage :
96
Lastpage :
102
Abstract :
We present MikeTalk, a text-to-audiovisual speech synthesizer which converts input text into an audiovisual speech stream. MikeTalk is built using visemes, which are a set of images spanning a large range of mouth shapes. The visemes are acquired from a recorded visual corpus of a human subject which is specifically designed to elicit one instantiation of each viseme. Using optical flow methods, correspondence from every viseme to every other viseme is computed automatically. By morphing along this correspondence, a smooth transition between viseme images may be generated. A complete visual utterance is constructed by concatenating viseme transitions. Finally, phoneme and timing information extracted from a text-to-speech synthesizer is exploited to determine which viseme transitions to use, and the rate at which the morphing process should occur. In this manner, we are able to synchronize the visual speech stream with the audio speech stream, and hence give the impression, of a photorealistic talking face
Keywords :
audio-visual systems; computer animation; face recognition; image sequences; speech synthesis; MikeTalk; audio speech stream; audiovisual speech stream; complete visual utterance; human subject; input text; instantiation; morphing process; morphing visemes; mouth shapes; optical flow methods; phoneme information; photorealistic talking face; recorded visual corpus; smooth transition; talking facial display; text-to-audiovisual speech synthesizer; text-to-speech synthesizer; timing information; viseme images; viseme transitions; visual speech stream; Auditory displays; Humans; Image converters; Image motion analysis; Mouth; Optical recording; Shape; Speech synthesis; Streaming media; Synthesizers;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Computer Animation 98. Proceedings
Conference_Location :
Philadelphia, PA
ISSN :
1087-4844
Print_ISBN :
0-8186-8541-7
Type :
conf
DOI :
10.1109/CA.1998.681913
Filename :
681913
Link To Document :
بازگشت