Automatic face cloning and animation using real-time facial feature tracking and speech acquisition

Author

Goto, Taro ; Kshirsagar, Sumedha ; Magnenat-Thalmann, Nadia

Volume

18

Issue

3

fYear

2001

fDate

5/1/2001 12:00:00 AM

Firstpage

17

Lastpage

25

Abstract

We describe the components of the system used for real-time facial communication using a cloned head. We begin with describing the automatic face cloning using two orthogonal photographs of a person. The steps in this process are the face model matching and texture generation. After an introduction to the MPEG-4 parameters that we are using, we proceed with the explanation of the facial feature tracking using a video camera. The technique requires an initialization step and is further divided into mouth and eye tracking. These steps are explained in detail. We then explain the speech processing techniques used for real-time phoneme extraction and subsequent speech animation module. We conclude with the results and comments on the integration of the modules towards a complete system

Keywords

computer animation; feature extraction; image matching; image segmentation; real-time systems; speech processing; tracking; video cameras; video signal processing; MPEG-4 parameters; automatic face animation; automatic face cloning; cloned head; eye tracking; face model matching; initialization step; interactive virtual environments; mouth tracking; orthogonal photographs; real-time facial communication; real-time facial feature tracking; real-time phoneme extraction; real-time speech acquisition; speech animation module; speech processing techniques; texture generation; video camera; Cloning; Facial animation; Facial features; Feature extraction; Mouth; Neural networks; Real time systems; Signal processing algorithms; Speech; Virtual environment;

fLanguage

English

Journal_Title

Signal Processing Magazine, IEEE

Publisher

ieee

ISSN

1053-5888

Type

jour

DOI

10.1109/79.924885

Filename

924885