A method of simulating the fusion of speech and facial expression for individuals based on the GFFD method

This paper proposes a novel method of simulating the combination of speech and facial expression with anatomical basis, which can cope with any individual faces without any expensive measuring machines. In our previous paper, we presented a method of modeling muscle models around the mouth, especially the orbicularis oris model and the ones supporting it, which play important roles to decide the movement of skin around the mouth when we speak. The facial muscle models are represented as wire-frame models and their vertices play an important role when we treat them as operation points when applying our GFFD (Generalized Free Form Deformation) method. We experimentally decided the position of operation points of those muscle models when we utter the vowel /a/, /i/, /u/, /e/, /o/ and /n/ and consonants on the generic model. In this paper, we first report the improvement on the above method by considering jaw rotation, etc. Then, we present a new method to represent the fusion of the skin motions caused by both speech and facial expression. Some examples will finally be shown.