Speech-to-visual speech synthesis using Chinese visual triphone

A visual speech synthesis approach using Chinese visual triphone is presented. According to Mandarin Chinese pronunciation principle and the relationship between phoneme and viseme, “Chinese visual triphone” model is constructed. Triphone hidden Markov model is established based on visual triphones. Joint features composed of visual features and audio features are used in the training stage. In the synthesis stage, sentence HMM is constructed by concatenating triphone HMMs. With the features extracted from sentence HMM, visual speech is synthesized. From the scores of subjective and objective estimation, the synthesized video is realistic and satisfactory.