Towards expressive Romanian speaking 3D avatars for multimedia interfaces

This paper presents an implemented interface for Romanian language of expressive 3D talking agents, also known as 3D avatars. The major contribution of this work regards adding synchronized 3D lips animation sequences to any given Romanian TTS-generated synthetic word/text. The synchronization is performed using a syllable by syllable approach. The application is based on the Romanian-specific visual speech coarticulation model and on the Romanian logopedics platform for deaf people, both presented in earlier works by the same authors. The proposed Romanian-particular 3D avatar multimedia interface allows users to follow the avatar as it speaks Romanian based on a given text and its associated TTS-generated wave file. The efficiency of the method was successfully validated through several subjective tests, including a large number of normal-hearing testers and using a five level Likert scale to verify whether the speech animations are well-synchronized with the played sound. The results are promising for future 3D avatar live interaction applications for Romanian natives, and also for Romanian language teaching applications.