Speech rhythm-driven gesture animation

Gesticulation is an essential component of face-to-face communication, and it contributes significantly to the natural and affective perception of human-to-human communication. In this work, we investigate a new multimodal analysis framework to model the relationship between speech rhythm and gesture phrases. We extract speech rhythm using Fourier analysis of the amplitude envelope of bandpass filtered speech rather than computing rhythm with time domain measurements of interval durations, a frequency domain representation is employed. The speech rhythm driven gesture animation framework effectively associates gesture phrases to speech phrases, while maintaining the realistic gesture rhythm. We evaluate the speech rhythm driven gesture animation by subjective tests.