Lip-Sync: the Evaluation of Audio-to-Video Timing Errors over Shot Intervals

The relative timing of sound and video components of the television signals is an important point of the viewers´ perception of television programs. There is a complicated problem of a local time-base error at the film dubbing. It needs to measure how well the pronounced words and audio fit the actor´s face. The evaluation of local audio-to-video timing errors over short time intervals is the main goal of current paper. This approach is based on representation of speech as a timed sequence of phonemes, prediction of corresponding visemes and measuring the degree of correlation between real and predicted visemes