Title :
Recovering audio-to-video synchronization by audiovisual correlation analysis
Author :
Liu, Yuyu ; Sato, Yoichi
Author_Institution :
Inst. of Ind. Sci., Univ. of Tokyo, Tokyo, Japan
Abstract :
Audio-to-video synchronization (AV-sync) may drift and is difficult to recover without dedicated human effort. In this work, we develop an interactive method to recover the drifted AV-sync by audiovisual correlation analysis. Given a video segment, a user specifies a rough time span during which a person is speaking. Our system first detects a speaker region using face detection. It then does a two-stage search to find the optimum AV-drift that can maximize the average audiovisual correlation inside the speaker region. The correlation is evaluated using quadratic mutual information with kernel density estimation. AV-sync is finally recovered by the detected optimum AV-drift. Experimental results demonstrate the effectiveness of our method.
Keywords :
audio signal processing; audio-visual systems; correlation methods; estimation theory; face recognition; image segmentation; interactive systems; search problems; speaker recognition; synchronisation; video signal processing; audio-to-video synchronization; audiovisual correlation analysis; drifted AV-sync recovery; face detection; interactive method; kernel density estimation; quadratic mutual information; speaker region detection; two-stage search; video segment; Cameras; Codecs; Degradation; Delay effects; Face detection; Hardware; Humans; Kernel; Laboratories; Mutual information;
Conference_Titel :
Pattern Recognition, 2008. ICPR 2008. 19th International Conference on
Conference_Location :
Tampa, FL
Print_ISBN :
978-1-4244-2174-9
Electronic_ISBN :
1051-4651
DOI :
10.1109/ICPR.2008.4760963
