Title :
Constrained speaker diarization of TV series based on visual patterns
Author :
Bost, Xavier ; Linares, Georges
Author_Institution :
LIA, Univ. of Avignon, Avignon, France
Abstract :
Speaker diarization, usually denoted as the “who spoke when” task, turns out to be particularly challenging when applied to fictional films, where many characters talk in various acoustic conditions (background music, sound effects...). Despite this acoustic variability, such movies exhibit specific visual patterns in the dialogue scenes. In this paper, we introduce a two-step method to achieve speaker diarization in TV series: a speaker diarization is first performed locally in the scenes detected as dialogues; then, the hypothesized local speakers are merged in a second agglomerative clustering process, with the constraint that speakers locally hypothesized to be distinct must not be assigned to the same cluster. The performances of our approach are compared to those obtained by standard speaker diarization tools applied to the same data.
Keywords :
pattern clustering; speaker recognition; video signal processing; TV series; acoustic conditions; acoustic variability; agglomerative clustering process; constrained speaker diarization; fictional films; hypothesized local speakers; scene detection; visual patterns; who spoke when task; Abstracts; Context; Histograms; Image segmentation; Motion pictures; Motion segmentation; Streaming media; Speaker diarization; kagglomerative clustering; video structuration;
Conference_Titel :
Spoken Language Technology Workshop (SLT), 2014 IEEE
DOI :
10.1109/SLT.2014.7078606