Association of Audio and Video Segmentations for Automatic Person Indexing

In the audiovisual indexing context, we propose and experiment a method that, from an audio speaker segmentation and a video costume segmentation made on an audiovisual document, makes an automatic association between each voice and the images containing the corresponding visual person. This association can be used as a preprocessing step for existing applications like person identification systems. The first step consists in fusing, without any a priori knowledge, the two indexes produced by audio and video segmentations, in order to make the information brought by each of them more robust. Evaluation is done on a corpus composed of French TV broadcasts. If both audio and video streams are correctly segmented, this automatic association yields excellent results. When the two streams are oversegmented, our system permits to detect the main persons in term of duration of appearance.