Content-Based TV Sports Video Retrieval Based on Audio-Visual Features and Text Information

In this paper, we propose content-based video retrieval, which is a kind of retrieval by its semantical contents. Because video data is composed of multimodal information streams such as visual, auditory and textual streams, we describe a strategy of using multimodal analysis for automatic parsing sports video. The paper first defines the basic structure of sports video database system, and then introduces a new approach that integrates visual streams analysis, speech recognition, speech signal processing and text extraction to realize video retrieval. The experimental results for TV sports video of football games indicate that multimodal analysis is effective for video retrieval by quickly browsing tree-like video clips or inputting keywords within predefined domain.