Image based tracking of news stories

Broadcasters often reuse video clips while reporting news stories. New clips are added to the story as it develops replacing old clips. A subset of these video clips may be replayed as follow up stories are investigated. This pattern of video repetition can be used to impose a story level organization on video clips that are broadcast by one or more news channels. In this paper, we describe an automated system for detecting and tracking repeated video clips in news broadcasts. We begin by performing temporal video segmentation to divide the video into shots and scenes. As each frame of the video source is processed, we extract low-level video features that are used to perform repeated sequence detection in realtime. Our matching algorithms have been adapted to recognize partial clip reuse while remaining robust to minor variations in the video source. Our system then builds a set of shots relevant to the news story being tracked, called a story core, and identifies new story episodes by detecting occurrences of shots belonging to this set. The story core is allowed to evolve dynamically over time to account for additional video footage as it becomes available. This technique was evaluated on a 24-hour broadcast of CNN News channel where we were very successful locating and tracking video clips associated with selected news stories.