Video storyboards: summarising video sequences for indexing and searching of video databases

Ongoing research into intelligent image databases continues to produce new and innovative techniques for the browsing and retrieval of digital images, but these techniques are generally not applicable to moving image (video) databases. Video sequences, by their very nature, contain a temporal element. This, coupled with the high volume of data needed to represent even small segments of video, makes many of the image database techniques impractical. We have developed a technique which eliminates the temporal content of video, and reduces the amount of data required for its representation, by automatically identifying and extracting key frames in a video sequence. The result is a `storyboard´ consisting of a number of video stills, each one chosen to best represent a shot in the video sequence, together with information about each shot, such as camera motion, start frame, duration, etc. The storyboard provides a pictorial index to the video sequence from which it is generated, and the task of locating a particular sequence in a video database is thus reduced to the task of locating one or more still images from a storyboard database. This can be achieved by image retrieval techniques such as query-by-image content (QBIC). In addition, the storyboard provides a summary of the video sequence, in which the semantic content is retained, but with far greater economy as regards the amount of data and time needed to represent it. Hence any `hit´ found by the database query can be presented to the user as a storyboard rather than the video sequence itself, allowing quick, low bandwidth browsing