Title :
Rule-based semantic summarization of instructional videos
Author :
Liu, Tiecheng ; Kender, John R.
Author_Institution :
Dept. of Comput. Sci., Columbia Univ., New York, NY, USA
Abstract :
We present a new content-based approach to summarize instructional videos. We first redefine "scene" in instructional videos. Focusing on one dominant scene type, that of handwritten lecture notes, we define semantic content as "ink pixels", and present a low-level retrieval technique to extract this content from each frame with consideration of various occlusion and illumination effects. "Key frames" in this video genre are redefined as a set of frames that cover the semantic content, and the fluctuating amount of visible ink is used to drive a heuristic real-time key frame extraction method. A rule-based method is also provided to synchronize key frames with audio. We extend our method to the extraction of key frame hierarchies. We show its application to a 17-minute (30K frames) instructional video sequence, resulting in seven key frames. These techniques create tunable instructional summaries over a wide and dynamically varying range of compression factors.
Keywords :
content-based retrieval; data compression; feature extraction; image retrieval; image sequences; knowledge based systems; video coding; audio; compression factors; content-based approach; handwritten lecture notes; heuristic real-time key frame extraction; illumination effects; ink pixels; instructional video sequence; low-level retrieval technique; occlusion effects; rule-based method; rule-based semantic summarization; semantic content; synchronization; Bandwidth; Cameras; Cinematography; Computer science; Content based retrieval; Ink; Layout; Lighting; MPEG 7 Standard; Video sequences;
Conference_Titel :
Image Processing. 2002. Proceedings. 2002 International Conference on
Print_ISBN :
0-7803-7622-6
DOI :
10.1109/ICIP.2002.1038095