A method and user interface for instructional video indexing via recognition of handwritten table-of-contents words

Efficient indexing and retrieval of digital videos are important needs within instructional video databases. We construct a word-based video index by spotting words taken from the table of contents (TOC) of the textbooks for the course given in the video. We present a segment-free method of word spotting based on a two step analysis. First, for each candidate word in the vocabulary and each unknown handwritten word image, we search for the location, orientation, and scale of every character of the candidate word; we use a modified Hausdorff distance (MHD) to robustly locate the character´s skeleton in the skeletons of the handwritten word image. Second, we verify words by creating a template image according to the result of character analysis, and by matching this template image to the skeleton of the handwritten word using the MHD. Results for recognition of handwritten words from real videos are shown (greater than 70% accuracy with 100 words). Applying these methods to three contiguous videos (of 25) of a course, we find that they recognize nearly disjoint subsets of highly descriptive indexing keywords.