AMIGO - automatic indexing of lecture footage

We present AMIGO, an automatic indexer for video presentations which - given an e-lecture and supplementary slides - localizes the exact time and position of each slide displayed in the video footage. This offers richer access to viewers, including a slide-accurate navigation and a text-based interaction with the video. AMIGO is based on a matching of local features between video frames and presentation slides. Our key contribution, however, is the combination of local feature matching with two temporal models (a Hidden Markov Model (HMM) and a simple heuristic filter), exploiting the alignment of the presentation with the reading order of its supplementary material. We demonstrate the effectiveness of our approach in quantitative experiments on a dataset of e-lectures and screencasts, which show - with an average accuracy of over 95% - that the approach works under occlusion and camera motion.