Social Indexing of TV Programs: Detection and Labeling of Significant TV Scenes by Twitter Analysis

Technology to analyze the content of TV programs, especially the extraction and annotation of important scenes and events within a program, is beneficial for users to enjoy recorded programs. In this paper, we propose a method of detecting significant scenes in TV programs and automatically annotating the content of the extracted scenes through Twitter analysis. Experiments conducted on baseball games indicate that the proposed method is capable of detecting major events in a baseball game with an accuracy of 90.6%. Moreover, the names of persons involved in the events were detected with an accuracy of 87.2%, and labels describing the event were applied with an accuracy of 66.8%. The proposed technology is very helpful, because it enables users to skip to the highlights of a recorded program.