Consumer video summarization based on image quality and representativeness measure

This paper presents an algorithm for the static summarization of consumer videos. For the fast summary of long and unedited consumer videos, we propose a two-step approach where the first step is for pruning the video and the second for content-aware clustering with keyframe selection. More specifically, the first step employs the conventional spectral clustering method with simple color histogram features for removing most redundant frames. This step yields a condensed video that is shorter and has clearer temporal boundaries than the original video, so that temporal segmentation can be easily performed. In the second step, refined clustering is performed for each of the temporal segments by the image classification method based on the sparse coding of SIFT features. The representativeness of each frame is defined from these coding results, which is one of the measures for the keyframe selection. We also formulate the quality measures of each frame such as contrast, blur, and image skew. The main problem is finding the frames that have both representativeness and high quality, which is formulated as an optimization problem. Experimental results on videos with various lengths show that our video summaries closely follow the important contents of videos.