Home Video Visual Quality Assessment With Spatiotemporal Factors

Compared with the video programs taken by professionals, home videos are always with low quality content resulted from non-professional capture skills. In this paper, we present a novel spatiotemporal quality assessment scheme in terms of low-level content features for home videos. In contrast to existing frame-level-based quality assessment approaches, a type of temporal segment of video, subshot, is selected as the basic unit for quality assessment. A set of spatiotemporal visual artifacts, regarded as the key factors affecting the overall perceived quality (i.e., unstableness and jerkiness as temporal factors; infidelity, blurring, brightness, and orientation as spatial factors), are mined from each subshot based on particular characteristics of home videos. The relationship between the overall quality metric and these factors are exploited by three different methods, including user study-based, rule-based and learning-based. To validate the proposed scheme, we present a scalable quality-based home video summarization system from a novel perspective-achieving the best visual quality while simultaneously preserving the most informative content. A comparison user study between this system and the attention model-based video skimming approach demonstrated the effectiveness of the proposed quality assessment scheme