A hierarchical framework for movie content analysis: Let computers watch films like humans

In this paper, we specially propose a hierarchical framework for movie content analysis. The purpose of our work is trying to realize computerspsila understanding for movie content, especially ldquowho, what, where, howrdquo which occur in the storyline by imitating human perception and cognition. The framework consists of two hierarchies. As for the low level part, we originally construct the human attention model with temporal information motivated by the Weber-Fechner Law to depict the variation of human perception in multiple modalities. As for the high level part, we focus on semantic understanding of different granularities of videos and simulate human cognition for movie content. Based on this hierarchical framework, we present its applications on semantic retrieval, video summarization and content filter. The promising results of userspsila subjective assessment indicate that the proposed framework is applicable for automatic analysis of movie content by computers.