Motion Analysis Based on Spatial-Temporal Visual Attention

Motion analysis can be applied in a number of fields including, e.g. video surveillance, human-computer interaction, smart homes, etc. Due to the large amounts of data associated with usual videos, it is essential for motion analysis algorithms to be computationally efficient. In view of the remarkable efficiency of biological vision systems in dealing with visual information, we propose a motion analysis framework based on spatial-temporal visual attention. More specifically, this paper has adopted a patch-correlation-based approach as the baseline for motion analysis. To achieve computational efficiency, a biologically plausible visual attention model has been adopted, which is based on spatial and temporal features. These features, including intensity, color and motion, are extracted and combined to form salient volumes, i.e. salient spatial-temporal regions in video. During the action matching procedure, only the salient patches or regions are correlated between the query video and the target video, which reduces dramatically the computational cost and improves the robustness of motion analysis. Experimental results of human action detection demonstrate the effectiveness of the proposed framework.