“Clustering by saliency” — Unsupervised discovery of crowd activities

In this paper, we develop a novel unsupervised crowd activity discovery algorithm aiming to automatically explore latent action patterns among crowd activities and partition them into meaningful clusters. Inspired by computational model of human vision system, we present a spatiotemporal saliency-based representation to simulate visual attention mechanism and encode human-focused components in an activity stream. Combining with feature pooling, we could obtain a more compact and robust activity representation. Based on the affinity matrix of activities, N-cut is performed to generate clusters with meaningful activity patterns. We carry out experiments on our proposed HIT-BJUT dataset and another public UMN dataset. The experimental results demonstrate that the proposed unsupervised discovery method is capable of automatically mining meaningful activities from large-scale video data with mixed crowd activities.