Mid-level parts mined by feature selection for action recognition

This paper develops a method to learn very few discriminative part detectors from training videos directly, for action recognition. We hold the opinion that being discriminative to action classification is of primary importance in selecting part detectors, not just intuitive. For this purpose, part selection based on feature selection is proposed, employing SVM method. Firstly, large number of candidate detectors are trained using k-means and Exemplar-LDA techniques in whitened feature space. Secondly, each candidate part detector is regarded as a visual feature, so that detector selection can be achieved by feature selection. Detectors with larger weight, indicating more discriminative, will be selected. Meanwhile, to keep space-volume structure information, we use the novel method saliency-driven pooling to form feature primitives which are concatenated into mid-level feature vector. Finally, we conduct experiments on three challenging action datasets (KTH, Olympic Sports, HMDB51) and the results outperform the state-of-the-art.