Learning human actions with an adaptive codebook

Learning a compact and yet discriminative codebook for classifying human actions is a challenging problem. One difficulty lies in that the learning procedure is split into two independent phases (dimension reduction and clustering) and thus results in the loss of discriminative information which clustering requires. Besides, traditional used principal component analysis is not optimized for class separability and may not help to improve data separation. In this paper, we propose a novel optimization framework which unifies dimension reduction and clustering. In contrast to previous methods, our method enables to dynamically select indispensable and crucial dimensions for building a discriminative codebook. We add metric learning before clustering to provide the clustering method with an optimized distance metric. Experimental results show that our approach constructs a highly discriminative codebook and achieves comparable results to other state-of-the-art approaches.