Multi-class classification strategies for Fisher scores of gesture and sign sequences

In this work, we propose a multi-class classification strategy based on Fisher kernels. Fisher kernels combine the powers of discriminative and generative classifiers by mapping variable-length sequences to a new fixed length feature space. The mapping is based on a single generative model and the classifier is intrinsically binary. We apply a multi-class classification, instead of a binary classification, on each Fisher score space and combine the decisions of multi-class classifiers. We show, through experiments on gesture and sign sequences, that the Fisher scores extracted from the HMM of one class provide discriminative information for other classes as well. Comparisons with other strategies show that the proposed method enhances the performance of the base classifier the most.