A Multi-modal Sparse Coding Classifier Using Dictionaries with Different Number of Atoms

Most of classification methods including the ones based on sparse representation (SRC), look at every training sample and its extracted modalities as a single point in a high dimensional space and a collection of these points build the training space used to train the classifier. In a multimodality classification problem, there might be lots of redundancies associated with different modalities of the training data which degrade the performance of the classifier. This paper considers the problem of multi-modality classification in a sparse representation framework which separately looks at each modality space and builds abstract and in the same time, precise models with different number of representatives for each individual modality. An optimization is also introduced to solve the classification problem using these models. In comparison to SRC which directly uses one modality from the training samples, the proposed method utilizes multiple abstract modalities to form efficient and comprehensive representation of the data in order to increase both the accuracy and efficiency of the classification process. Experimental results on face and digit recognition applications show that the proposed method has higher recognition rate compared to single-modality as well as multi-modality methods based on SRC.