Post-processing of unsupervised dictionary learning in handwritten digit recognition

To achieve high performance in object recognition, a high-level feature representation is play an essential role to transform a raw input data (low-level) into a new representation. Unsupervised feature learning is one of the most successful methods that is widely used in machine learning literatures for creating a high-level feature to improve the supervised learning problems. The main concept of unsupervised feature learning is, in some sense, to encode input knowledge for gaining not only latent features but compact high-fidelity representation. In particular, a sparse coding has proven to be an effective tool as a prior lead to state-of-the-art performance in many benchmark datasets. In sparse coding, an input data can be represented as a sparse linear combination of a set of training overcomplete dictionary. However, an open problem in sparse coding is how to create and correctly choose the dictionary for representing a given input. Moreover, some bases of the learned dictionary is highly dependent and non-orthogonal among itself. In this study, we propose a post-processing scheme to transform the overcomplete dictionary obtained from unsupervised feature learning using Principle Component Analysis (PCA). The goal of our post-processing is to reduce the number of bases in the unsupervised dictionary learning for real-time applications. In experimental results, while the performance without post-processing the unsupervised dictionary learning is better than that of using PCA, it takes more computation and requires more resources during the test time.