Unsupervised feature learning using Markov deep belief network

Recently, deep architectures, such as Deep Belief Network (DBN), have been used to learn features from unlabeled data. However, since DBN supports bi-directional inference and the units between two layers are fully connected, it is difficult to directly apply the traditional convolutional network to DBN, or scale DBN to fit the large images (e.g. 1024×768). In this paper, a new deep learning model, named Markov DBN (MDBN), is proposed to address these problems. This model employs a new way for DBN to reduce computational burden and handle large images. Markov sub-layers are also adopted to take the neighboring relationship of the inputs into consideration. To train MDBN, we devise Block Restricted Boltzmann Machine (BRBM) which chooses non-overlapping blocks as input. Furthermore, SIFT descriptor is employed to enable this model to learn translation, scaling and rotation invariant features. Experimental results on datasets Caltech-101 and Caltech-256 have demonstrated the superiority of our model.