Tag-aware image classification via Nested Deep Belief nets

With the rising of internet photos-sharing web sites, the rich aware text information surrounding images on the sites are proved helpful to improve the image classification. This paper presents a novel nested deep learning model called Nested Deep Belief Network(NDBN) for tag-aware image classification. A multi-layer structure of Deep Belief Network(DBN) is established to learn a unified representation of visual feature and tag feature for an image, and an additional Gaussian Restricted Boltzmann Machine is built to capture the tag-tag dependency. Compared with conventional methods, the proposed model can not only find correlations across modalities, but mine the importance for different tags, and also bring about low-rank tag feature representation. We conduct experiments over the MIR Flickr dataset and the results show that the proposed NDBN model outperforms the existing image classification techniques.