A Hierarchical Generative Model for Overcomplete Topographic Representations in Natural Images

In this paper we propose a hierarchical generative model based on sparse coding and analysis of topographic energy dependencies. We further formulate the basic sparse coding into a hierarchical fashion by defining a higher-order topography on the coefficients of nearby basis functions. An algorithm for learning overcomplete topographic basis functions is derived from a direct approximation to the data likelihood. The basis functions learned by the algorithm demonstrate the topographic organization and the emergence of phase-and shift-invariant features - the similar properties of visual complex cells. Moreover, the proposed model yields overcomplete representations. We apply the model to the problem of image denoising. This task suits the model well since Gaussian additive noise is explicitly included in the model. The simulation results suggest that the proposed method outperforms conventional denoising algorithms. Our model is promising in a wide range of fields, such as signal processing and pattern recognition.