An Image Restoration Architecture using Abstract Features and Generative Models

Image restoration and its different variations are important topics in the low-level image processing. One of the main challenges in image restoration is the dependency of the current methods to the corruption characteristics. In this paper, we propose an image restoration architecture that enables us to address different types of corruption regardless of the type, amount, and location. The main intuition behind our approach is to restore original images from the abstracted perceptual features. Using an encoder-decoder architecture, image restoration can be defined as an image transformation task. Abstraction of the perceptual features is done in the encoder part of the model, and determines the sampling point within the Probability Density Function (PDF) of the original images. PDF of the original images is learned in the decoder section using a Generative Adversarial Network (GAN) that receives the sampling point from the encoder part. The pre-trained network extracts the perceptual features, and the Restricted Boltzmann Machine (RBM) makes the abstraction over them in the encoder section. By developing a new algorithm for training RBM, the features of the corrupted images are refined. In the decoder, the generator network restores the original images from the abstracted perceptual features, while the discriminator determines how good the restoration result is. The proposed approach is compared with both traditional approaches like BM3D and with modern deep models like IRCNN and NCSR. We also consider three different categories of corruption including denoising, inpainting, and deblurring. The experimental results confirm the performance of the model.