Title :
A New Pan-Sharpening Method With Deep Neural Networks
Author :
Wei Huang ; Liang Xiao ; Zhihui Wei ; Hongyi Liu ; Songze Tang
Author_Institution :
Sch. of Comput. Sci. & Eng., Nanjing Univ. of Sci. & Technol., Nanjing, China
Abstract :
A deep neural network (DNN)-based new pansharpening method for the remote sensing image fusion problem is proposed in this letter. Research on representation learning suggests that the DNN can effectively model complex relationships between variables via the composition of several levels of nonlinearity. Inspired by this observation, a modified sparse denoising autoencoder (MSDA) algorithm is proposed to train the relationship between high-resolution (HR) and low-resolution (LR) image patches, which can be represented by the DNN. The HR/LR image patches only sample from the HR/LR panchromatic (PAN) images at hand, respectively, without requiring other training images. By connecting a series of MSDAs, we obtain a stacked MSDA (S-MSDA), which can effectively pretrain the DNN. Moreover, in order to better train the DNN, the entire DNN is again trained by a back-propagation algorithm after pretraining. Finally, assuming that the relationship between HR/LR multispectral (MS) image patches is the same as that between HR/LR PAN image patches, the HR MS image will be reconstructed from the observed LR MS image using the trained DNN. Comparative experimental results with several quality assessment indexes show that the proposed method outperforms other pan-sharpening methods in terms of visual perception and numerical measures.
Keywords :
geophysical image processing; geophysical techniques; image coding; image denoising; image fusion; image reconstruction; image representation; image resolution; neural nets; numerical analysis; remote sensing; DNN; HR MS image reconstruction; MSDA series; back-propagation algorithm; deep neural network; high-resolution image patches; high-resolution multispectral image patches; high-resolution panchromatic images; low-resolution image patches; low-resolution multispectral image patches; low-resolution panchromatic images; model complex relationships; modified sparse denoising autoencoder algorithm; nonlinearity; numerical measures; pan-sharpening method; quality assessment indexes; remote sensing image fusion problem; representation learning; visual perception; Image fusion; Image reconstruction; Neural networks; Remote sensing; Spatial resolution; Training; Deep neural networks (DNNs); multispectral (MS) image; pan-sharpening; panchromatic (PAN) image;
Journal_Title :
Geoscience and Remote Sensing Letters, IEEE
DOI :
10.1109/LGRS.2014.2376034