Novel Cross-Resolution Feature-Level Fusion for Joint Classification of Multispectral and Panchromatic Remote Sensing Images

With the increasing availability and resolution of satellite sensor data, multispectral (MS) and panchromatic (PAN) images are the most popular data that are used in remote sensing among applications. This article proposes a novel cross-resolution hidden layer feature fusion (CRHFF) approach for joint classification of multiresolution MS and PAN images. In particular, shallow spectral and spatial features at a global scale are first extracted from an MS image. Then, deep cross-resolution hidden layer features extracted from MS and PAN are fused from patches at a local scale according to an autoencoder (AE)-like deep network. Finally, the selected multiresolution hidden layer features are classified in a supervised manner. By taking advantage of integrated shallow-to-deep and global-to-local features from the high-resolution MS and PAN images, the cross-resolution latent information can be extracted and fused in order to better model imaged objects from the multimodal representation and finally increase the classification accuracy. Experimental results obtained on three real multiresolution datasets covering complex urban scenarios confirm the effectiveness of the proposed approach in terms of higher accuracy and robustness with respect to literature methods.