With more detailed spatial information being represented in very-high-resolution (VHR) remote sensing images, stringent requirements are imposed on accurate image classification. Due to the diverse land objects with intraclass variation and interclass similarity, efficient and fine classification of VHR images especially in complex scenes are challenging. Even for some popular deep learning (DL) frameworks, geometric details of land objects may be lost in deep feature levels, so it is difficult to maintain the highly detailed spatial information (e.g., edges, small objects) only relying on the last high-level layer. Moreover, many of the newly developed DL methods require massive well-labeled samples, which inevitably deteriorates the model generalization ability under the few-shot learning. Therefore, in this article, a lightweight shallow-to-deep feature fusion network ((SDFN)-N-2) is proposed for VHR image classification, where the traditional machine learning (ML) and DL schemes are integrated to learn rich and representative information to improve the classification accuracy. In particular, the shallow spectral-spatial features are first extracted and then a novel triple-stage fusion (TSF) module is designed to learn the saliency and discriminative information at different levels for classification. The TSF module includes three feature fusion stages, that is, low-level spectral-spatial feature fusion, middle-level multiscale feature fusion, and high-level multilayer feature fusion. The proposed (SDFN)-N-2 takes the advantage of the shallow-to-deep features, which can extract representative and complementary information from crossing layers. It is important to note that even with limited training samples, the (SDFN)-N-2 still can achieve satisfying classification performance. Experimental results obtained on three real VHR remote sensing datasets including two multispectral and one airborne hyperspectral images covering complex urban scenarios confirm the effectiveness of the proposed approach compared with the state-of-the-art methods.

A Shallow-to-Deep Feature Fusion Network for {VHR} Remote Sensing Image Classification / Liu, Sicong; Zheng, Yongjie; Du, Qian; Bruzzone, Lorenzo; Samat, Alim; Tong, Xiaohua; Jin, Yanmin; Wang, Chao. - In: IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING. - ISSN 0196-2892. - 60:(2022), pp. 541021301-541021313. [10.1109/tgrs.2022.3179288]

A Shallow-to-Deep Feature Fusion Network for {VHR} Remote Sensing Image Classification

Liu, Sicong;Zheng, Yongjie;Bruzzone, Lorenzo;
2022-01-01

Abstract

With more detailed spatial information being represented in very-high-resolution (VHR) remote sensing images, stringent requirements are imposed on accurate image classification. Due to the diverse land objects with intraclass variation and interclass similarity, efficient and fine classification of VHR images especially in complex scenes are challenging. Even for some popular deep learning (DL) frameworks, geometric details of land objects may be lost in deep feature levels, so it is difficult to maintain the highly detailed spatial information (e.g., edges, small objects) only relying on the last high-level layer. Moreover, many of the newly developed DL methods require massive well-labeled samples, which inevitably deteriorates the model generalization ability under the few-shot learning. Therefore, in this article, a lightweight shallow-to-deep feature fusion network ((SDFN)-N-2) is proposed for VHR image classification, where the traditional machine learning (ML) and DL schemes are integrated to learn rich and representative information to improve the classification accuracy. In particular, the shallow spectral-spatial features are first extracted and then a novel triple-stage fusion (TSF) module is designed to learn the saliency and discriminative information at different levels for classification. The TSF module includes three feature fusion stages, that is, low-level spectral-spatial feature fusion, middle-level multiscale feature fusion, and high-level multilayer feature fusion. The proposed (SDFN)-N-2 takes the advantage of the shallow-to-deep features, which can extract representative and complementary information from crossing layers. It is important to note that even with limited training samples, the (SDFN)-N-2 still can achieve satisfying classification performance. Experimental results obtained on three real VHR remote sensing datasets including two multispectral and one airborne hyperspectral images covering complex urban scenarios confirm the effectiveness of the proposed approach compared with the state-of-the-art methods.
2022
Liu, Sicong; Zheng, Yongjie; Du, Qian; Bruzzone, Lorenzo; Samat, Alim; Tong, Xiaohua; Jin, Yanmin; Wang, Chao
A Shallow-to-Deep Feature Fusion Network for {VHR} Remote Sensing Image Classification / Liu, Sicong; Zheng, Yongjie; Du, Qian; Bruzzone, Lorenzo; Samat, Alim; Tong, Xiaohua; Jin, Yanmin; Wang, Chao. - In: IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING. - ISSN 0196-2892. - 60:(2022), pp. 541021301-541021313. [10.1109/tgrs.2022.3179288]
File in questo prodotto:
File Dimensione Formato  
TGRS3179288.pdf

accesso aperto

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 10.08 MB
Formato Adobe PDF
10.08 MB Adobe PDF Visualizza/Apri
A_Shallow-to-Deep_Feature_Fusion_Network_for_VHR_Remote_Sensing_Image_Classification.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 5.58 MB
Formato Adobe PDF
5.58 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/401492
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 24
  • ???jsp.display-item.citation.isi??? 16
social impact