Recently, pansharpening methods based on deep learning (DL) have achieved state-of-the-art results. However, current existing DL-based pansharpening methods need to be trained repetitively for different satellite sensors to obtain satisfactory fusion performance and therefore require a large number of training images for each satellite. To deal with these issues, in this article, we propose a unified two-stage spatial and spectral network (UTSN) for pansharpening. A branch of networks is constructed for each different satellite, in which the spatial enhancement network (SEN) is shared to improve the spatial details in the fused images from different satellites. A spectral adjustment network (SAN) is employed to capture the spectral characteristics of the specific satellite. Through SAN, the spectral information in the intermediate image from SEN is refined to produce the final fusion results. Such a framework can integrate the datasets from different satellites together for sufficient training of SEN. The proposed method is able to achieve promising pansharpening results also for a new satellite with limited training images by only learning a new SAN on the few-shot datasets due to the simple but efficient structure of SAN. The experimental results show that the proposed method can produce state-of-the-art fusion results in both the standard and few-shot cases. The source code is publicly available at https://github.com/RSMagneto/UTSN.
A Unified Two-Stage Spatial and Spectral Network With Few-Shot Learning for Pansharpening / Sheng, Zhi; Zhang, Feng; Sun, Jiande; Tan, Yanyan; Zhang, Kai; Bruzzone, Lorenzo. - In: IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING. - ISSN 0196-2892. - 61:(2023), pp. 1-17. [10.1109/TGRS.2023.3281602]
A Unified Two-Stage Spatial and Spectral Network With Few-Shot Learning for Pansharpening
Bruzzone, Lorenzo
2023-01-01
Abstract
Recently, pansharpening methods based on deep learning (DL) have achieved state-of-the-art results. However, current existing DL-based pansharpening methods need to be trained repetitively for different satellite sensors to obtain satisfactory fusion performance and therefore require a large number of training images for each satellite. To deal with these issues, in this article, we propose a unified two-stage spatial and spectral network (UTSN) for pansharpening. A branch of networks is constructed for each different satellite, in which the spatial enhancement network (SEN) is shared to improve the spatial details in the fused images from different satellites. A spectral adjustment network (SAN) is employed to capture the spectral characteristics of the specific satellite. Through SAN, the spectral information in the intermediate image from SEN is refined to produce the final fusion results. Such a framework can integrate the datasets from different satellites together for sufficient training of SEN. The proposed method is able to achieve promising pansharpening results also for a new satellite with limited training images by only learning a new SAN on the few-shot datasets due to the simple but efficient structure of SAN. The experimental results show that the proposed method can produce state-of-the-art fusion results in both the standard and few-shot cases. The source code is publicly available at https://github.com/RSMagneto/UTSN.File | Dimensione | Formato | |
---|---|---|---|
TGRS3266799.pdf
accesso aperto
Tipologia:
Post-print referato (Refereed author’s manuscript)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
5.06 MB
Formato
Adobe PDF
|
5.06 MB | Adobe PDF | Visualizza/Apri |
A_Unified_Two-Stage_Spatial_and_Spectral_Network_With_Few-Shot_Learning_for_Pansharpening.pdf
Solo gestori archivio
Tipologia:
Versione editoriale (Publisher’s layout)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
7.55 MB
Formato
Adobe PDF
|
7.55 MB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione