MSCD-Net: From Unimodal to Multimodal Semantic Change Detection

Semantic change detection (SCD) involves temporal changes and spatial semantics. Its working principle and processing flow usually include land semantic segmentation (LSS) and binary change detection (BCD). Due to its significant impact and practical value, SCD has received consistently wide attention in Earth observation. Nowadays, remote sensing (RS) data in various modalities are proliferating, calling for an urgent need to develop intelligent algorithms for multimodal RS data. However, no efficient multimodal SCD methods exist currently. To address this limitation, this work proposes the first deep learning-based multimodal SCD method: MSCD-Net. MSCD-Net extracts multiscale semantic and difference features after fusing multimodal features, and then aggregates and refines these features to output high-quality semantic segmentation and change maps. Additionally, a semantic difference decoder (SDD) module is designed to model semantic and difference features jointly. It can be integrated with existing methods to increase accuracy. Experimental results demonstrate that MSCD-Net achieves state-of-the-art performance on both multimodal and unimodal SCD datasets, and SDD has strong feature learning ability and compatibility. These findings imply that MSCD-Net is expected to promote the development and application of multimodal SCD.