用语义先验融合光学与雷达图像,提升变化检测精度
Prior-guided Fusion of Multimodal Features for Change Detection from Optical-SAR Images

- 基于视觉基础模型的语义先验,自适应融合多模态特征
- 在三个数据集上比当前最优方法平均提升1.47%的交并比
- 首次公开高分辨率极化雷达-光学多类变化检测数据集
多模态变化检测(MMCD)在土地利用监测和城市可持续发展中具有重要应用价值。然而现有方法在跨模态交互与模态特异性特征利用方面存在不足,导致细粒度变化信息建模不充分,影响语义变化的精确识别。为此,本文提出STSF-Net框架,用于光学与SAR图像间的变化检测。该框架联合建模模态特异性与时空共性特征,以增强变化表征:模态特异性特征捕捉真实语义变化信号,时空共性特征抑制成像机制差异引起的伪变化。此外,提出一种基于视觉基础模型获取的语义先验的自适应融合策略,动态调整多模态特征权重。最后构建了首个公开可用的多类别MMCD基准数据集Delta-SN6,包含超高分辨率全极化SAR与光学图像。在Delta-SN6、BRIGHT和Wuhan数据集上的实验表明,所提方法在mIoU上分别优于当前最优方法3.21%、0.87%和1.32%。
原文摘要 · Abstract (English)
Multimodal change detection (MMCD) identifies changed areas in multimodal remote sensing data, demonstrating significant application value in land use monitoring and urban sustainable development. However, literature MMCD approaches exhibit limitations in both cross-modal interaction and exploiting modality-specific characteristics. This leads to insufficient modeling of fine-grained change information, thus hindering the precise detection of semantic changes. To address these problems, we propose STSF-Net, a framework designed for MMCD between optical and SAR images. STSF-Net jointly models modality-specific and spatio-temporal common features to enhance change representations. Specifically, modality-specific features are exploited to capture genuine semantic change signals, while spatio-temporal common features are embedded to suppress pseudo-changes caused by differences in imaging mechanisms. Furthermore, we introduce an optical and SAR feature fusion strategy that adaptively adjusts multimodal feature importance based on semantic priors obtained from visual foundation models. Finally, we introduce the novel Delta-SN6 dataset, the first openly-accessible multiclass MMCD benchmark consisting of very-high-resolution fully polarimetric SAR and optical images. Experimental results on Delta-SN6, BRIGHT, and Wuhan datasets demonstrate that our method outperforms the state-of-the-art by 3.21%, 0.87%, and 1.32% in mIoU, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。