arXiv:2504.19136cs.CVcs.AI2025-04被引 7

通过分离相位与振幅实现多模态遥感图像融合,提升土地覆盖分类精度。

PAD: Phase-Amplitude Decoupling Fusion for Multi-Modal Land Cover Classification

  • 在频域中解耦共享结构与互补信息,避免特征冲突。
  • 在WHU-OPT-SAR和DDHR-SK数据集上达到当前最优性能。
  • 适合关注遥感多模态融合与物理机制建模的研究者。

合成孔径雷达(SAR)与可见光(RGB)图像融合用于土地覆盖分类仍面临模态异质性和光谱互补性未被充分利用的挑战。现有方法常无法分离共享结构特征与模态互补的辐射特性,导致特征冲突与信息损失。为此,我们提出相位-振幅解耦(PAD)框架,该框架在傅里叶域中分离相位(模态共享)与振幅(模态互补)成分,强化共享结构的同时保留互补特性,从而提升融合质量。不同于以往忽略频谱中不同物理属性的方法,PAD显式引入振幅-相位解耦机制。具体包含两个核心组件:1)相位谱校正(PSC),通过卷积引导缩放对齐跨模态相位特征,增强几何一致性;2)振幅谱融合(ASF),利用频率自适应多层感知机动态整合高低频模式,有效利用SAR的形态敏感性与RGB的光谱丰富性。在WHU-OPT-SAR和DDHR-SK数据集上的大量实验表明,PAD达到当前最优性能。本工作为遥感物理感知多模态融合建立了新范式。代码将公开于https://github.com/RanFeng2/PAD。

原文摘要 · Abstract (English)

The fusion of Synthetic Aperture Radar (SAR) and RGB imagery for land cover classification remains challenging due to modality heterogeneity and underexploited spectral complementarity. Existing approaches often fail to decouple shared structural features from modality-complementary radiometric attributes, resulting in feature conflicts and information loss. To address this, we propose Phase-Amplitude Decoupling (PAD), a frequency-aware framework that separates phase (modality-shared) and amplitude (modality-complementary) components in the Fourier domain. This design reinforces shared structures while preserving complementary characteristics, thereby enhancing fusion quality. Unlike previous methods that overlook the distinct physical properties encoded in frequency spectra, PAD explicitly introduces amplitude-phase decoupling for multi-modal fusion. Specifically, PAD comprises two key components: 1) Phase Spectrum Correction (PSC), which aligns cross-modal phase features via convolution-guided scaling to improve geometric consistency; and 2) Amplitude Spectrum Fusion (ASF), which dynamically integrates high- and low-frequency patterns using frequency-adaptive multilayer perceptrons, effectively exploiting SAR's morphological sensitivity and RGB's spectral richness. Extensive experiments on WHU-OPT-SAR and DDHR-SK demonstrate state-of-the-art performance. This work establishes a new paradigm for physics-aware multi-modal fusion in remote sensing. The code will be available at https://github.com/RanFeng2/PAD.

遥感融合多模态学习傅里叶域土地覆盖

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。