arXiv:2504.06220cs.CV2025-04AAAI被引 5

针对遥感图像伪影问题,提出频域混合适配方法提升模型性能。

Earth-Adapter: Bridge the Geospatial Domain Gaps with Mixture of Frequency Adaptation

  • 通过离散傅里叶变换分解特征频域成分,分离伪影与原始信息。
  • 采用混合专家架构动态加权不同频段适配器,提升鲁棒性。
  • 在遥感语义分割任务中,相比基线提升9.0%(域适应)和3.1%(域泛化)mIoU。

参数高效微调(PEFT)可将强大基础模型适配至多样化下游任务,同时保留并释放其固有能力。然而,现有主要面向自然图像设计的PEFT方法在遥感(RS)场景中表现不佳,主因是难以处理图像中的伪影干扰,此类问题在遥感特征中尤为严重。为此,我们提出Earth-Adapter,首个专为攻克遥感伪影设计的PEFT方法。Earth-Adapter引入新颖的频域混合适配机制,结合混合专家(MoA)与离散傅里叶变换(DFT)。通过DFT,模型可将特征分解为不同频率分量,精准分离伪影与原始内容;随后MoA动态分配各适配器专家权重,实现多频段特征融合。该方法显著提升基础模型在遥感任务中的抗干扰能力,实验表明在域适应(DA)和域泛化(DG)语义分割基准上,相较基线Rein分别提升9.0%和3.1% mIoU。代码将开源于https://github.com/VisionXLab/Earth-Adapter。

原文摘要 · Abstract (English)

Parameter-Efficient Fine-Tuning (PEFT) is a technique that allows us to adapt powerful Foundation Models (FMs) to diverse downstream tasks while preserving and unleashing their inherent capabilities. However, we have observed that existing PEFT methods, which are often designed with natural imagery in mind, struggle when applied to Remote Sensing (RS) scenarios. This is primarily due to their inability to handle artifact influences, a problem particularly severe in RS image features. To tackle this challenge, we introduce Earth-Adapter, the first PEFT method specifically designed for RS artifacts conquering. Earth-Adapter introduces a novel Mixture of Frequency Adaptation process that combines a Mixture of Adapter (MoA) with Discrete Fourier Transformation (DFT). By utilizing DFT, Earth-Adapter can decompose features into different frequency components, precisely separating artifacts from original features. The MoA then dynamically assigns weights to each adapter expert, allowing for the combination of features across various frequency domains. These simple-yet-effective approaches enable Earth-Adapter to more efficiently overcome the disturbances caused by artifacts than previous PEFT methods, significantly enhancing the FMs' performance on RS scenarios. Experiments on Domain Adaptation (DA), and Domain Generalization (DG) semantic segmentation benchmarks showcase the Earth-Adapter's effectiveness. Compared with baseline Rein, Earth-Adapter significantly improves 9.0% mIoU in DA and 3.1% mIoU in DG benchmarks. Our code will be released at https://github.com/VisionXLab/Earth-Adapter.

遥感图像参数高效微调频域分析语义分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。