针对遥感图像跨域分割,提出自适应模块选择方法提升效率与精度。
CrossEarth-Gate: Fisher-Guided Adaptive Tuning Engine for Efficient Adaptation of Cross-Domain Remote Sensing Semantic Segmentation
- 基于费雪信息动态筛选关键模块,精准激活适配层。
- 在18个基准上16项达顶尖水平,显著提升跨域适应性能。
- 适合遥感领域高效微调需求,尤其适用于多源异构数据场景。
在遥感(RS)领域,参数高效微调(PEFT)已成为激活基础模型泛化能力的关键方法。然而,现有专用PEFT方法在大规模地球观测任务中常因无法有效应对多维度且不可预测的域间差异(如空间、语义、频率偏移)而失效。为此,本文提出CrossEarth-Gate,包含两项核心贡献:其一,构建涵盖空间、语义和频率三个维度的综合性遥感模块工具箱;其二,设计基于费雪信息的自适应选择机制,在该工具箱上动态激活最相关的模块。该机制通过费雪信息量化各模块对任务梯度流的贡献,仅在合适层级激活关键模块,以优化梯度传播路径,提升适应效果与效率。大量实验验证了方法的有效性与泛化能力,CrossEarth-Gate在18个跨域遥感语义分割基准中,有16项达到当前最优性能。
原文摘要 · Abstract (English)
In Remote Sensing (RS), Parameter-Efficient Fine-Tuning (PEFT) has emerged as a key approach to activate the generalizable representation ability of foundation models for downstream tasks. However, existing specialized PEFT methods often fail when applied to large-scale Earth observation tasks, as they are unable to fully handle the multifaceted and unpredictable domain gaps (e.g., spatial, semantic, and frequency shifts) inherent in RS data. To overcome this, we propose CrossEarth-Gate, which introduces two primary contributions. First, we establish a comprehensive RS module toolbox to address multifaceted domain gaps, comprising spatial, semantic, and frequency modules. Second, we develop a Fisher-guided adaptive selection mechanism that operates on this toolbox. This selection is guided by Fisher Information to quantify each module's importance by measuring its contribution to the task-specific gradient flow. It dynamically activates only the most critical modules at the appropriate layers, guiding the gradient flow to maximize adaptation effectiveness and efficiency. Comprehensive experiments validate the efficacy and generalizability of our method, where CrossEarth-Gate achieves state-of-the-art performance on 16 out of 18 cross-domain benchmarks for RS semantic segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。