arXiv:2508.01731cs.CV2025-08被引 34

让现有遥感模型轻松处理多光谱数据,不需重训

SpectralX: Parameter-efficient Domain Generalization for Spectral Remote Sensing Foundation Models

  • 用两阶段微调框架,只改少量参数适应新光谱数据
  • 在多个区域和季节数据上实现跨域泛化,精度提升显著
  • 适合想快速适配遥感模型到新场景的研究者使用

近期遥感基础模型(RSFMs)取得重要进展,但多数基于光学影像预训练,多光谱/高光谱数据仍缺乏相应基础模型。为利用光谱影像在地球观测中的优势,我们探索如何在不进行大规模光谱预训练的前提下,有效适配现有RSFMs以处理多种光谱模态。为此,提出SpectralX:一种参数高效的微调框架,以现有RSFMs为骨干网络,采用两阶段训练策略处理不同光谱输入,显著提升跨域泛化性能。第一阶段通过掩码重建任务,设计专用的超令牌生成器(HyperT),从空间与光谱维度提取属性令牌;同时构建面向属性的适配器混合(AoMoA),动态聚合多属性专家知识并执行层间微调。第二阶段以语义分割为下游任务,在第一阶段框架中插入属性优化适配器(Are-adapter),通过迭代查询低层语义特征与高层表示,使模型聚焦于任务有益属性,实现对RSFMs的定制化调整。经此两阶段适配,SpectralX可解析新区域或季节的光谱影像。代码将发布于https://github.com/YuxiangZhang-BIT。

原文摘要 · Abstract (English)

Recent advances in Remote Sensing Foundation Models (RSFMs) have led to significant breakthroughs in the field. While many RSFMs have been pretrained with massive optical imagery, more multispectral/hyperspectral data remain lack of the corresponding foundation models. To leverage the advantages of spectral imagery in earth observation, we explore whether existing RSFMs can be effectively adapted to process diverse spectral modalities without requiring extensive spectral pretraining. In response to this challenge, we proposed SpectralX, an innovative parameter-efficient fine-tuning framework that adapt existing RSFMs as backbone while introducing a two-stage training approach to handle various spectral inputs, thereby significantly improving domain generalization performance. In the first stage, we employ a masked-reconstruction task and design a specialized Hyper Tokenizer (HyperT) to extract attribute tokens from both spatial and spectral dimensions. Simultaneously, we develop an Attribute-oriented Mixture of Adapter (AoMoA) that dynamically aggregates multi-attribute expert knowledge while performing layer-wise fine-tuning. With semantic segmentation as downstream task in the second stage, we insert an Attribute-refined Adapter (Are-adapter) into the first stage framework. By iteratively querying low-level semantic features with high-level representations, the model learns to focus on task-beneficial attributes, enabling customized adjustment of RSFMs. Following this two-phase adaptation process, SpectralX is capable of interpreting spectral imagery from new regions or seasons. The codes will be available from the website: https://github.com/YuxiangZhang-BIT.

遥感跨域泛化参数高效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。