arXiv:2509.14104cs.CV2025-09被引 2

用软门控专家模型提升遥感大模型效率,训练更快、推理更省。

CSMoE: An Efficient Remote Sensing Foundation Model with Soft Mixture-of-Experts

  • 引入软门控专家机制,实现跨传感器高效特征学习。
  • 训练仅需1/4的浮点运算量,任务表现仍达顶尖水平。
  • 按气候主题采样数据,大幅降低预训练成本,适合资源有限团队。

自监督学习通过掩码自编码器(MAE)在遥感基础模型(FM)发展中备受关注,可提升多传感器与下游任务的表征能力。然而现有遥感模型常面临训练与推理计算开销大、表征能力受限的问题,且预训练数据中存在冗余图像,增加成本却无助于提升表示性能。为此,本文从模型效率与数据效率两方面优化:一是将软门控专家(Soft MoE)机制融入模型,使各模态专家并行处理同时共享跨传感器表征,显著降低训练与推理复杂度;二是设计基于主题-气候描述符的采样策略,从大规模影像库中构建精简但地理与主题多样性强的训练集,减少预训练成本。在场景分类、语义分割和内容检索任务上的实验表明,CSMoE 在仅需1/4浮点运算量的情况下,仍保持与当前最优遥感模型相当的性能。代码、数据集及预训练权重将公开于https://git.tu-berlin.de/rsim/csmoe。

原文摘要 · Abstract (English)

Self-supervised learning (SSL) through masked autoencoders (MAEs) has recently attracted great attention for remote sensing (RS) foundation model (FM) development, enabling improved representation learning across diverse sensors and downstream tasks. However, existing RS FMs often either suffer from substantial computational complexity during both training and inference or exhibit limited representational capacity. In addition, their pretraining datasets can contain redundant images, which increases training cost without improving the learned representations. These issues restrict their practical applicability in RS. To address these limitations, we improve the computational efficiency of RS FMs along two axes: i) model efficiency; and ii) training-data efficiency. Model efficiency is achieved by integrating the Soft mixture-of-experts (MoE) mechanism into the FM, which allows modality-specific expert processing alongside shared cross-sensor representation learning while reducing computational complexity at both training and inference time. We apply this adaptation to the Cross-Sensor Masked Autoencoder (CSMAE) model, which serves as our main baseline, resulting in the Cross-Sensor Mixture-of-Experts (CSMoE) model. Training-data efficiency is achieved by a thematic-climatic descriptor-driven sampling strategy, which constructs a reduced training set from a large-scale image archive while retaining its geographic and thematic-climatic diversity, and thus reduces pretraining cost. Extensive experiments on scene classification, semantic segmentation, and content-based image retrieval (CBIR) show that CSMoE remains competitive with state-of-the-art RS FMs while requiring substantially fewer floating-point operations (FLOPs). The associated code for the model and the training set creation, as well as the pretrained model weights, will be available at https://git.tu-berlin.de/rsim/csmoe.

遥感专家模型自监督效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。