用光谱指数引导遮蔽,让多光谱遥感预训练更精准
SIGMAE: A Spectral-Index-Guided Foundation Model for Multispectral Remote Sensing
- 用光谱指数指导动态遮蔽,聚焦语义重要区域
- 90%遮蔽率下仍保持强重建能力,提升复杂目标识别
- 适合遥感图像分类、分割等下游任务,尤其标签少时表现优
预训练与微调已成为遥感图像解译的新范式。其中,基于掩码自编码器(MAE)的预训练因其通过重建被遮蔽区域来学习通用特征表示而表现突出。然而,将MAE应用于多光谱遥感图像仍面临挑战,主要源于复杂背景、目标不明显以及遮蔽过程缺乏语义引导,导致难以学习深层结构与有意义的空间-光谱特征。为此,我们提出一种简单有效的多光谱图像预训练方法——光谱指数引导的MAE(SIGMAE)。核心思想是引入领域特定的光谱指数作为先验知识,指导动态标记选择以聚焦信息丰富区域。SIGMAE提出语义显著性引导的动态标记策略(SSDTM),通过量化每个图像块的语义丰富度和内部异质性,自适应选择最具信息量的标记,在训练中优先关注语义显著区域,并逐步增加样本难度。该策略增强了光谱丰富且结构感知的表征学习,缓解过拟合,减少冗余计算。在五个广泛使用的数据集上进行的大量实验表明,该方法在场景分类、语义分割、目标提取和变化检测等多种下游任务中均优于其他预训练地理空间基础模型。此外,即使在90%遮蔽率下,其仍具备强大的空间-光谱重建能力,并在标注数据有限条件下显著提升复杂目标识别性能。代码与模型权重将公开于 https://github.com/zxk688/SIGMAE。
原文摘要 · Abstract (English)
Pretraining and fine-tuning have emerged as a new paradigm in remote sensing image interpretation. Among them, Masked Autoencoder (MAE)-based pretraining stands out for its strong capability to learn general feature representations via reconstructing masked image regions. However, applying MAE to multispectral remote sensing images remains challenging due to complex backgrounds, indistinct targets, and the lack of semantic guidance during masking, which hinders the learning of underlying structures and meaningful spatial-spectral features. To address this, we propose a simple yet effective approach, Spectral Index-Guided MAE (SIGMAE), for multispectral image pretraining. The core idea is to incorporate domain-specific spectral indices as prior knowledge to guide dynamic token masking toward informative regions. SIGMAE introduces Semantic Saliency-Guided Dynamic Token Masking (SSDTM), a curriculum-style strategy that quantifies each patch's semantic richness and internal heterogeneity to adaptively select the most informative tokens during training. By prioritizing semantically salient regions and progressively increasing sample difficulty, SSDTM enhances spectrally rich and structurally aware representation learning, mitigates overfitting, and reduces redundant computation compared with random masking. Extensive experiments on five widely used datasets covering various downstream tasks, including scene classification, semantic segmentation, object extraction and change detection, demonstrate that SIGMAE outperforms other pretrained geospatial foundation models. Moreover, it exhibits strong spatial-spectral reconstruction capability, even with a 90% mask ratio, and improves complex target recognition under limited labeled data. The source codes and model weights will be released at https://github.com/zxk688/SIGMAE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。