用双维度掩码训练高光谱图像模型,提升分类准确率。
HyperspectralMAE: The Hyperspectral Imagery Classification Model using Fourier-Encoded Dual-Branch Masked Autoencoder
- 通过随机遮蔽50%空间块和50%光谱波段,强制模型学习跨维重建能力。
- 在印第安纳菠萝数据集上达到当前最优迁移学习精度,显著优于基线。
- 适合需要高光谱图像分析的遥感、环境监测领域研究者使用。
高光谱图像虽富含光谱细节,但因空间与光谱维度高而带来挑战。本文提出基于Transformer的高光谱基础模型HyperspectralMAE,采用双掩码策略:预训练时随机遮蔽50%空间块和50%光谱波段,迫使模型学习跨维度补全能力。为编码光谱顺序,引入基于波长的可学习谐波傅里叶位置嵌入。重建目标结合均方误差(MSE)与光谱角映射(SAM),兼顾像素级精度与光谱形状保真度。模型参数量约1.8×10⁸,输出768维嵌入,具备强迁移学习能力。在两个大规模高光谱数据集——NASA EO-1 Hyperion(约1600景,约3×10¹¹个像素光谱)和DLR EnMAP Level-0(约1300景,约3×10¹¹个像素光谱)上预训练,并在印第安纳菠萝基准数据集上微调用于地物分类。实验表明,该模型在印度菠萝数据集上达到当前最优迁移学习精度,验证了双掩码预训练能生成鲁棒的光谱-空间表征。结果证明,双掩码机制与波长感知嵌入有效推动高光谱图像重建与下游分析。
原文摘要 · Abstract (English)
Hyperspectral imagery provides rich spectral detail but poses unique challenges because of its high dimensionality in both spatial and spectral domains. We propose \textit{HyperspectralMAE}, a Transformer-based foundation model for hyperspectral data that employs a \textit{dual masking} strategy: during pre-training we randomly occlude 50\% of spatial patches and 50\% of spectral bands. This forces the model to learn representations capable of reconstructing missing information across both dimensions. To encode spectral order, we introduce learnable harmonic Fourier positional embeddings based on wavelength. The reconstruction objective combines mean-squared error (MSE) with the spectral angle mapper (SAM) to balance pixel-level accuracy and spectral-shape fidelity. The resulting model contains about $1.8\times10^{8}$ parameters and produces 768-dimensional embeddings, giving it sufficient capacity for transfer learning. We pre-trained HyperspectralMAE on two large hyperspectral corpora -- NASA EO-1 Hyperion ($\sim$1\,600 scenes, $\sim$$3\times10^{11}$ pixel spectra) and DLR EnMAP Level-0 ($\sim$1\,300 scenes, $\sim$$3\times10^{11}$ pixel spectra) -- and fine-tuned it for land-cover classification on the Indian Pines benchmark. HyperspectralMAE achieves state-of-the-art transfer-learning accuracy on Indian Pines, confirming that masked dual-dimensional pre-training yields robust spectral-spatial representations. These results demonstrate that dual masking and wavelength-aware embeddings advance hyperspectral image reconstruction and downstream analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。