arXiv:2501.04944cs.CV2025-01中稿 · IEEE TGRS被引 342

用Mamba模型实现高光谱图像高效分类,兼顾空间与光谱信息融合。

MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classification

  • 设计空间与光谱双分支Mamba模块,分别建模像素长程关系与波段间关联。
  • 在四个数据集上达到领先性能,速度远快于传统Transformer。
  • 适合需要快速高精度分类的遥感图像分析任务。

Transformer在高光谱图像(HSI)分类中被广泛研究,但其二次计算复杂度导致速度慢、内存占用高。近期,Mamba模型因其线性计算复杂度和强长距离建模能力成为新方向。然而,现有Mamba难以同时捕捉HSI的空间-光谱联合特征。为此,我们提出首个基于Mamba的图像级HSI分类模型MambaHSI,可自适应地融合空间与光谱信息。具体地,设计空间Mamba块(SpaMB)在像素级建模全局长程依赖;提出光谱Mamba块(SpeMB)将光谱向量分组,挖掘跨组关系并提取光谱特征;构建空间-光谱融合模块(SSFM)实现自适应特征融合。在四个不同数据集上的大量实验表明,该模型显著优于现有方法,验证了Mamba作为下一代HSI骨干网络的巨大潜力。代码已开源。

原文摘要 · Abstract (English)

Transformer has been extensively explored for hyperspectral image (HSI) classification. However, transformer poses challenges in terms of speed and memory usage because of its quadratic computational complexity. Recently, the Mamba model has emerged as a promising approach, which has strong long-distance modeling capabilities while maintaining a linear computational complexity. However, representing the HSI is challenging for the Mamba due to the requirement for an integrated spatial and spectral understanding. To remedy these drawbacks, we propose a novel HSI classification model based on a Mamba model, named MambaHSI, which can simultaneously model long-range interaction of the whole image and integrate spatial and spectral information in an adaptive manner. Specifically, we design a spatial Mamba block (SpaMB) to model the long-range interaction of the whole image at the pixel-level. Then, we propose a spectral Mamba block (SpeMB) to split the spectral vector into multiple groups, mine the relations across different spectral groups, and extract spectral features. Finally, we propose a spatial-spectral fusion module (SSFM) to adaptively integrate spatial and spectral features of a HSI. To our best knowledge, this is the first image-level HSI classification model based on the Mamba. We conduct extensive experiments on four diverse HSI datasets. The results demonstrate the effectiveness and superiority of the proposed model for HSI classification. This reveals the great potential of Mamba to be the next-generation backbone for HSI models. Codes are available at https://github.com/li-yapeng/MambaHSI .

高光谱分类Mamba空间光谱融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。