arXiv:2507.04409cs.CV2025-07被引 2

融合Mamba与Transformer的新型网络,高效处理高光谱图像分类

MVNet: Hyperspectral Remote Sensing Image Classification Based on Hybrid Mamba-Transformer Vision Backbone Architecture

  • 结合3D-CNN、Transformer与线性复杂度Mamba,实现空间-光谱特征高效提取
  • 在IN/UP/KSC数据集上准确率超主流方法,计算效率显著提升
  • 适合高光谱遥感图像分析、需要低延迟部署的科研与工程场景

高光谱图像(HSI)分类面临高维数据、训练样本有限及光谱冗余等问题,常导致过拟合和泛化能力不足。本文提出一种新型MVNet网络架构,融合3D-CNN局部特征提取、Transformer全局建模能力以及Mamba的线性复杂度序列建模优势,实现高效的时空-光谱特征提取与融合。MVNet设计了改进的双分支Mamba模块,包含状态空间模型(SSM)分支与采用1D卷积加SiLU激活的非SSM分支,增强对短程与长程依赖的建模能力,同时降低传统Mamba的计算延迟。优化的HSI-MambaVision Mixer模块突破因果卷积的单向限制,通过解耦注意力机制在一次前向传播中捕捉双向空间-光谱依赖,缓解参数冗余与维度灾难问题。在IN、UP和KSC数据集上,MVNet在分类准确率和计算效率方面均优于主流方法,展现出处理复杂高光谱数据的强大能力。

原文摘要 · Abstract (English)

Hyperspectral image (HSI) classification faces challenges such as high-dimensional data, limited training samples, and spectral redundancy, which often lead to overfitting and insufficient generalization capability. This paper proposes a novel MVNet network architecture that integrates 3D-CNN's local feature extraction, Transformer's global modeling, and Mamba's linear complexity sequence modeling capabilities, achieving efficient spatial-spectral feature extraction and fusion. MVNet features a redesigned dual-branch Mamba module, including a State Space Model (SSM) branch and a non-SSM branch employing 1D convolution with SiLU activation, enhancing modeling of both short-range and long-range dependencies while reducing computational latency in traditional Mamba. The optimized HSI-MambaVision Mixer module overcomes the unidirectional limitation of causal convolution, capturing bidirectional spatial-spectral dependencies in a single forward pass through decoupled attention that focuses on high-value features, alleviating parameter redundancy and the curse of dimensionality. On IN, UP, and KSC datasets, MVNet outperforms mainstream hyperspectral image classification methods in both classification accuracy and computational efficiency, demonstrating robust capability in processing complex HSI data.

高光谱图像MambaTransformer遥感分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。