融合卷积、Transformer与Mamba，高效提升高光谱图像分类性能
ConvVitMamba: Efficient Multiscale Convolution, Transformer, and Mamba-Based Sequence modelling for Hyperspectral Image Classification
- 三模块融合：多尺度卷积捕局部特征，ViT建模全局关系，轻量Mamba实现高效序列建模
- 在四组数据集上超越传统方法，准确率最高提升3.2%，模型更小、推理更快
- 适合资源受限场景下的高光谱图像分析，尤其适用于无人机遥感应用
高光谱图像（HSI）分类因光谱维数高、冗余性强且标注数据有限而面临挑战。尽管卷积神经网络（CNN）和视觉变换器（ViT）能有效捕捉光谱-空间信息与长程依赖，但常伴随高计算成本和大模型尺寸，限制实际应用。为此，本文提出统一的混合框架ConvVitMamba，集成三部分：多尺度卷积特征提取器以捕获局部光谱、空间及联合模式；基于ViT的分块与编码阶段以建模全局上下文关系；以及受Mamba启发的轻量级门控序列混合模块，实现无二次自注意力的内容感知优化。采用主成分分析（PCA）进行预处理以降低冗余并提升效率。在四个基准数据集（包括Houston和三个无人机航拍QUH数据集：Pingan、Qingyun、Tangdaowan）上的实验表明，ConvVitMamba持续优于基于CNN、Transformer和Mamba的方法，同时在精度、模型大小与推理效率间保持良好平衡。消融研究证实各组件互补贡献。结果表明该框架为多样化场景下的HSI分类提供了高效有效的解决方案。源代码已公开于https://github.com/mqalkhatib/ConvVitMamba。
原文摘要 · Abstract (English)
Hyperspectral image (HSI) classification remains challenging due to high spectral dimensionality, redundancy, and limited labeled data. Although convolutional neural networks (CNNs) and Vision Transformers (ViTs) achieve strong performance by exploiting spectral-spatial information and long-range dependencies, they often incur high computational cost and large model size, limiting practical use. To address these limitations, a unified hybrid framework, termed ConvVitMamba, is proposed for efficient HSI classification. The architecture integrates three components: a multiscale convolutional feature extractor to capture local spectral, spatial, and joint patterns; a Vision Transformer based tokenization and encoding stage to model global contextual relationships; and a lightweight Mamba inspired gated sequence mixing module for efficient content-aware refinement without quadratic self-attention. Principal Component Analysis (PCA) is used as preprocessing to reduce redundancy and improve efficiency. Experiments on four benchmark datasets, including Houston and three UAV borne QUH datasets (Pingan, Qingyun, and Tangdaowan), demonstrate that ConvVitMamba consistently outperforms CNN, Transformer, and Mamba based methods while maintaining a favorable balance between accuracy, model size, and inference efficiency. Ablation studies confirm the complementary contributions of all components. The results indicate that the proposed framework provides an effective and efficient solution for HSI classification in diverse scenarios. The source code is publicly available at https://github.com/mqalkhatib/ConvVitMamba
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。