arXiv:2509.04669cs.CVcs.AI2025-09ICCV被引 5

融合卷积与多方向Mamba,高效提取视觉特征

VCMamba: Bridging Convolutions with Multi-Directional Mamba for Efficient Visual Representation

  • 用卷积块提取局部特征,再用多方向Mamba建模全局依赖
  • ImageNet-1K准确率达82.6%,参数减少37%仍优于基线
  • 适合追求高效率与强表征能力的视觉任务应用

近期视觉变压器(ViTs)和状态空间模型(SSMs)挑战了卷积神经网络(CNNs)在计算机视觉中的主导地位。ViTs擅长捕捉全局上下文,而像Mamba这样的SSMs能以线性复杂度处理长序列,但对细粒度局部特征的建模不如CNNs。相反,CNNs具备强局部特征归纳偏置,却缺乏ViTs和Mamba的全局推理能力。为此,我们提出VCMamba,一种新型视觉骨干网络,融合了CNN与多方向Mamba SSM的优势。VCMamba采用卷积茎干和分层结构,在早期阶段使用卷积块提取丰富局部特征;后期阶段引入多方向Mamba块,高效建模长程依赖与全局上下文。该混合设计在保持图像分辨率线性复杂度的同时实现更优特征表示。我们在ImageNet-1K分类和ADE20K语义分割上进行了大量实验。VCMamba-B在ImageNet-1K上达到82.6%的top-1准确率,比PlainMamba-L3高0.3%,参数量减少37%;优于Vision GNN-B 0.3%,参数减少64%。此外,VCMamba-B在ADE20K上取得47.1 mIoU,超过EfficientFormer-L7 2.0 mIoU,参数量减少62%。代码已开源:https://github.com/Wertyuui345/VCMamba。

原文摘要 · Abstract (English)

Recent advances in Vision Transformers (ViTs) and State Space Models (SSMs) have challenged the dominance of Convolutional Neural Networks (CNNs) in computer vision. ViTs excel at capturing global context, and SSMs like Mamba offer linear complexity for long sequences, yet they do not capture fine-grained local features as effectively as CNNs. Conversely, CNNs possess strong inductive biases for local features but lack the global reasoning capabilities of transformers and Mamba. To bridge this gap, we introduce \textit{VCMamba}, a novel vision backbone that integrates the strengths of CNNs and multi-directional Mamba SSMs. VCMamba employs a convolutional stem and a hierarchical structure with convolutional blocks in its early stages to extract rich local features. These convolutional blocks are then processed by later stages incorporating multi-directional Mamba blocks designed to efficiently model long-range dependencies and global context. This hybrid design allows for superior feature representation while maintaining linear complexity with respect to image resolution. We demonstrate VCMamba's effectiveness through extensive experiments on ImageNet-1K classification and ADE20K semantic segmentation. Our VCMamba-B achieves 82.6% top-1 accuracy on ImageNet-1K, surpassing PlainMamba-L3 by 0.3% with 37% fewer parameters, and outperforming Vision GNN-B by 0.3% with 64% fewer parameters. Furthermore, VCMamba-B obtains 47.1 mIoU on ADE20K, exceeding EfficientFormer-L7 by 2.0 mIoU while utilizing 62% fewer parameters. Code is available at https://github.com/Wertyuui345/VCMamba.

视觉模型Mamba卷积融合高效架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。