轻量级视觉Mamba模型,兼顾速度与精度。
MobileMamba: Lightweight Multi-Receptive Visual Mamba Network
- 三阶段架构+多感受野模块,提升细节捕捉能力
- 顶1准确率达83.6%,比LocalVim快21倍
- 适合移动端或高分辨率视觉任务部署
轻量级模型研究长期集中于CNN和Transformer架构。CNN局部感受野难以建模长程依赖,而Transformer在高分辨率场景下受二次计算复杂度限制。最近,状态空间模型因线性计算复杂度在视觉领域兴起。尽管FLOPs低,现有轻量级Mamba模型仍存在吞吐量不足问题。本文提出MobileMamba框架,在效率与性能间取得平衡。设计三阶段网络显著提升推理速度;在细粒度层面引入多感受野特征交互(MRFFI)模块,包含长程小波增强Mamba(WTE-Mamba)、高效多核深度卷积(MK-DeConv)及冗余身份组件移除机制,融合多尺度信息并强化高频细节提取。结合训练与测试策略进一步提升性能。MobileMamba在顶1准确率上达83.6%,超越现有最优方法,且在GPU上较LocalVim快21倍。大量高分辨率下游任务实验表明,该模型在速度与精度间实现最优平衡。
原文摘要 · Abstract (English)
Previous research on lightweight models has primarily focused on CNNs and Transformer-based designs. CNNs, with their local receptive fields, struggle to capture long-range dependencies, while Transformers, despite their global modeling capabilities, are limited by quadratic computational complexity in high-resolution scenarios. Recently, state-space models have gained popularity in the visual domain due to their linear computational complexity. Despite their low FLOPs, current lightweight Mamba-based models exhibit suboptimal throughput. In this work, we propose the MobileMamba framework, which balances efficiency and performance. We design a three-stage network to enhance inference speed significantly. At a fine-grained level, we introduce the Multi-Receptive Field Feature Interaction(MRFFI) module, comprising the Long-Range Wavelet Transform-Enhanced Mamba(WTE-Mamba), Efficient Multi-Kernel Depthwise Convolution(MK-DeConv), and Eliminate Redundant Identity components. This module integrates multi-receptive field information and enhances high-frequency detail extraction. Additionally, we employ training and testing strategies to further improve performance and efficiency. MobileMamba achieves up to 83.6% on Top-1, surpassing existing state-of-the-art methods which is maximum x21 faster than LocalVim on GPU. Extensive experiments on high-resolution downstream tasks demonstrate that MobileMamba surpasses current efficient models, achieving an optimal balance between speed and accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。