arXiv:2603.16423cs.CVcs.AI2026-03被引 2

提出SF-Mamba,让视觉Mamba实现高效双向信息传递与并行计算。

SF-Mamba: Rethinking State Space Model for Vision

  • 通过补丁交换实现单向扫描下的双向信息流动
  • 批量折叠与周期状态重置提升GPU并行效率,吞吐量显著提高
  • 在分类、检测等任务上超越现有模型,适合追求速度的视觉任务

近年来,Mamba被用于替代因二次复杂度而受限的视觉Transformer(ViTs)。尽管Mamba的循环扫描机制具备计算效率,但其天然限制了图像块间的非因果交互。先前工作尝试通过多扫描策略解决此问题,但存在扫描设计不佳和频繁数据重排导致的效率低下。此外,Mamba在短序列长度下(常见于视觉任务)计算速度较慢。为此,我们重新思考视觉任务中的扫描操作与Mamba的计算效率,提出SF-Mamba:引入辅助补丁交换以在单向扫描中编码双向信息流,并采用批量折叠与周期状态重置以实现更优的GPU并行性。大量实验表明,无论在图像分类、目标检测还是实例与语义分割任务中,SF-Mamba均显著优于现有先进基线,且在不同模型规模下均提升了吞吐量。

原文摘要 · Abstract (English)

The realm of Mamba for vision has been advanced in recent years to strike for the alternatives of Vision Transformers (ViTs) that suffer from the quadratic complexity. While the recurrent scanning mechanism of Mamba offers computational efficiency, it inherently limits non-causal interactions between image patches. Prior works have attempted to address this limitation through various multi-scan strategies; however, these approaches suffer from inefficiencies due to suboptimal scan designs and frequent data rearrangement. Moreover, Mamba exhibits relatively slow computational speed under short token lengths, commonly used in visual tasks. In pursuit of a truly efficient vision encoder, we rethink the scan operation for vision and the computational efficiency of Mamba. To this end, we propose SF-Mamba, a novel visual Mamba with two key proposals: auxiliary patch swapping for encoding bidirectional information flow under an unidirectional scan and batch folding with periodic state reset for advanced GPU parallelism. Extensive experiments on image classification, object detection, and instance and semantic segmentation consistently demonstrate that our proposed SF-Mamba significantly outperforms state-of-the-art baselines while improving throughput across different model sizes. We will release the source code after publication.

视觉Mamba高效模型并行计算图像识别

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。