将Mamba的高效设计引入分层自注意力，提升视觉模型性能。
A Separable Self-attention Inspired by the State Space Model for Computer Vision
- 用Mamba思想改进分层自注意力，实现线性计算复杂度。
- VMINet在图像分类与高分辨率预测任务中表现媲美主流模型。
- 无需复杂模块，仅用基础下采样和新注意力结构即达佳绩。
Mamba是一种具有线性计算复杂度的高效状态空间模型（SSM）。尽管SSM不适用于非因果数据，但视觉领域中的Vision Mamba（ViM）方法在图像分类和目标检测等任务上仍表现出色。近期研究揭示了状态空间模型与注意力机制之间的深刻理论联系。本文首次将Mamba中的优秀设计理念引入分层自注意力，提出一种新型可分离自注意力方法。为公平对比ViM，我们构建了仅由新注意力模块与最基础下采样层堆叠而成的简单而强大的原型架构VMINet。值得注意的是，VMINet与传统Transformer架构有显著差异。实验表明，VMINet在图像分类和高分辨率密集预测任务上取得了具有竞争力的结果。代码已开源：https://github.com/yws-wxs/VMINet。
原文摘要 · Abstract (English)
Mamba is an efficient State Space Model (SSM) with linear computational complexity. Although SSMs are not suitable for handling non-causal data, Vision Mamba (ViM) methods still demonstrate good performance in tasks such as image classification and object detection. Recent studies have shown that there is a rich theoretical connection between state space models and attention variants. We propose a novel separable self attention method, for the first time introducing some excellent design concepts of Mamba into separable self-attention. To ensure a fair comparison with ViMs, we introduce VMINet, a simple yet powerful prototype architecture, constructed solely by stacking our novel attention modules with the most basic down-sampling layers. Notably, VMINet differs significantly from the conventional Transformer architecture. Our experiments demonstrate that VMINet has achieved competitive results on image classification and high-resolution dense prediction tasks.Code is available at: https://github.com/yws-wxs/VMINet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。