DefMamba通过可变形扫描路径提升图像结构感知能力
DefMamba: Deformable Visual State Space Model
- 引入可变形扫描策略，动态调整图像遍历路径
- 在多个视觉任务中达到领先性能，如分类与分割
- 适合需要精细空间建模的视觉任务研究者
近期，状态空间模型（SSM）尤其是Mamba因其在计算效率与性能间的良好平衡受到广泛关注。然而，现有的视觉Mamba方法通常采用预设扫描顺序将图像展平为一维序列，导致模型在特征提取过程中难以有效利用图像的空间结构信息。为此，我们提出一种新型视觉基础模型DefMamba，包含多尺度骨干结构和可变形Mamba（DM）模块，通过动态调整扫描路径以优先处理重要信息，从而增强对相关输入特征的捕捉与处理能力。结合可变形扫描（DS）策略，该模型显著提升了对图像结构的学习能力和对物体细节变化的检测能力。大量实验表明，DefMamba在图像分类、目标检测、实例分割和语义分割等多个视觉任务中均达到当前最优性能。代码已开源。
原文摘要 · Abstract (English)
Recently, state space models (SSM), particularly Mamba, have attracted significant attention from scholars due to their ability to effectively balance computational efficiency and performance. However, most existing visual Mamba methods flatten images into 1D sequences using predefined scan orders, which results the model being less capable of utilizing the spatial structural information of the image during the feature extraction process. To address this issue, we proposed a novel visual foundation model called DefMamba. This model includes a multi-scale backbone structure and deformable mamba (DM) blocks, which dynamically adjust the scanning path to prioritize important information, thus enhancing the capture and processing of relevant input features. By combining a deformable scanning(DS) strategy, this model significantly improves its ability to learn image structures and detects changes in object details. Numerous experiments have shown that DefMamba achieves state-of-the-art performance in various visual tasks, including image classification, object detection, instance segmentation, and semantic segmentation. The code is open source on DefMamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。