用多滤波扫描减少视觉数据冗余,提升图像建模效果
MFil-Mamba: Multi-Filter Scanning for Spatial Redundancy-Aware Visual State Space Models
- 设计多滤波扫描架构,动态捕捉不同空间信息
- 在ImageNet上达83.2%准确率,COCO检测指标超47%
- 适合追求高效高精度视觉建模的研究与应用
状态空间模型(SSMs)尤其是Mamba架构在序列建模中表现优异。然而将SSMs扩展到计算机视觉仍面临挑战,主要源于视觉数据的非序列结构和复杂的二维空间依赖性。现有方法多依赖对同一输入的多种遍历策略,导致冗余并破坏图像内复杂的空间关系。为此,我们提出MFil-Mamba,一种基于多滤波扫描主干的新型视觉状态空间架构。不同于固定多方向遍历,该设计使每次扫描都能捕获独特且上下文相关的空间信息,同时最小化冗余。此外,引入自适应加权机制,有效融合多路扫描输出,并结合架构优化。MFil-Mamba在多个基准上超越现有最先进模型,涵盖图像分类、目标检测、实例分割和语义分割。例如,其小型变体在ImageNet-1K上达到83.2%的top-1准确率,在MS COCO上取得47.3%的box AP和42.7%的mask AP,ADE20K上达到48.5%的mIoU。代码与模型已开源。
原文摘要 · Abstract (English)
State Space Models (SSMs), especially recent Mamba architecture, have achieved remarkable success in sequence modeling tasks. However, extending SSMs to computer vision remains challenging due to the non-sequential structure of visual data and its complex 2D spatial dependencies. Although several early studies have explored adapting selective SSMs for vision applications, most approaches primarily depend on employing various traversal strategies over the same input. This introduces redundancy and distorts the intricate spatial relationships within images. To address these challenges, we propose MFil-Mamba, a novel visual state space architecture built on a multi-filter scanning backbone. Unlike fixed multi-directional traversal methods, our design enables each scan to capture unique and contextually relevant spatial information while minimizing redundancy. Furthermore, we incorporate an adaptive weighting mechanism to effectively fuse outputs from multiple scans in addition to architectural enhancements. MFil-Mamba achieves superior performance over existing state-of-the-art models across various benchmarks that include image classification, object detection, instance segmentation, and semantic segmentation. For example, our tiny variant attains 83.2% top-1 accuracy on ImageNet-1K, 47.3% box AP and 42.7% mask AP on MS COCO, and 48.5% mIoU on the ADE20K dataset. Code and models are available at https://github.com/puskal-khadka/MFil-Mamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。