用Mamba提升YOLO-World的开放词汇检测能力
Mamba-YOLO-World: Marrying YOLO-World with Mamba for Open-Vocabulary Detection
- 引入MambaFusion-PAN结构，以线性复杂度实现全局感知特征融合
- 在COCO和LVIS上零样本与微调均超越原版YOLO-World
- 参数与算力更低，适合高效开放词汇检测场景
开放词汇检测（OVD）旨在识别预定义类别之外的物体。作为首个将YOLO系列融入OVD的模型，YOLO-World在速度与效率上表现优异，但其颈部特征融合机制存在二次复杂度高、引导感受野受限的问题。为此，本文提出Mamba-YOLO-World，一种基于YOLO的新型OVD模型，采用所提出的MambaFusion路径聚合网络（MambaFusion-PAN）作为颈部架构。具体而言，设计了一种基于状态空间模型的特征融合机制，包含并行引导选择扫描与串行引导选择扫描算法，具备线性复杂度与全局引导感受野。该机制利用多模态输入序列与Mamba隐藏状态引导选择扫描过程。实验表明，该模型在COCO与LVIS基准上，在零样本与微调设置下均优于原始YOLO-World，且保持相近的参数量与浮点运算量（FLOPs）。此外，其性能超越现有最先进的OVD方法，同时使用更少的参数与FLOPs。
原文摘要 · Abstract (English)
Open-vocabulary detection (OVD) aims to detect objects beyond a predefined set of categories. As a pioneering model incorporating the YOLO series into OVD, YOLO-World is well-suited for scenarios prioritizing speed and efficiency. However, its performance is hindered by its neck feature fusion mechanism, which causes the quadratic complexity and the limited guided receptive fields. To address these limitations, we present Mamba-YOLO-World, a novel YOLO-based OVD model employing the proposed MambaFusion Path Aggregation Network (MambaFusion-PAN) as its neck architecture. Specifically, we introduce an innovative State Space Model-based feature fusion mechanism consisting of a Parallel-Guided Selective Scan algorithm and a Serial-Guided Selective Scan algorithm with linear complexity and globally guided receptive fields. It leverages multi-modal input sequences and mamba hidden states to guide the selective scanning process. Experiments demonstrate that our model outperforms the original YOLO-World on the COCO and LVIS benchmarks in both zero-shot and fine-tuning settings while maintaining comparable parameters and FLOPs. Additionally, it surpasses existing state-of-the-art OVD methods with fewer parameters and FLOPs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。