arXiv:2412.20645cs.CV2024-12被引 11

YOLO-UniOW让目标检测能识别未知物体,速度还快。

YOLO-UniOW: Efficient Universal Open-World Object Detection

  • 用轻量级文本图像对齐替代复杂跨模态融合,提升效率。
  • 在LVIS上达34.6 AP、30.0 APr,推理速度69.6 FPS。
  • 可动态识别未知物体并扩展词汇,适合开放世界应用。

传统目标检测模型受限于封闭集数据集,仅能检测训练中见过的类别。尽管多模态模型通过图文对齐扩展了类别识别能力,但其跨模态融合带来显著推理开销,且仍受预定义词汇表限制,在开放世界场景中难以处理未知物体。本文提出通用开放世界目标检测(Uni-OWD)新范式,设计了YOLO-UniOW模型,突破效率、泛化与性能边界。该模型采用自适应决策学习,在CLIP隐空间实现轻量对齐,取代高成本的跨模态融合,保持良好泛化性。同时提出通配符学习策略,将分布外物体识别为“未知”,支持动态词汇扩展,无需增量学习即可适应新类别。大量实验验证其优越性:在LVIS上达34.6 AP和30.0 APr,推理速度69.6 FPS;在M-OWODB、S-OWODB和nuScenes上均创基准纪录,展现出卓越的开放世界检测性能。代码与模型已开源。

原文摘要 · Abstract (English)

Traditional object detection models are constrained by the limitations of closed-set datasets, detecting only categories encountered during training. While multimodal models have extended category recognition by aligning text and image modalities, they introduce significant inference overhead due to cross-modality fusion and still remain restricted by predefined vocabulary, leaving them ineffective at handling unknown objects in open-world scenarios. In this work, we introduce Universal Open-World Object Detection (Uni-OWD), a new paradigm that unifies open-vocabulary and open-world object detection tasks. To address the challenges of this setting, we propose YOLO-UniOW, a novel model that advances the boundaries of efficiency, versatility, and performance. YOLO-UniOW incorporates Adaptive Decision Learning to replace computationally expensive cross-modality fusion with lightweight alignment in the CLIP latent space, achieving efficient detection without compromising generalization. Additionally, we design a Wildcard Learning strategy that detects out-of-distribution objects as "unknown" while enabling dynamic vocabulary expansion without the need for incremental learning. This design empowers YOLO-UniOW to seamlessly adapt to new categories in open-world environments. Extensive experiments validate the superiority of YOLO-UniOW, achieving achieving 34.6 AP and 30.0 APr on LVIS with an inference speed of 69.6 FPS. The model also sets benchmarks on M-OWODB, S-OWODB, and nuScenes datasets, showcasing its unmatched performance in open-world object detection. Code and models are available at https://github.com/THU-MIG/YOLO-UniOW.

目标检测开放世界YOLO多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。