轻量级框架DOSOD实现高效开集目标检测,适合机器人实时部署。
A Light-Weight Framework for Open-Set Object Detection with Decoupled Feature Alignment in Joint Space
- 分离视觉语言特征,在联合空间直接对齐,减少计算开销。
- 在LVIS数据集上达到26.7%固定AP,推理速度比基线快57.1%。
- 模型轻量,适合边缘设备部署,代码开源可复现。
开集目标检测(OSOD)在非结构化环境中的机器人操作中极具价值。然而,现有方法因计算负担重、部署复杂而难以满足机器人应用需求。为此,本文提出轻量级框架DOSOD,基于YOLO-World架构,融合视觉语言模型(VLM)与检测器。通过多层感知机(MLP)适配器将VLM提取的文本嵌入转换至联合空间,使检测器在该空间中学习类无关提议的区域表示。跨模态特征在联合空间直接对齐,避免复杂特征交互,提升效率。测试阶段如传统闭集检测器运行,有效弥合闭集与开集检测差距。相比基线YOLO-World,DOSOD显著提升实时性能,保持相近精度:DOSOD-S在相同骨干网络下于LVIS minival数据集上取得26.7%的固定AP,优于YOLO-World-v1-S的26.2%和YOLO-World-v2-S的22.7%;同时,其帧率较YOLO-World-v1-S高出57.1%,较v2-S高出29.6%。实验表明,DOSOD适用于边缘设备部署。代码与模型已公开于https://github.com/D-Robotics-AI-Lab/DOSOD。
原文摘要 · Abstract (English)
Open-set object detection (OSOD) is highly desirable for robotic manipulation in unstructured environments. However, existing OSOD methods often fail to meet the requirements of robotic applications due to their high computational burden and complex deployment. To address this issue, this paper proposes a light-weight framework called Decoupled OSOD (DOSOD), which is a practical and highly efficient solution to support real-time OSOD tasks in robotic systems. Specifically, DOSOD builds upon the YOLO-World pipeline by integrating a vision-language model (VLM) with a detector. A Multilayer Perceptron (MLP) adaptor is developed to transform text embeddings extracted by the VLM into a joint space, within which the detector learns the region representations of class-agnostic proposals. Cross-modality features are directly aligned in the joint space, avoiding the complex feature interactions and thereby improving computational efficiency. DOSOD operates like a traditional closed-set detector during the testing phase, effectively bridging the gap between closed-set and open-set detection. Compared to the baseline YOLO-World, the proposed DOSOD significantly enhances real-time performance while maintaining comparable accuracy. The slight DOSOD-S model achieves a Fixed AP of $26.7\%$, compared to $26.2\%$ for YOLO-World-v1-S and $22.7\%$ for YOLO-World-v2-S, using similar backbones on the LVIS minival dataset. Meanwhile, the FPS of DOSOD-S is $57.1\%$ higher than YOLO-World-v1-S and $29.6\%$ higher than YOLO-World-v2-S. Meanwhile, we demonstrate that the DOSOD model facilitates the deployment of edge devices. The codes and models are publicly available at https://github.com/D-Robotics-AI-Lab/DOSOD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。