通过多步推理提升开放词汇目标检测的伪标签质量
MSPL: Multi-Step Pseudo-Labeling for Open-Vocabulary Object Detection
- 将复杂场景理解拆分为定位、分类、背景判定三步推理
- 在OV-COCO上新类别检测性能提升9.4 AP50
- 适合研究开放词汇检测与视觉语言模型融合的学者
开放词汇目标检测（OVD）旨在识别和定位训练集外的物体类别。现有方法利用视觉-语言模型通过图像-文本对齐生成伪标签，使检测器可泛化至未见类别而无需显式标注。然而，这些方法严重依赖单步图像-文本匹配，忽略了在拥挤或遮挡等语义复杂场景中至关重要的中间推理步骤。本文提出MSPL框架，将多步视觉推理融入伪标签生成过程。该框架将复杂场景理解分解为三个可解释的步骤：物体定位、类别识别与背景归因，其中中间推理状态作为丰富的监督信号。在标准OVD评估协议上的大量实验表明，MSPL达到当前最优性能，伪标签效率显著提升：在OV-COCO上对新类别检测的AP50提高9.4，在OV-LVIS上框和掩码的APr分别提升3.2和2.2。代码与模型已公开于https://github.com/hchoi256/mspl。
原文摘要 · Abstract (English)
Open-vocabulary object detection (OVD) aims to recognize and localize object categories beyond the training set. Recent approaches leverage vision-language models to generate pseudo-labels using image-text alignment, allowing detectors to generalize to unseen classes without explicit supervision. However, these methods depend heavily on single-step image-text matching, neglecting the intermediate reasoning steps crucial for interpreting semantically complex visual contexts, such as crowding or occlusion. In this paper, we introduce MSPL, a framework that incorporates multi-step visual reasoning into the pseudo-labeling process for OVD. It decomposes complex scene understanding into three interpretable steps-object localization, category recognition, and background grounding-where these intermediate reasoning states serve as rich supervision sources. Extensive experiments on standard OVD evaluation protocols demonstrate that MSPL achieves state-of-the-art performance with superior pseudo-labeling efficiency, outperforming the strong baseline by 9.4 AP50 for novel classes on OV-COCO and improving box and mask APr by 3.2 and 2.2, respectively, on OV-LVIS. Code and models are available at https://github.com/hchoi256/mspl.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。