解决开放词汇目标检测中新类别召回率低的问题
NoOVD: Novel Category Discovery and Embedding for Open-Vocabulary Object Detection
- 用冻结的视觉语言模型知识引导发现新类别物体
- 在OV-LVIS等数据集上提升新类别检测准确率
- 适合研究开放词汇检测与模型泛化能力的读者
尽管开放词汇目标检测（OVD）取得了显著进展，但训练与测试阶段仍存在显著差距。训练时，区域提议网络（RPN）和区域提议头常将未标注的新类别物体误判为背景，导致部分候选框被提前过滤，其余则在分类头被进一步误判。测试时，这些候选框得分低，在后处理中被移除，造成召回率大幅下降，削弱新类别检测性能。为此，我们提出新框架NoOVD，创新性地引入基于冻结视觉语言模型（VLM）知识的自蒸馏机制。具体设计了K-FPN，利用预训练的VLM知识指导模型发现新类别物体，并实现无额外数据的知识蒸馏，避免新物体被迫对齐背景。同时引入R-RPN，调整推理阶段候选框置信度，提升新类别召回率。在OV-LVIS、OV-COCO和Objects365上的跨数据集评估表明，该方法在多个指标上均表现优异。
原文摘要 · Abstract (English)
Despite the remarkable progress in open-vocabulary object detection (OVD), a significant gap remains between the training and testing phases. During training, the RPN and RoI heads often misclassify unlabeled novel-category objects as background, causing some proposals to be prematurely filtered out by the RPN while others are further misclassified by the RoI head. During testing, these proposals again receive low scores and are removed in post-processing, leading to a significant drop in recall and ultimately weakening novel-category detection performance.To address these issues, we propose a novel training framework-NoOVD-which innovatively integrates a self-distillation mechanism grounded in the knowledge of frozen vision-language models (VLMs). Specifically, we design K-FPN, which leverages the pretrained knowledge of VLMs to guide the model in discovering novel-category objects and facilitates knowledge distillation-without requiring additional data-thus preventing forced alignment of novel objects with background.Additionally, we introduce R-RPN, which adjusts the confidence scores of proposals during inference to improve the recall of novel-category objects. Cross-dataset evaluations on OV-LVIS, OV-COCO, and Objects365 demonstrate that our approach consistently achieves superior performance across multiple metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。