无需额外监督,通过循环知识迁移提升开放词汇目标检测性能。
Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection
- 构建语言查询与视觉区域特征的动态循环对齐机制。
- 在COCO上相比现有方法提升2.9%和10.2%的AP50。
- 适合追求轻量级且可扩展的开放词汇检测场景。
为实现对预定义类别之外物体的全面检测,现有开放词汇目标检测(OVD)方法通常依赖预训练视觉语言模型(VLMs)进行基础类到新类的泛化。然而,为缓解上游图像-文本预训练与下游区域级感知之间的错位,常需额外监督信号,如图像-文本对或自训练生成的伪标注。本文提出无需任何额外监督的CCKT-Det框架。该框架利用VLM提取的语言查询与视觉区域特征,构建循环且动态的知识传递,迫使检测器紧密对齐VLM的视觉-语义空间。具体而言:1)预筛选并注入语义先验以指导查询学习;2)引入区域对比损失增强查询对新类物体的感知能力。CCKT-Det在VLM规模增大时持续提升性能,同时保持适中计算开销。大量实验证明,其在挑战性COCO基准上相比先前最优方法,在无强教师模型和有强教师模型条件下分别取得+2.9%和+10.2%的AP50提升。
原文摘要 · Abstract (English)
In pursuit of detecting unstinted objects that extend beyond predefined categories, prior arts of open-vocabulary object detection (OVD) typically resort to pretrained vision-language models (VLMs) for base-to-novel category generalization. However, to mitigate the misalignment between upstream image-text pretraining and downstream region-level perception, additional supervisions are indispensable, eg, image-text pairs or pseudo annotations generated via self-training strategies. In this work, we propose CCKT-Det trained without any extra supervision. The proposed framework constructs a cyclic and dynamic knowledge transfer from language queries and visual region features extracted from VLMs, which forces the detector to closely align with the visual-semantic space of VLMs. Specifically, 1) we prefilter and inject semantic priors to guide the learning of queries, and 2) introduce a regional contrastive loss to improve the awareness of queries on novel objects. CCKT-Det can consistently improve performance as the scale of VLMs increases, all while requiring the detector at a moderate level of computation overhead. Comprehensive experimental results demonstrate that our method achieves performance gain of +2.9% and +10.2% AP50 over previous state-of-the-arts on the challenging COCO benchmark, both without and with a stronger teacher model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。