测试时自动精简用户自定义词汇，提升语言驱动目标检测效果
Test-time Vocabulary Adaptation for Language-driven Object Detection
- 通过图像描述生成与名词解析，动态筛选相关类别
- 在COCO和Objects365上显著提升三大主流检测器性能
- 无需训练，可直接部署于任意开放词汇检测模型
开放词汇目标检测模型允许用户在测试时自由指定自然语言描述的类别词汇，以引导特定目标的检测。然而，词汇可能过于宽泛或错误定义，影响检测性能。本文提出一种即插即用的词汇适配器（VocAda），在推理时自动优化用户定义的词汇，使其更贴合当前图像内容。VocAda不需训练，分三步运行：首先使用图像字幕生成器描述可见物体，其次从字幕中解析名词，最后从用户词汇中筛选出与图像相关的类别并剔除无关项。在COCO和Objects365数据集上，使用三种先进检测器的实验表明，VocAda能持续提升性能，验证了其通用性。代码已开源。
原文摘要 · Abstract (English)
Open-vocabulary object detection models allow users to freely specify a class vocabulary in natural language at test time, guiding the detection of desired objects. However, vocabularies can be overly broad or even mis-specified, hampering the overall performance of the detector. In this work, we propose a plug-and-play Vocabulary Adapter (VocAda) to refine the user-defined vocabulary, automatically tailoring it to categories that are relevant for a given image. VocAda does not require any training, it operates at inference time in three steps: i) it uses an image captionner to describe visible objects, ii) it parses nouns from those captions, and iii) it selects relevant classes from the user-defined vocabulary, discarding irrelevant ones. Experiments on COCO and Objects365 with three state-of-the-art detectors show that VocAda consistently improves performance, proving its versatility. The code is open source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。