不用训练，用视觉语言模型判断区域对错，就能实现顶尖零样本指代理解。
Zero-Shot Referring Expression Comprehension via Vison-Language True/False Verification
- 将指代理解转为区域真假判断，避免不同区域间的干扰。
- 在三个数据集上超越了需训练的基线模型，包括有微调的GroundingDINO。
- 适合希望跳过训练、快速部署零样本指代理解的开发者使用。
指代表达理解（REC）通常依赖任务特定的定位模型。本文表明，无需任何REC训练的零样本流程也能达到竞争力甚至更优的性能。方法将REC重定义为基于区域的视觉-语言真假验证：在使用COCO-clean通用检测器（YOLO-World）生成候选区域后，通用视觉语言模型（VLM）独立判断每个区域是否匹配描述。该流程简化了推理，减少跨区域干扰，支持拒绝回答和多匹配情况，且无需微调。在RefCOCO、RefCOCO+和RefCOCOg三个数据集上，该方法不仅优于零样本的GroundingDINO基线，还超过了在REC上训练过的GroundingDINO及GroundingDINO+CRG的报告结果。使用相同候选框的对照实验确认，验证机制显著优于基于选择的提示方法，且在开放VLM上依然有效。整体表明，流程设计比任务专用预训练更能驱动强零样本性能。
原文摘要 · Abstract (English)
Referring Expression Comprehension (REC) is usually addressed with task-trained grounding models. We show that a zero-shot workflow, without any REC-specific training, can achieve competitive or superior performance. Our approach reformulates REC as box-wise visual-language verification: given proposals from a COCO-clean generic detector (YOLO-World), a general-purpose VLM independently answers True/False queries for each region. This simple procedure reduces cross-box interference, supports abstention and multiple matches, and requires no fine-tuning. On RefCOCO, RefCOCO+, and RefCOCOg, our method not only surpasses a zero-shot GroundingDINO baseline but also exceeds reported results for GroundingDINO trained on REC and GroundingDINO+CRG. Controlled studies with identical proposals confirm that verification significantly outperforms selection-based prompting, and results hold with open VLMs. Overall, we show that workflow design, rather than task-specific pretraining, drives strong zero-shot REC performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。