arXiv:2601.12781cs.AIcs.CV2026-01中稿 · CVPR被引 1

VIRO通过验证机制提升视觉语言推理的鲁棒性,有效避免错误传播。

VIRO: Robust and Efficient Neuro-Symbolic Reasoning with Verification for Referring Expression Comprehension

  • 在每步推理中嵌入轻量级验证器,检查对象存在与空间关系
  • 平衡准确率达61.1%,在无目标场景下仍能正确拒绝错误匹配
  • 适合需要高可靠性与零样本泛化的视觉理解任务

指代表达理解(REC)旨在定位与自然语言查询对应的图像区域。近期神经符号方法利用大语言模型(LLMs)和视觉语言模型(VLMs)进行组合推理,将查询分解为结构化程序并逐步执行。尽管此类方法具备可解释性和强零样本泛化能力,但假设中间推理步骤准确,导致错误传播:误检与无效关系沿推理链累积,产生高置信度的假阳性结果。为此,我们提出验证集成推理算子(VIRO),在推理步骤中嵌入轻量级算子级验证器。每个算子执行并验证输出,如对象存在性或空间关系,实现对无目标情况的验证感知放弃。框架在目标存在与无目标设置下达到61.1%的平衡准确率,展现出对真实世界第一人称数据的泛化能力。VIRO程序失败率不超过0.3%,单次查询运行高效,且支持程序生成与执行解耦,具备良好可扩展性。

原文摘要 · Abstract (English)

Referring Expression Comprehension (REC) aims to localize the image region corresponding to a natural language query. Recent neuro-symbolic REC approaches leverage large language models (LLMs) and vision-language models (VLMs) to perform compositional reasoning, decomposing queries into structured programs and executing them step-by-step. While such approaches achieve interpretable reasoning and strong zero-shot generalization, they assume that intermediate reasoning steps are accurate. However, this assumption causes cascading errors: false detections and invalid relations propagate through the reasoning chain, yielding high-confidence false positives even when no target is present in the image. To address this limitation, we introduce Verification-Integrated Reasoning Operators (VIRO), a neuro-symbolic framework that embeds lightweight operator-level verifiers within reasoning steps. Each operator executes and validates its output, such as object existence or spatial relationships, allowing the system to robustly handle no-target cases through verification-aware abstention. Our framework achieves state-of-the-art performance, reaching 61.1% balanced accuracy across target-present and no-target settings, and demonstrates generalization to real-world egocentric data. VIRO also shows high reliability with a program failure rate of at most 0.3%, efficient per-query runtime, and scalability through decoupled program generation and execution.

视觉理解神经符号推理验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。