无需标注数据，用AI自校验训练视觉推理模型
No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers
- 用大模型自动生成推理过程，再通过强化学习优化
- 利用视觉模型自动挖掘错误样例，提升定位准确性
- 适合追求高精度视觉推理的开发者和研究者
视觉推理需要精确的对象定位和对复杂空间关系的理解。现有方法分为两类：仅依赖语言的思维链方法需大规模图像-查询-答案监督，而程序合成方法虽免于训练但存在逻辑缺陷和定位错误。本文提出一种无标注训练框架，通过AI驱动的验证器改进推理与定位能力：语言大模型验证器通过强化学习优化推理路径，视觉大模型验证器通过自动化硬负样本挖掘增强视觉定位。该设计融合了先进语言模型的任务分解能力和高性能视觉模型的批判反馈能力。我们在多种空间推理任务上评估，结果表明该方法在视觉推理性能上超越开源及商用模型，结合改进的视觉定位模型后，进一步优于近期纯文本视觉推理方法。
原文摘要 · Abstract (English)
Visual reasoning is challenging, requiring both precise object grounding and understanding complex spatial relationships. Existing methods fall into two camps: language-only chain-of-thought approaches, which demand large-scale (image, query, answer) supervision, and program-synthesis approaches which use pre-trained models and avoid training, but suffer from flawed logic and erroneous grounding. We propose an annotation-free training framework that improves both reasoning and grounding. Our framework uses AI-powered verifiers: an LLM verifier refines LLM reasoning via reinforcement learning, while a VLM verifier strengthens visual grounding through automated hard-negative mining, eliminating the need for ground truth labels. This design combines the strengths of modern AI systems: advanced language-only reasoning models for decomposing spatial queries into simpler subtasks, and strong vision specialist models improved via performant VLM critics. We evaluate our approach across diverse spatial reasoning tasks, and show that our method improves visual reasoning and surpasses open-source and proprietary models, while with our improved visual grounding model we further outperform recent text-only visual reasoning methods. Project webpage: https://glab-caltech.github.io/valor/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。