新基准+新方法,让AI看图推理更精准可解释
Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology
- 设计可追踪证据的视觉推理评测基准TreeBench
- 主流模型在该基准上最高仅54.87%准确率
- 提出联合训练方法,提升定位与推理可解释性
OpenAI-o3等模型通过动态参考视觉区域实现类人图像推理,但缺乏全面评估基准。为此,我们构建了基于三个原则的TreeBench(可追踪证据评估基准):(1)对复杂场景中微小目标的精准感知,(2)通过边界框评估实现证据可追踪,(3)测试对象交互与空间层次的高阶推理。从SA-1B采样1000张高质量图像,经八位多模态专家标注,三轮质量控制后形成405个挑战性图文问答对,最先进模型如OpenAI-o3准确率仅为54.87%。我们进一步提出TreeVGR训练范式,结合强化学习联合监督定位与推理,初始化自Qwen2.5-VL-7B,在V* Bench(+16.8)、MME-RealWorld(+12.6)和TreeBench(+13.4)上显著提升,证明可追踪性是推进视觉接地推理的关键。代码已开源。
原文摘要 · Abstract (English)
Models like OpenAI-o3 pioneer visual grounded reasoning by dynamically referencing visual regions, just like human "thinking with images". However, no benchmark exists to evaluate these capabilities holistically. To bridge this gap, we propose TreeBench (Traceable Evidence Evaluation Benchmark), a diagnostic benchmark built on three principles: (1) focused visual perception of subtle targets in complex scenes, (2) traceable evidence via bounding box evaluation, and (3) second-order reasoning to test object interactions and spatial hierarchies beyond simple object localization. Prioritizing images with dense objects, we initially sample 1K high-quality images from SA-1B, and incorporate eight LMM experts to manually annotate questions, candidate options, and answers for each image. After three stages of quality control, TreeBench consists of 405 challenging visual question-answering pairs, even the most advanced models struggle with this benchmark, where none of them reach 60% accuracy, e.g., OpenAI-o3 scores only 54.87. Furthermore, we introduce TreeVGR (Traceable Evidence Enhanced Visual Grounded Reasoning), a training paradigm to supervise localization and reasoning jointly with reinforcement learning, enabling accurate localizations and explainable reasoning pathways. Initialized from Qwen2.5-VL-7B, it improves V* Bench (+16.8), MME-RealWorld (+12.6), and TreeBench (+13.4), proving traceability is key to advancing vision-grounded reasoning. The code is available at https://github.com/Haochen-Wang409/TreeVGR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。