让每一步工具调用都精准获取必要证据,提升视觉语言模型的推理效率。
Making Every Tool Call Count: Necessary Tool-Evidence Path Rewards for Agentic Vision-Language Models

- 通过指定关键证据与对应工具调用路径,实现细粒度监督。
- 在7个基准上,8B模型准确率和工具使用效率显著提升。
- 适合需要高效调用外部工具的复杂视觉问答任务研究者。
现代视觉语言模型(VLMs)可直接回答许多图像相关问题,但在需要细粒度视觉细节或外部知识的复杂查询中表现不佳。为获取缺失证据,代理型VLM会调用图像裁剪、图像搜索、文本搜索等工具。然而现有训练范式主要基于最终答案正确性评估工具使用,导致证据获取与利用缺乏充分监督。这引发两个核心问题:(i) 模型常发出冗余或偏离目标的工具调用,无法获取必要证据;(ii) 即使调用恰当工具,也常无法从观察结果中提取必要信息。为此,我们提出NTEP(必要工具-证据路径)标注方案,明确每个查询所需的必要外部证据及对应工具调用。基于此,我们设计NTEP-R(NTEP奖励机制),确保每次工具调用均严格推进推理过程。具体而言,该方法奖励代理在调用前意图与必要证据目标一致,并在调用后能从观察中总结出与必要证据对齐的信息。此外,引入非重复目标正则化项,惩罚重复调用已满足的NTEP目标。在七个图像相关基准上的广泛评估表明,我们的8B参数模型NTEP-8B在统一三工具框架下,显著提升了搜索导向准确率与工具使用效率。结果凸显了细粒度工具-证据路径监督对训练鲁棒代理型VLM的关键价值。
原文摘要 · Abstract (English)
Modern vision-language models (VLMs) can directly answer many image-grounded questions, yet they often struggle with complex queries requiring fine-grained visual details or external knowledge. To acquire this missing evidence, agentic VLMs invoke tools such as image cropping, image search, and text search. However, existing training paradigms primarily evaluate tool-use based on final answer correctness, leaving evidence acquisition and utilization insufficiently supervised. This leads to two critical shortcomings: (i) models frequently issue redundant or off-target tool calls that fail to gather necessary evidence, and (ii) even when appropriate tools are called, models often fail to extract the necessary information from the resulting observations. To address these limitations, we introduce the NTEP (Necessary Tool-Evidence Path), a novel annotation scheme that explicitly specifies the essential external evidence and corresponding tool calls for each query. Building upon this, we propose NTEP-R (NTEP Reward), a supervision mechanism ensuring that each tool invocation strictly advances the reasoning process toward the final solution. Specifically, our approach rewards the agent for aligning its pre-call intent with a necessary evidence-seeking goal, and for ensuring the information summarized from the post-call observation aligns with the necessary evidence. Furthermore, we introduce a non-repeated-goal regularizer to penalize redundant calls that revisit satisfied NTEP goals. Extensive evaluations on seven image-grounded benchmarks demonstrate that our 8B-parameter instantiation, NTEP-8B, significantly improves both search-oriented accuracy and tool-use efficiency within a unified three-tool framework. These results highlight the critical value of fine-grained tool-evidence path supervision for training robust agentic VLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。