提出Beacon模型,让AI更智能地判断何时用工具、如何用工具。
Beacon: Knowing When and How to Perform Agentic Visual Reasoning

- 通过自适应奖励机制,让模型学会判断是否需要工具
- 在复杂任务上提升表现,同时减少简单任务的错误
- 适合需要精准决策的多模态推理场景
当前多模态大语言模型(MLLMs)在复杂任务中进行代理式视觉推理时,常因过度使用工具导致效率低下。本文从工具使用的两个维度——模式适应性(MA)与工具效应(TE)出发,分析发现现有模型普遍缺乏对工具必要性的识别能力,且工具带来的增益常被简单任务中的错误抵消。为此,提出Beacon模型,其核心为必要性感知的自适应奖励机制与提示引导的能力扩展机制,在强化学习阶段分别促进基于任务需求的工具调用和对难题的工具能力增强。在多个基准测试中,Beacon展现出更强的整体性能,显著提升模式适应性和工具实际效果。
原文摘要 · Abstract (English)
The fundamental goal of agentic visual reasoning is to improve the success rate of multimodal large language models (MLLMs) on complex tasks, rather than merely equipping them with a sophisticated yet inefficient reasoning paradigm. In this work, we rethink agentic visual reasoning through two key dimensions of tool use: Mode Adaptiveness (MA) and Tool Effect (TE). Mode Adaptiveness characterizes whether an MLLM can recognize when tools are truly necessary and invoke them accordingly, thereby avoiding unnecessary computational overhead while improving performance on challenging problems that require tool assistance. Tool Effect characterizes the actual impact of tool use: tools should extend the model's capabilities on problems unsolvable through text-only reasoning, while avoiding additional errors on problems that the model can already solve without tools. We conduct a comprehensive analysis to quantify these two properties and empirically reveal that existing agentic visual reasoning models exhibit limited Mode Adaptiveness, while the gains produced by tool use on hard examples are largely offset by the harm introduced on easy examples that the models can already solve. Motivated by these observations, we propose Beacon, a novel agentic visual reasoning model that achieves stronger overall performance, improved Mode Adaptiveness, and genuine tool-induced performance gains. At the core of Beacon are the Necessity-Aware Adaptive Reward and the Hint-Guided Capability Expansion mechanism in the reinforcement learning stage, which respectively encourage adaptive tool invocation based on task necessity and strengthen the model's tool-use capability on the most challenging problems. Extensive experiments across diverse benchmarks demonstrate the strong overall performance of Beacon and its substantial improvements in both Mode Adaptiveness and Tool Effect.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。