arXiv:2512.08980cs.CVcs.AI2025-12被引 5

让视觉模型学会看多张图并主动检查,提升复杂问答准确率。

Training Multi-Image Vision Agents via End2End Reinforcement Learning

  • 用端到端强化学习训练模型,支持单图或多图输入。
  • 在多个基准上达到当前最优,多图任务提升显著。
  • 适合研究视觉推理与智能体交互的开发者参考。

近期基于视觉语言模型(VLM)的智能体试图复现 OpenAI O3 的“图像思考”能力,但多数开源方法仅限于单图输入,难以应对真实场景中的多图问答任务。为此,我们提出 IMAgent,一个基于端到端强化学习训练的开源视觉智能体,可实现精细的单图与多图推理。推理时,视觉语言模型常逐渐忽略图像内容;为此,我们设计了两个专用工具——视觉反思与验证工具,使模型能主动重聚焦注意力。此外,我们首次从注意力角度揭示工具使用如何提升智能体性能。通过精心设计的两层运动轨迹掩码策略和工具使用奖励增益,IMAgent 仅通过强化学习即获得有效的工具使用范式,无需昂贵的监督微调数据。为激发基础 VLM 的工具使用潜力并填补数据空白,我们利用多智能体系统构建了一个具有挑战性的、视觉丰富的多图问答数据集。大量实验表明,IMAgent 在主流单图与多图基准上均取得当前最优性能,深入分析为社区提供了可操作的洞察。代码与数据将很快发布。

原文摘要 · Abstract (English)

Recent VLM-based agents aim to replicate OpenAI O3's "thinking with images" via tool use, yet most open-source methods restrict inputs to a single image, limiting their applicability to real-world multi-image QA tasks. To address this gap, we propose IMAgent, an open-source visual agent trained with end-to-end reinforcement learning for fine-grained single/multi-image reasoning. During inference, VLMs tend to gradually neglect visual inputs; to mitigate this issue, we design two dedicated tools for visual reflection and verification, enabling the model to actively refocus attention on image content. Beyond that, we, for the first time, reveal how tool usage enhances agent performance from an attention perspective. Equipped with a carefully designed two-layer motion trajectory masking strategy and tool-use reward gain, IMAgent acquires an effective tool-use paradigm through pure reinforcement learning, eliminating the need for costly supervised fine-tuning data. To further unleash the inherent tool-usage potential of the base VLM and fill data gaps, we construct a challenging, visually enriched multi-image QA dataset via multi-agent system. Extensive experiments validate that IMAgent achieves SOTA performance across mainstream single and multi-image benchmarks, and our in-depth analysis offers actionable insights for the community. Code and data will be released soon.

视觉智能体多图推理强化学习工具使用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。