arXiv:2606.11652cs.LG2026-06

让小模型更懂用工具,通过模仿强模型的注意力来提升多模态任务准确率。

IAPO: Input Attribution-Aware Policy Optimization for Tool Use in Small Multimodal Agents

论文配图:IAPO: Input Attribution-Aware Policy Optimization for Tool Use in Small Multimodal Agents
图 1 · 摘自论文原文
  • 基于输入注意力对齐设计新强化学习算法
  • 多模态任务下平均提升3%问答准确率
  • 适合研究小模型工具调用与多模态推理

本文研究强化学习方法以提升多模态小型语言模型(SLM)在工具调用方面的能力。现有方法在训练小模型时存在固有局限,尤其在多模态场景中:一是依赖精确匹配真实路径或预定义格式来评估工具使用正确性,但多模态任务常存在多种有效路径,且标注轨迹通常不可得;二是稀疏且脆弱的二值奖励无法提供优化决策过程的有效指导,使小模型难以学习。为此,我们提出输入注意力感知策略优化(IAPO),通过使小模型在输入各组件上的注意力分布与更强教师模型对齐,改进其工具调用能力。在Qwen2.5-VL-3B上的实验表明,该方法在六个测试集上平均提升视觉问答准确率3%,主要得益于模型更好地关注关键输入证据。

原文摘要 · Abstract (English)

This paper investigates reinforcement learning (RL) methods for improving tool-calling capabilities in multimodal small language model (SLM) agents. While existing works have explored various reward designs to improve agentic tool-calling ability, these approaches face inherent limitations for SLM training, especially under multimodal scenarios. First, many existing methods evaluate tool use correctness through exact matching against certain ground-truth or predefined formats. However, this assumption is often unsuitable for multimodal tasks, where multiple tool use paths may be valid and annotated tool trajectories are typically unavailable. Second, such sparse and brittle binary rewards provide little guidance on how to improve the underlying decision process, making them particularly difficult for multimodal SLM to learn from. To address these issues, we propose Input Attribution-Aware Policy Optimization (IAPO), an RL algorithm for improving tool use in multimodal SLM by aligning the model's attribution across input components with that of a stronger teacher. Experiments on Qwen2.5-VL-3B show that the proposed method improves visual question answering accuracy by an average of 3% across six test sets compared with existing visual tool use work, by helping the model attend to the most relevant input evidence.

强化学习多模态小模型工具调用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。