arXiv:2605.13228cs.CVcs.AI2026-05

让视频智能体像搭积木一样递归调用工具,更精准理解复杂视频内容。

ReTool-Video: Recursive Tool-Using Video Agents with Meta-Augmented Tool Grounding

论文配图:ReTool-Video: Recursive Tool-Using Video Agents with Meta-Augmented Tool Grounding
图 1 · 摘自论文原文
  • 构建细粒度工具库,支持134个可组合的多模态操作。
  • 递归调用工具链,将抽象意图逐步转为具体执行动作。
  • 在多个数据集上显著超越现有方法,适合复杂视频理解任务。

视频理解需要主动寻找证据,促使使用工具的视频智能体进行时序推理、跨模态理解与复杂问答。现有视频智能体虽通过检索、记忆、帧检查和验证工具提升了推理能力,但仍存在两大局限:(1) 工具空间粗略,缺乏支持组合推理的细粒度操作;(2) 行动空间扁平,迫使高层视频意图直接映射到原始工具调用。本文提出两种互补设计:首先构建可扩展的MetaAug-Video工具库(MVTL),包含134个注册工具,其中26个为基础工具用于通用多模态信号处理,108个为元工具,支持过滤、聚合、重排序、格式化等中间结果操作。该库支持对结构化视频信息与原始模态证据的双层访问,适配多样推理场景。其次提出ReTool-Video,一种递归工具调用方法,将高层视频意图转化为可执行工具链。匹配的动作直接执行,未匹配的意图交由解析器修复参数、替换工具或拆解任务。这使得时间合并、跨模态验证、重复事件聚合等抽象操作可在运行时逐步转化为具体的多模态操作。在MVBench、MLVU和Video-MME w/o sub上的实验表明,ReTool-Video持续优于强基线。进一步分析显示,递归对齐与细粒度元工具显著提升了复杂视频理解的稳定性与有效性。

原文摘要 · Abstract (English)

Video understanding requires active evidence seeking, motivating tool-augmented video agents for temporal reasoning, cross-modal understanding, and complex question answering. Existing video agents have improved video reasoning with retrieval, memory, frame inspection, and verifier tools, but they still face two limitations: (1) a coarse tool space that lacks fine-grained operations for compositional reasoning; and (2) a flat action space that forces high-level video intents into primitive executable tool calls. In this paper, we address these challenges with two complementary designs. First, we construct a MetaAug-Video Tool Library (MVTL), an extensible tool library with 134 registered tools, including 26 base tools for general multimodal signal processing and 108 meta tools for filtering, aggregation, reranking, formatting, and other intermediate-result operations. MVTL supports dual-level access to both structured video information and raw modal evidence, enabling diverse video reasoning scenarios. Second, we propose ReTool-Video, a recursive tool-using method that grounds high-level video intents into executable tool chains. In ReTool-Video, matched actions are executed directly, while unmatched intents are delegated to a resolver for parameter repair, tool substitution, or decomposition. This allows abstract actions such as temporal merging, cross-modal verification, or repeated-event aggregation to be progressively translated into concrete multimodal operations at runtime. Experiments on MVBench, MLVU, and Video-MME w/o sub. show that ReTool-Video consistently outperforms strong baselines. Further analysis demonstrates that recursive grounding and fine-grained meta tools improve the stability and effectiveness of complex video understanding.

视频理解工具调用递归推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。