让视觉语言模型自动进化出推理技能和视觉工具，无需重训练
Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents
- 通过分析自身对错案例，自动生成可复用的推理技能与视觉工具
- 在20个模型-基准组合上平均提升5.6%准确率，且无需更新模型权重
- 适合想提升现有模型性能但无法重训练的研究者或工程师
提升视觉语言模型（VLM）在视觉推理任务上的表现通常需要重新训练或手工设计提示与工具。我们提出Dynamo，一种无需训练的框架，可在不更新模型权重的情况下适应冻结的VLM。在少量标注数据子集上，该智能体分析自身正确与错误尝试，逐步演化出两类互补能力：针对认知瓶颈的可复用推理技能，以及针对感知瓶颈的可执行视觉工具。每个生成的工具均配有指定调用时机的技能，两类能力共同积累至持久化库中。在四个视觉推理基准和五种VLM主干模型上，Dynamo在所有20个模型-基准组合中均提升了直接推理性能（平均+5.6%准确率）。当工具集预先给定时，框架学会何时调用各工具，每步工具选择均优于所有测试骨干模型。相比任务特定的强化学习方法（VTool-R1、DeepEyes），Dynamo仅以极低计算成本就弥补了65%-99%的强化学习差距，并在可用时与强化学习产生叠加增益。
原文摘要 · Abstract (English)
Improving vision-language models (VLMs) on visual reasoning typically requires retraining or hand-designed prompts and tools. We present Dynamo, a training-free framework that adapts a frozen VLM without any weight updates. On a small labeled training subset, the agent inspects its own correct and incorrect attempts and evolves two complementary capabilities: reusable reasoning skills for cognitive bottlenecks, and executable visual tools for perceptual ones. Each generated tool is paired with a skill that specifies when to invoke it, and both capability types accumulate in a persistent library. Across four visual reasoning benchmarks and five VLM backbones, Dynamo improves direct inference on all 20 model--benchmark settings (avg. +5.6 acc). When the tool set is given in advance, the framework learns when to call each tool, and per-step tool choice improves on every tested backbone. Against task-specific RL (VTool-R1, DeepEyes), Dynamo closes 65--99% of the RL gap at a fraction of the compute, and combines additively with RL when available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。