让大模型同时学会写工具和用工具，性能超越现有方法。
Joint Optimization of Tool Creation and Use for Large Language Model Agents
- 用强化学习统一训练写工具和用工具的策略，双向反馈提升效果。
- 40.4分（TabMWP-Hard）、42.6分（GQA）表现领先，无需视觉/表格数据。
- 小模型写的工具能提升更大模型表现，适合智能体系统研发者。
增强型语言模型受限于人类编写的API；现有工具创建系统在推理时调用冻结的LLM，导致写工具与用工具的模型脱节，缺乏执行信号。本文提出SMITH（Schema-grounded Multi-task Iterative Tool Honing），一种强化学习框架，将工具创建与使用联合训练于单一策略中。每次迭代为构建任务（根据示例写工具）或使用任务（在保留问题上调用工具池）。三个独立奖励轴分别捕捉模式、代码和结果失败，各自贡献梯度。使用13个程序性推理任务和精确验证器训练的4B Qwen3模型，在保留任务上达到79.8的宏平均准确率，优于所有对比方法，超过未训练的30B-A3B工具写作者。在TabMWP-Hard上达40.4，在跨域GQA上达42.6（比最优同架构推理基线高7.6），且无需视觉或表格训练数据。由4B模型生成的工具也提升了LFM-2.5-350M和Qwen3-30B-A3B在相同推理任务上的表现。
原文摘要 · Abstract (English)
Tool-augmented language models are bounded by the APIs humans bothered to write; existing tool-creation systems patch this by prompting a frozen LLM at inference time, leaving the model that writes a tool decoupled from the one that uses it, with no signal that the schemas it produces are schemas it can invoke. We propose SMITH (Schema-grounded Multi-task Iterative Tool Honing), a reinforcement learning framework that jointly trains tool creation and tool use inside a single policy. Each rollout is either a build task (write a tool from a few examples) or a use task (invoke a pooled tool on a held-out question). Three separate reward axes catch schema, code, and outcome failures independently, so each failure mode contributes its own gradient. A 4B Qwen3 trained with SMITH on 13 procedural reasoning tasks with exact verifiers reaches 79.8 macro-average accuracy on held-out tasks, the best across all evaluated methods and ahead of an untrained 30B-A3B tool-writer. It also reaches 40.4 on TabMWP-Hard and 42.6 on out-of-domain GQA (+7.6 over the best same-backbone inference-time baseline), without any visual or tabular training data. Tools written by our 4B models also lifted the performance of LFM-2.5-350M and Qwen3-30B-A3B under same reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。