工具增强模型比记忆模型能记住更多事实，且理论上可无限扩展。
Provable Benefits of In-Tool Learning for Large Language Models
- 用外部工具检索事实，比把事实存入模型参数更高效。
- 模型参数数量限制了其记忆能力，但工具使用可突破此瓶颈。
- 适合想提升大模型可扩展性的研究者与工程师。
配备检索、记忆或外部API的工具增强型语言模型正在重塑人工智能，但其理论优势仍不明确。本文通过证明，在工具学习（外部检索）相较于权重学习（记忆）在事实召回方面具有根本性优势。我们指出，仅靠模型参数所能记忆的事实数量受制于其参数量。相反，我们证明工具使用可通过一种简单高效的电路结构实现无限的事实召回。控制实验验证了使用工具的模型始终优于仅依赖记忆的模型。此外，对于预训练的大语言模型，教授工具使用和通用规则比微调具体事实更有效。本工作为工具增强流程提供了理论与实证基础，证明其不仅是实用方案，更是可扩展性上的必然选择。
原文摘要 · Abstract (English)
Tool-augmented language models, equipped with retrieval, memory, or external APIs, are reshaping AI, yet their theoretical advantages remain underexplored. In this paper, we address this question by demonstrating the benefits of in-tool learning (external retrieval) over in-weight learning (memorization) for factual recall. We show that the number of facts a model can memorize solely in its weights is fundamentally limited by its parameter count. In contrast, we prove that tool-use enables unbounded factual recall via a simple and efficient circuit construction. These results are validated in controlled experiments, where tool-using models consistently outperform memorizing ones. We further show that for pretrained large language models, teaching tool-use and general rules is more effective than finetuning facts into memory. Our work provides both a theoretical and empirical foundation, establishing why tool-augmented workflows are not just practical, but provably more scalable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。