用外部资料生成可执行工具,让大模型在知识盲区也能推理
RefTool: Reference-Guided Tool Creation for Knowledge-Intensive Reasoning
- 基于参考文本生成可执行工具并按层级组织
- 在因果、物理、化学任务上平均准确率提升12.3%
- 适合需要跨领域推理和低资源场景的复杂任务
大语言模型可通过使用外部工具增强推理能力,但许多任务缺乏预定义工具。现有方法依赖模型内部知识,当任务超出其知识范围时表现不佳。为此,我们提出RefTool,一种基于参考内容的自动工具创建框架,利用教科书和知识片段等外部材料。RefTool包含两个模块:(1)工具创建,模型从参考内容生成可执行工具,通过示例验证并分层组织为工具箱;(2)工具使用,模型导航工具箱结构选择并应用合适工具解决问题。在因果、物理和化学基准测试中,RefTool平均准确率比现有方法高12.3%,且成本低、泛化性强,可拓展至非科学任务,如极低资源语言翻译。分析表明,以参考材料为基础生成工具更具准确性和忠实性,层级结构有助于高效选择。RefTool使大模型突破内部知识限制,在知识密集型领域实现可泛化的推理。
原文摘要 · Abstract (English)
Large Language Models (LLMs) can enhance their reasoning capabilities by using external tools. However, many tasks lack predefined tools. Prior works have explored instructing LLMs to generate tools on their own, but such approaches depend heavily on internal knowledge and struggle when tasks fall outside the model's knowledge scope. To address this limitation, we propose RefTool, a reference-guided framework for automatic tool creation that leverages external materials, such as textbooks and knowledge snippets. RefTool consists of two modules: (1) tool creation, where LLMs generate executable tools from reference content, validate them using illustrative examples, and organize them hierarchically into a toolbox; and (2) tool utilization, where LLMs navigate the toolbox structure to select and apply the appropriate tools to solve problems. Experiments on causality, physics, and chemistry benchmarks demonstrate that RefTool outperforms existing tool-creation and domain-specific reasoning methods by 12.3% on average accuracy, while being cost-efficient and broadly generalizable to non-scientific tasks, e.g., extremely low-resource language translation. Analyses reveal that grounding tool creation in references produces accurate and faithful tools, and that the hierarchical structure facilitates effective tool selection. RefTool enables LLMs to overcome internal knowledge limitations, advancing generalizable reasoning in knowledge-intensive domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。