让大模型无示范地学会用工具，提升推理准确性。
Self-Training Large Language Models for Tool-Use Without Demonstrations
- 用自训练生成工具使用轨迹，无需人工示范。
- 在PopQA上提升3.7%，其他数据集效果参差。
- 适合想减少人工标注、探索自主工具使用的研究者。
大语言模型仍易出现事实错误和计算失误，包括幻觉和数学推理错误。近期工作通过引入工具缓解这些问题，但通常依赖精心筛选的示范数据。本文探讨大模型能否在无示范情况下学习使用工具。首先分析零样本提示策略引导模型使用工具；其次提出一种自训练方法，利用模型自身合成工具使用轨迹。在TriviaQA和GSM8K等问答数据集构建的训练集上，对比监督微调与偏好微调的效果。实验表明，工具使用显著提升了长尾知识任务的表现：在仅用于评估的PopQA上提升3.7%；但在TriviaQA、GSM8K和NQ-Open上结果混合。研究揭示了无示范下集成外部工具的潜力与挑战。
原文摘要 · Abstract (English)
Large language models (LLMs) remain prone to factual inaccuracies and computational errors, including hallucinations and mistakes in mathematical reasoning. Recent work augmented LLMs with tools to mitigate these shortcomings, but often requires curated gold tool-use demonstrations. In this paper, we investigate whether LLMs can learn to use tools without demonstrations. First, we analyse zero-shot prompting strategies to guide LLMs in tool utilisation. Second, we propose a self-training method to synthesise tool-use traces using the LLM itself. We compare supervised fine-tuning and preference fine-tuning techniques for fine-tuning the model on datasets constructed using existing Question Answering (QA) datasets, i.e., TriviaQA and GSM8K. Experiments show that tool-use enhances performance on a long-tail knowledge task: 3.7% on PopQA, which is used solely for evaluation, but leads to mixed results on other datasets, i.e., TriviaQA, GSM8K, and NQ-Open. Our findings highlight the potential and challenges of integrating external tools into LLMs without demonstrations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。