保留工具调用轨迹能显著提升大模型持续学习效果
Trajectory Supervision for Continual Tool-Use Learning in LLMs
- 训练时保留工具调用过程记录，而非只看最终结果
- 保留轨迹使准确率从39.2%提升至56.9%，命名准确率提高7.7点
- 适合研究持续学习与工具使用的大模型开发者
大多数语言模型训练数据仅包含最终结果，缺乏生成过程。本文在工具使用场景下研究这一问题：当模型持续学习新API领域时，保留工具调用轨迹是否优于仅保留最终输出？我们基于API-Bank数据集，使用QLoRA对Llama 3.1 8B Instruct进行微调，设置两种条件：条件A从提示中移除之前的API请求/响应行，仅预测下一步调用；条件B保留完整调用轨迹。单种子实验显示，条件B的最终全调用精确率为56.9%，显著高于条件A的39.2%；同时API名称准确率提升7.7点。但条件B需多消耗25.1%训练令牌，且实验仅使用一个随机种子，任务为下一步调用预测而非完整对话成功。
原文摘要 · Abstract (English)
Most language-model training data shows final artifacts, not the process that produced them. We study a tractable version of this question in tool use: when a model learns a stream of new API domains, does keeping tool-use trajectories help compared with stripping the intermediate API trace? We fine-tune Llama 3.1 8B Instruct with QLoRA on API-Bank using four sequential domain blocks. Condition A strips previous API request/response lines from the prompt and trains the model to predict the next API call. Condition B keeps the trajectory context. In a single-seed pilot, full held-out generation evaluation shows that Condition B reaches 56.9\% final exact full-call accuracy compared with 39.2\% for Condition A. B also improves final API-name accuracy by 7.7 points. However, B uses 25.1\% more training tokens, the run uses one seed, and the task is next-call prediction rather than full dialogue success.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。