用更自然连贯的对话数据提升大模型调用工具的能力
ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis
- 基于图结构采样相关工具组合，提升数据多样性
- 通过计划引导生成连贯对话，提升合成数据质量
- 适合想提升大模型工具调用能力的研究者和开发者
监督微调（SFT）是增强大语言模型（LLM）工具调用能力的常用方法，训练数据通常由合成生成。当前的数据合成过程常随机采样工具，导致工具间缺乏关联性，难以组合，降低数据多样性；同时忽略对话回合间的连贯性，使合成数据与真实场景存在差距。为此，我们提出基于图的采样策略以获取更相关的工具组合，并设计计划生成策略来指导连贯对话的合成。将二者结合，通过多智能体交互式协作构建了名为ToolFlow的工具调用数据合成管道。数据质量评估显示，合成对话在自然性和连贯性上均有提升。最终，我们使用ToolFlow生成的8,000条合成对话对LLaMA-3.1-8B进行SFT，结果表明模型在工具调用性能上达到甚至超越GPT-4水平，同时保持强泛化能力。
原文摘要 · Abstract (English)
Supervised fine-tuning (SFT) is a common method to enhance the tool calling capabilities of Large Language Models (LLMs), with the training data often being synthesized. The current data synthesis process generally involves sampling a set of tools, formulating a requirement based on these tools, and generating the call statements. However, tools sampled randomly lack relevance, making them difficult to combine and thus reducing the diversity of the data. Additionally, current work overlooks the coherence between turns of dialogues, leading to a gap between the synthesized data and real-world scenarios. To address these issues, we propose a Graph-based Sampling strategy to sample more relevant tool combinations, and a Planned-generation strategy to create plans that guide the synthesis of coherent dialogues. We integrate these two strategies and enable multiple agents to synthesize the dialogue data interactively, resulting in our tool-calling data synthesis pipeline ToolFlow. Data quality assessments demonstrate improvements in the naturalness and coherence of our synthesized dialogues. Finally, we apply SFT on LLaMA-3.1-8B using 8,000 synthetic dialogues generated with ToolFlow. Results show that the model achieves tool-calling performance comparable to or even surpassing GPT-4, while maintaining strong general capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。