arXiv:2511.09148cs.CLcs.AI2025-11ACL被引 20

让大模型自动发现并修复工具调用缺陷,提升真实任务表现。

LoopTool: Closing the Data-Training Loop for Robust LLM Tool Calls

  • 通过闭环迭代,自动诊断模型弱点并生成针对性训练数据。
  • 8B模型在两个基准上超越32B模型,达到同规模最佳水平。
  • 无需闭源接口,低成本实现数据自净化与能力增强。

将外部工具引入大语言模型可使其完成复杂多步任务。然而,当前工具学习受限于静态合成数据流程——数据生成与模型训练分离,无法动态聚焦模型薄弱环节,且噪声标签持续存在,降低训练效率。本文提出LoopTool,一种全自动、模型感知的数据演化框架,通过紧密集成数据合成与模型训练,实现闭环优化。其包含三个协同模块:(1)贪婪能力探测(GCP)识别模型已掌握与失败的能力;(2)判断引导的标签验证(JGLV)利用开源判别模型发现并修正标注错误,逐步净化数据集;(3)错误驱动的数据扩展(EDDE)基于识别出的失败生成更具挑战性的新样本。该闭环过程运行在低成本开源生态中,摆脱对昂贵闭源API依赖。实验表明,使用LoopTool训练的8B模型显著优于其32B数据生成器,并在BFCL-v3和ACEBench基准上达到同规模新最优性能。研究证明,闭环自修正数据流可大幅增强大模型的工具使用能力。

原文摘要 · Abstract (English)

Augmenting Large Language Models (LLMs) with external tools enables them to execute complex, multi-step tasks. However, tool learning is hampered by the static synthetic data pipelines where data generation and model training are executed as two separate, non-interactive processes. This approach fails to adaptively focus on a model's specific weaknesses and allows noisy labels to persist, degrading training efficiency. We introduce LoopTool, a fully automated, model-aware data evolution framework that closes this loop by tightly integrating data synthesis and model training. LoopTool iteratively refines both the data and the model through three synergistic modules: (1) Greedy Capability Probing (GCP) diagnoses the model's mastered and failed capabilities; (2) Judgement-Guided Label Verification (JGLV) uses an open-source judge model to find and correct annotation errors, progressively purifying the dataset; and (3) Error-Driven Data Expansion (EDDE) generates new, challenging samples based on identified failures. This closed-loop process operates within a cost-effective, open-source ecosystem, eliminating dependence on expensive closed-source APIs. Experiments show that our 8B model trained with LoopTool significantly surpasses its 32B data generator and achieves new state-of-the-art results on the BFCL-v3 and ACEBench benchmarks for its scale. Our work demonstrates that closed-loop, self-refining data pipelines can dramatically enhance the tool-use capabilities of LLMs.

大模型工具调用闭环训练数据进化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。