让大模型自己出题自己练，提升工具使用能力
Self-Challenging Language Model Agents
- 模型自动生成带验证函数的任务，确保质量
- 在两个基准上性能提升超2倍，仅用自生成数据
- 适合研究智能体训练与自动评估的学者
大语言模型正成为具备工具使用能力的智能体基础，但训练这类智能体面临任务、工具和评估标准需人工构建的挑战。本文提出自挑战框架（Self-Challenging），让智能体自行生成高质量任务进行训练。模型先以挑战者角色，通过与工具交互生成任务，任务形式为「代码即任务」（Code-as-Task），包含指令、验证函数及正负例测试集，可有效筛选优质任务。随后模型切换为执行者角色，利用强化学习在这些任务上训练，并以评估反馈作为奖励信号。在M3ToolEval和TauBench两个多轮工具使用基准上的实验表明，该框架使Llama-3.1-8B-Instruct模型性能提升超过两倍，且仅依赖自生成数据。
原文摘要 · Abstract (English)
Large language models are quickly becoming the foundation for intelligent agents that are capable of using tools. However, training such agents is challenging because it requires human creation and annotation of a diverse set of tasks, tools, and evaluation criteria. In this paper, we propose the Self-Challenging framework for training an agent on high-quality tasks that are generated by itself. The agent first plays the role of challenger and generates a task after interacting with the given tools. The tasks take the form of a novel general class of problems termed Code-as-Task, which are defined by an instruction, a verification function and solution and failure cases which serve as tests, allowing to filter only for high-quality tasks. The agent then takes an executor role and trains on those tasks with reinforcement learning using the evaluation feedback as a reward. Evaluation on two existing multi-turn tool-use agent benchmarks, M3ToolEval and TauBench, shows the Self-Challenging framework achieves over a two-fold improvement in Llama-3.1-8B-Instruct, despite using only self-generated training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。