通过行为校准提升智能体工具使用效率，解决冗余调用问题。
ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration
- 构建自进化数据飞轮生成高质量训练数据
- 分两阶段校准错误行为，提升工具调用准确率与推理简洁性
- 适合研究工具集成推理的AI系统设计者
大型语言模型可通过工具集成推理（TIR）突破参数知识限制。然而现有训练框架多关注答案准确性，忽视行为模式对齐，导致智能体在执行TIR任务时出现冗余或不足的工具调用。本文提出ET-Agent，一种基于自进化数据飞轮与双阶段行为校准训练的框架，通过生成增强数据提升模型探索能力，并逐步纠正错误行为模式至最优。大量实验验证其在正确性、效率、推理简洁性和工具执行准确率上均优于基线。该框架为TIR领域提供了可落地的训练思路。代码已开源。
原文摘要 · Abstract (English)
Large Language Models (LLMs) can extend their parameter knowledge limits by adopting the Tool-Integrated Reasoning (TIR) paradigm. However, existing LLM-based agent training framework often focuses on answers' accuracy, overlooking specific alignment for behavior patterns. Consequently, agent often exhibits ineffective actions during TIR tasks, such as redundant and insufficient tool calls. How to calibrate erroneous behavioral patterns when executing TIR tasks, thereby exploring effective trajectories, remains an open-ended problem. In this paper, we propose ET-Agent, a training framework for calibrating agent's tool-use behavior through two synergistic perspectives: Self-evolving Data Flywheel and Behavior Calibration Training. Specifically, we introduce a self-evolutionary data flywheel to generate enhanced data, used to fine-tune LLM to improve its exploration ability. Based on this, we implement an two-phases behavior-calibration training framework. It is designed to progressively calibrate erroneous behavioral patterns to optimal behaviors. Further in-depth experiments confirm the superiority of \ourmodel{} across multiple dimensions, including correctness, efficiency, reasoning conciseness, and tool execution accuracy. Our ET-Agent framework provides practical insights for research in the TIR field. Codes can be found in https://github.com/asilverlight/ET-Agent
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。