让大模型智能体通过自我进化优化工具使用策略，提升复杂任务解决能力。
EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selection
- 分模块设计工具使用策略，用演化算法迭代优化
- 在多个基准上性能超基线5点以上，支持不同模型迁移
- 基于错误定位和多样性选择，避免策略过拟合
基于大模型的智能体依赖高效的工具使用策略来完成复杂任务，但长期轨迹中的延迟反馈和信用分配难题使策略优化困难。现有方法或过于整体化易导致行为纠缠，或单一维度忽视模块间误差传播。为此，我们提出EvoTool，一种基于无梯度演化范式的自进化框架，将智能体工具使用策略分解为规划、选择、调用和合成四个模块，通过三种新机制在自迭代循环中持续改进。轨迹基错误归因利用诊断痕迹定位失败模块；反馈引导的定向突变通过自然语言批评仅修改对应模块；多样性感知种群选择保留互补候选以保障解多样性。在四个基准测试中，EvoTool在GPT-4.1和Qwen3-8B上均超越强基线超过5个百分点，同时具备更高效率与可迁移性。代码将在论文接收后开源。
原文摘要 · Abstract (English)
LLM-based agents depend on effective tool-use policies to solve complex tasks, yet optimizing these policies remains challenging due to delayed supervision and the difficulty of credit assignment in long-horizon trajectories. Existing optimization approaches tend to be either monolithic, which are prone to entangling behaviors, or single-aspect, which ignore cross-module error propagation. To address these limitations, we propose EvoTool, a self-evolving framework that optimizes a modular tool-use policy via a gradient-free evolutionary paradigm. EvoTool decomposes agent's tool-use policy into four modules, including Planner, Selector, Caller, and Synthesizer, and iteratively improves them in a self-improving loop through three novel mechanisms. Trajectory-Grounded Blame Attribution uses diagnostic traces to localize failures to a specific module. Feedback-Guided Targeted Mutation then edits only that module via natural-language critique. Diversity-Aware Population Selection preserves complementary candidates to ensure solution diversity. Across four benchmarks, EvoTool outperforms strong baselines by over 5 points on both GPT-4.1 and Qwen3-8B, while achieving superior efficiency and transferability. The code will be released once paper is accepted.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。