让大模型代理通过自我迭代优化推理路径,显著提升复杂问题解决能力。
SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents
- 基于历史推理轨迹进行修正、重组与精炼,实现智能进化。
- 在SWE-bench Verified上使五种大模型平均提升55%性能,达到开源代理最优。
- 适合需要长期推理和高可靠性的自动化代码修复场景。
基于大语言模型(LLM)的智能体在多步交互中展现出强大的复杂推理与工具使用能力。然而,其任务求解过程——即通向目标的交互轨迹——仍被低估。这些轨迹蕴含丰富反馈,可引导智能体走向正确解题方向。现有方法如蒙特卡洛树搜索虽能平衡探索与利用,却忽视轨迹间关联性,缺乏搜索空间多样性,导致冗余推理与次优结果。为此,我们提出SE-Agent,一种支持智能体自我演化的框架,通过修订、重组与精炼先前轨迹,实现推理过程的持续优化。该机制一方面通过跨轨迹启发扩展搜索空间,突破局部最优;另一方面有效抑制低质量推理路径的影响。在SWE-bench Verified上评估表明,集成SE-Agent后,五种主流大模型相对性能提升最高达55%,达成当前开源代理最佳表现。代码与演示材料已公开于https://github.com/JARVIS-Xs/SE-Agent。
原文摘要 · Abstract (English)
Large Language Model (LLM)-based agents have recently shown impressive capabilities in complex reasoning and tool use via multi-step interactions with their environments. While these agents have the potential to tackle complicated tasks, their problem-solving process, i.e., agents' interaction trajectory leading to task completion, remains underexploited. These trajectories contain rich feedback that can navigate agents toward the right directions for solving problems correctly. Although prevailing approaches, such as Monte Carlo Tree Search (MCTS), can effectively balance exploration and exploitation, they ignore the interdependence among various trajectories and lack the diversity of search spaces, which leads to redundant reasoning and suboptimal outcomes. To address these challenges, we propose SE-Agent, a Self-Evolution framework that enables Agents to optimize their reasoning processes iteratively. Our approach revisits and enhances former pilot trajectories through three key operations: revision, recombination, and refinement. This evolutionary mechanism enables two critical advantages: (1) it expands the search space beyond local optima by intelligently exploring diverse solution paths guided by previous trajectories, and (2) it leverages cross-trajectory inspiration to efficiently enhance performance while mitigating the impact of suboptimal reasoning paths. Through these mechanisms, SE-Agent achieves continuous self-evolution that incrementally improves reasoning quality. We evaluate SE-Agent on SWE-bench Verified to resolve real-world GitHub issues. Experimental results across five strong LLMs show that integrating SE-Agent delivers up to 55% relative improvement, achieving state-of-the-art performance among all open-source agents on SWE-bench Verified. Our code and demonstration materials are publicly available at https://github.com/JARVIS-Xs/SE-Agent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。