让大模型智能体通过自身经验持续进化，自动优化解题策略。
EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
- 构建闭环经验生命周期，从行为中提炼可复用的策略
- 在多跳问答任务上超越现有基线模型，表现更优
- 适合研究自主智能体与持续学习的开发者
当前大语言模型（LLM）智能体在工具使用方面表现优异，但缺乏系统性地从自身经验中学习的能力。现有框架多关注外部知识缺口，却忽视了更根本的问题：无法迭代优化解决问题的策略。本文提出EvolveR，一个支持智能体通过完整闭环经验生命周期实现自我进化的框架。该生命周期包含两个阶段：(1) 离线自蒸馏，将智能体的交互轨迹提炼为结构化、可复用的战略原则库；(2) 在线交互，智能体在任务中主动调用蒸馏出的原则指导决策，并积累多样化的行为轨迹。该循环采用策略强化机制，根据性能反馈持续更新智能体。我们在复杂多跳问答基准上验证了EvolveR的有效性，其表现优于多个强基线智能体。本工作为智能体不仅从外部数据学习，也从自身行动后果中学习提供了全面蓝图，推动更自主、持续进化的系统发展。代码已开源：https://github.com/Edaizi/EvolveR。
原文摘要 · Abstract (English)
Current Large Language Model (LLM) agents show strong performance in tool use, but lack the crucial capability to systematically learn from their own experiences. While existing frameworks mainly focus on mitigating external knowledge gaps, they fail to address a more fundamental limitation: the inability to iteratively refine problem-solving strategies. In this work, we introduce EvolveR, a framework designed to enable agent to self-improve through a complete, closed-loop experience lifecycle. This lifecycle comprises two key stages: (1) Offline Self-Distillation, where the agent's interaction trajectories are synthesized into a structured repository of abstract, reusable strategic principles; (2) Online Interaction, where the agent interacts with tasks and actively retrieves distilled principles to guide its decision-making, accumulating a diverse set of behavioral trajectories. This loop employs a policy reinforcement mechanism to iteratively update the agent based on its performance. We demonstrate the effectiveness of EvolveR on complex multi-hop question-answering benchmarks, where it achieves superior performance over strong agentic baselines. Our work presents a comprehensive blueprint for agents that learn not only from external data but also from the consequences of their own actions, paving the way for more autonomous and continuously improving systems. Code is available at https://github.com/Edaizi/EvolveR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。