让大模型代理通过反思失败经验自我进化
Self-Consolidation for Self-Evolving Agents
- 用对比反思提取错误模式,挖掘失败中的学习价值
- 将历史经验压缩为可学习参数,减少上下文负担
- 适合长期自主演化的智能体研究与应用
尽管大型语言模型(LLM)代理展现出强大的问题求解能力,但它们通常作为静态系统运行,缺乏通过持续交互实现自我演化的能力。现有方法主要依赖检索过往成功轨迹作为示范,但存在两大缺陷:一是仅关注成功案例,忽视失败中的教学价值,难以识别并规避重复错误;二是持续积累文本经验导致检索耗时增加,引入噪声,并耗尽当前LLM的最大上下文窗口。为此,我们提出一种新型自演化框架,引入互补机制:首先,采用对比反思策略显式总结易错模式并捕获可复用洞见;其次,提出自巩固机制,将非参数化文本经验提炼为紧凑的可学习参数,使代理能直接将海量历史经验内化至其隐空间。大量实验表明,该方法在长期代理演化中具有显著优势。
原文摘要 · Abstract (English)
While large language model (LLM) agents have demonstrated impressive problem-solving capabilities, they typically operate as static systems, lacking the ability to evolve through lifelong interaction. Existing attempts to bridge this gap primarily rely on retrieving successful past trajectories as demonstrations. However, this paradigm faces two critical limitations. First, by focusing solely on success, agents overlook the rich pedagogical value embedded in failed attempts, preventing them from identifying and avoiding recurrent pitfalls. Second, continually accumulating textual experiences not only increases the time consumption during retrieval but also inevitably introduces noise and exhausts the largest context window of current LLMs. To address these challenges, we propose a novel self-evolving framework for LLM agents that introduces a complementary evolution mechanism: First, a contrastive reflection strategy is introduced to explicitly summarize error-prone patterns and capture reusable insights. Second, we propose a self-consolidation mechanism that distills non-parametric textual experience into compact learnable parameters. This enables the agent to internalize extensive historical experience directly into its latent space. Extensive experiments demonstrate the advantages of our method in long-term agent evolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。