让语言代理在推理时自我改进，提升复杂任务成功率。
Contextual Experience Replay for Self-Improvement of Language Agents
- 用动态记忆池整合过往经验，增强上下文中的知识调用
- 在WebArena上使成功率提升51%，达36.7%
- 适合需持续学习的智能体应用，如网页导航
大型语言模型（LLM）代理已应用于网页导航等序列决策任务，但缺乏环境特异性经验时往往表现不佳。当前的LLM代理也未设计为在推理阶段持续学习过去经验，而这对于获取环境特定知识至关重要。为此，我们提出上下文体验回放（CER），一种无需训练的框架，可在上下文窗口内实现语言代理的高效自我改进。具体而言，CER将历史经验累积并合成至动态记忆缓冲区，涵盖环境动态与常见决策模式，使代理能在新任务中检索并增强自身相关知识，从而提升在复杂环境中的适应能力。我们在具有挑战性的WebArena和VisualWebArena基准上评估CER。在VisualWebArena上，CER取得31.9%的竞争力表现；在WebArena上，平均成功率达36.7%，相较GPT-4o基线提升51.0%。我们还进行了全面分析，验证其效率、有效性并加深理解。
原文摘要 · Abstract (English)
Large language model (LLM) agents have been applied to sequential decision-making tasks such as web navigation, but without any environment-specific experiences, they often fail in these complex tasks. Moreover, current LLM agents are not designed to continually learn from past experiences during inference time, which could be crucial for them to gain these environment-specific experiences. To address this, we propose Contextual Experience Replay (CER), a training-free framework to enable efficient self-improvement for language agents in their context window. Specifically, CER accumulates and synthesizes past experiences into a dynamic memory buffer. These experiences encompass environment dynamics and common decision-making patterns, allowing the agents to retrieve and augment themselves with relevant knowledge in new tasks, enhancing their adaptability in complex environments. We evaluate CER on the challenging WebArena and VisualWebArena benchmarks. On VisualWebArena, CER achieves a competitive performance of 31.9%. On WebArena, CER also gets a competitive average success rate of 36.7%, relatively improving the success rate of the GPT-4o agent baseline by 51.0%. We also conduct a comprehensive analysis on it to prove its efficiency, validity and understand it better.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。