让语言智能体学会从经验中积累知识，避免重复试错。
Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge
- 将智能体交互轨迹提炼为可检索的自然语言知识库
- 在三个基准上任务成功率提升10%-20%，步骤减少10%-15%
- 知识库模块化可查，适合需要可解释性的应用
基于大语言模型的智能体虽能完成复杂多步任务，如GUI自动化、工具使用和数据操作，但无法从经验中学习：每次新会话都需从头探索。本文提出BREW（Bootstrapping expeRientially-learned Environmental knoWledge）框架，将智能体过往交互轨迹提炼为结构化、可检索的知识库（KB），包含自然语言配方，记录操作内容、适用场景及注意事项。借鉴程序合成中的“图书馆学习”思想，BREW将记忆分解为模块化、概念聚焦的文档，并将知识库构建形式化为状态空间搜索问题。为此设计了Expand-and-Gather Monte Carlo Tree Search（EG-MCTS）算法，通过奖励引导，在并行的各概念搜索树中联合优化配方准确率与可检索性。进一步引入事后重标注技术，将接近成功的轨迹转化为正向示范，挖掘潜在能力并转化为可复用知识。在OSWorld、tau^2-Bench和SpreadSheetBench三个领域基准上，BREW相较基础智能体实现10%-20%的任务成功率提升，执行步骤减少10%-15%，且持续优于现有内存增强基线，后者甚至可能低于无记忆表现。生成的知识库具备可检查性、模块化与可扩展性，为智能体优化提供透明可控的基础。
原文摘要 · Abstract (English)
Large Language Model (LLM)-based agents are increasingly capable of complex, multi-step tasks such as GUI automation, tool use, and data manipulation, yet they cannot learn from experience: each new session rediscovers solutions from scratch. We introduce BREW (Bootstrapping expeRientially-learned Environmental knoWledge), a framework that distills an agent's past interaction trajectories into a structured, retrievable knowledge base (KB) of natural-language recipes, concept-level procedural documents that capture what to do, when it applies, and what to watch out for. Drawing on the principle of library learning from program synthesis, BREW decomposes agent memory into modular, concept-localized documents and formalizes KB construction as a state-space search problem. To navigate this space, we introduce Expand-and-Gather Monte Carlo Tree Search (EG-MCTS), a reward-guided algorithm that jointly optimizes recipe accuracy and retrievability across parallel, per-concept search trees. We further adapt hindsight relabeling to convert near-miss trajectories into positive demonstrations, surfacing latent agent competencies as reusable knowledge. On three domain-grounded benchmarks, OSWorld, tau^2-Bench, and SpreadSheetBench, BREW achieves 10-20% gains in task success and 10-15% fewer execution steps over base agents, while consistently outperforming existing memory-augmented baselines that can degrade below memoryless performance. The resulting KB is inspectable, modular, and extensible, providing a transparent and controllable substrate for agent optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。