让智能体通过试错自主进化技能,摆脱人工流程设计束缚。
Rethinking Agent Design: From Top-Down Workflows to Bottom-Up Skill Evolution
- 智能体从经验中自我学习,通过试错与反思逐步积累技能。
- 在《弑神者》和《文明5》中实现纯自主交互,无需游戏特化提示。
- 多智能体协作加速技能演化,适合开放复杂环境应用。
当前基于大模型的智能体框架多采用自上而下的设计:人类拆解任务、定义流程并分配智能体执行各步骤。尽管在基准任务中有效,但此类系统依赖人工维护,忽视了智能体从经验中学习的潜力。近期,Silver 和 Sutton(2025)提出转向以经验驱动的学习新范式。本文通过引入自下而上的智能体设计,实现了这一愿景——模仿人类学习过程,智能体通过试错与反思逐步获得能力,并抽象出可共享、可扩展的技能。随着部署的智能体增多,多样化的经验推动集体演化,而非静态复制。我们在《弑神者》和《文明5》中验证该范式,智能体以原始视觉输入和鼠标输出感知与操作,使用统一、游戏无关的代码库,无任何游戏特化提示或特权接口,完全通过自主交互习得技能,展现了该范式在复杂真实环境中的潜力。代码已开源:https://github.com/AngusDujw/Bottom-Up-Agent。
原文摘要 · Abstract (English)
Most LLM-based agent frameworks adopt a top-down philosophy: humans decompose tasks, define workflows, and assign agents to execute each step. While effective on benchmark-style tasks, such systems rely on designer updates and overlook agents' potential to learn from experience. Recently, Silver and Sutton(2025) envision a shift into a new era, where agents could progress from a stream of experiences. In this paper, we instantiate this vision of experience-driven learning by introducing a bottom-up agent paradigm that mirrors the human learning process. Agents acquire competence through a trial-and-reasoning mechanism-exploring, reflecting on outcomes, and abstracting skills over time. Once acquired, skills can be rapidly shared and extended, enabling continual evolution rather than static replication. As more agents are deployed, their diverse experiences accelerate this collective process, making bottom-up design especially suited for open-ended environments. We evaluate this paradigm in Slay the Spire and Civilization V, where agents perceive through raw visual inputs and act via mouse outputs, the same as human players. Using a unified, game-agnostic codebase without any game-specific prompts or privileged APIs, our bottom-up agents acquire skills entirely through autonomous interaction, demonstrating the potential of the bottom-up paradigm in complex, real-world environments. Our code is available at https://github.com/AngusDujw/Bottom-Up-Agent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。