用语言模型让智能体通过推理和规划自主学会玩游戏
Cogito, Ergo Ludo: An Agent that Learns to Play by Reasoning and Planning
- 用大语言模型构建环境规则与策略的显式语言描述
- 从零开始通过反思轨迹,自动发现游戏规则并生成策略手册
- 适合追求可解释智能体和少样本学习的研究者
追求能掌握复杂环境的人工智能体已取得显著进展,但现有深度强化学习方法往往依赖海量经验,且知识隐含于神经网络权重中。我们提出一种新范式:智能体通过推理与规划来学习游戏。引入名为 Cogito, ergo ludo (CEL) 的新型架构,利用大语言模型(LLM)建立对环境机制与自身策略的显式语言理解。从无先验知识(仅知动作集)出发,CEL 在交互与反思循环中运行:每轮结束后分析完整轨迹,同时进行规则归纳(优化环境动态模型)与策略与玩法手册提炼(将经验转化为可执行战略)。在多种网格世界任务(如 Minesweeper、Frozen Lake、Sokoban)上评估表明,CEL 能自主发现规则并从稀疏奖励中发展出有效策略。消融实验确认迭代反思过程对持续学习至关重要。本工作展示了一条通向更通用、可解释智能体的道路——不仅高效行动,还能通过显式推理不断改进对世界的透明认知。
原文摘要 · Abstract (English)
The pursuit of artificial agents that can learn to master complex environments has led to remarkable successes, yet prevailing deep reinforcement learning methods often rely on immense experience, encoding their knowledge opaquely within neural network weights. We propose a different paradigm, one in which an agent learns to play by reasoning and planning. We introduce Cogito, ergo ludo (CEL), a novel agent architecture that leverages a Large Language Model (LLM) to build an explicit, language-based understanding of its environment's mechanics and its own strategy. Starting from a tabula rasa state with no prior knowledge (except action set), CEL operates on a cycle of interaction and reflection. After each episode, the agent analyzes its complete trajectory to perform two concurrent learning processes: Rule Induction, where it refines its explicit model of the environment's dynamics, and Strategy and Playbook Summarization, where it distills experiences into an actionable strategic playbook. We evaluate CEL on diverse grid-world tasks (i.e., Minesweeper, Frozen Lake, and Sokoban), and show that the CEL agent successfully learns to master these games by autonomously discovering their rules and developing effective policies from sparse rewards. Ablation studies confirm that the iterative process is critical for sustained learning. Our work demonstrates a path toward more general and interpretable agents that not only act effectively but also build a transparent and improving model of their world through explicit reasoning on raw experience.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。