用大模型+记忆机制提升文本游戏规划效率，一次尝试即胜出。
Monte Carlo Planning with Large Language Model for Text-Based Game Agents
- 结合大模型语言理解与树搜索探索，引入动态记忆机制。
- 在Jericho基准上首次规划即超越多轮迭代的强方法。
- 适合需要高效推理的文本交互类智能体研究者。
文本游戏为基于语言的自主智能体提供了重要实验环境。然而，传统的先规划后学习范式（如结合蒙特卡洛树搜索MCTS与强化学习RL）因需大量迭代而耗时严重，且虽能驱动不确定性探索，却缺乏语言理解和推理能力。本文提出蒙特卡洛规划与动态记忆引导的大语言模型算法（MC-DML），利用大语言模型（LLM）的语言理解与推理能力，结合树搜索的探索优势。具体地，通过引入试内与跨试次记忆机制，使模型可从过往经验中学习，并在规划过程中动态调整动作评估。我们在Jericho基准的一系列文本游戏中进行实验，结果表明，MC-DML在初始规划阶段即显著提升性能，优于需多次迭代的现有先进方法。这验证了该算法的有效性，为复杂环境中更高效的语言引导规划开辟了新路径。
原文摘要 · Abstract (English)
Text-based games provide valuable environments for language-based autonomous agents. However, planning-then-learning paradigms, such as those combining Monte Carlo Tree Search (MCTS) and reinforcement learning (RL), are notably time-consuming due to extensive iterations. Additionally, these algorithms perform uncertainty-driven exploration but lack language understanding and reasoning abilities. In this paper, we introduce the Monte Carlo planning with Dynamic Memory-guided Large language model (MC-DML) algorithm. MC-DML leverages the language understanding and reasoning capabilities of Large Language Models (LLMs) alongside the exploratory advantages of tree search algorithms. Specifically, we enhance LLMs with in-trial and cross-trial memory mechanisms, enabling them to learn from past experiences and dynamically adjust action evaluations during planning. We conduct experiments on a series of text-based games from the Jericho benchmark. Our results demonstrate that the MC-DML algorithm significantly enhances performance across various games at the initial planning phase, outperforming strong contemporary methods that require multiple iterations. This demonstrates the effectiveness of our algorithm, paving the way for more efficient language-grounded planning in complex environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。