用语言模型实现内外部规划，让棋类AI达到国际象棋大师水平。
Mastering Board Games by External and Internal Planning with Language Models
- 通过外部搜索引导MCTS，内部训练生成决策树，提升推理能力。
- 在国际象棋中达到大师级水平，仅需接近人类的计算预算。
- 方法可推广至其他复杂任务，适合想提升AI规划能力的研究者。
提升大型语言模型（LLM）的规划与推理能力，是其在复杂高影响力领域可靠应用的关键前提。本文以国际象棋、费舍尔随机棋（Chess960）、连四和六子棋为例，展示基于搜索的规划能显著增强LLM的游戏对弈实力。提出两种主要方法：外部搜索中，模型指导蒙特卡洛树搜索（MCTS）的展开与评估，不依赖外部游戏引擎；内部搜索中，模型被训练生成上下文中的线性化搜索树并输出最终选择。二者均基于在相关领域知识上预训练的语言模型，有效捕捉环境转移与价值函数，且极少出现幻觉。评估结果显示，相比基础模型，新方法在各类棋类任务中均有显著提升，尤其在国际象棋中达到国际大师水平，且计算开销接近人类水平。该结合搜索与领域知识的方法具有通用性，为未来更广泛的应用提供可能。
原文摘要 · Abstract (English)
Advancing planning and reasoning capabilities of Large Language Models (LLMs) is one of the key prerequisites towards unlocking their potential for performing reliably in complex and impactful domains. In this paper, we aim to demonstrate this across board games (Chess, Fischer Random / Chess960, Connect Four, and Hex), and we show that search-based planning can yield significant improvements in LLM game-playing strength. We introduce, compare and contrast two major approaches: In external search, the model guides Monte Carlo Tree Search (MCTS) rollouts and evaluations without calls to an external game engine, and in internal search, the model is trained to generate in-context a linearized tree of search and a resulting final choice. Both build on a language model pre-trained on relevant domain knowledge, reliably capturing the transition and value functions in the respective environments, with minimal hallucinations. We evaluate our LLM search implementations against game-specific state-of-the-art engines, showcasing substantial improvements in strength over the base model, and reaching Grandmaster-level performance in chess while operating closer to the human search budget. Our proposed approach, combining search with domain knowledge, is not specific to board games, hinting at more general future applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。