用大模型提升游戏博弈搜索,让AI在宝可梦对战中达到人类顶尖水平。
PokéChamp: an Expert-level Minimax Language Agent
- 用大模型替代搜索中的动作采样、对手建模和估值函数
- 不训练模型即实现84%胜率,对战人类高手胜率超64%
- 开源300万场对战数据与评测基准,适合研究博弈AI的学者
我们提出PokéChamp,一种基于大语言模型(LLMs)的极小极大博弈代理,用于宝可梦对战。该框架适用于双人竞技类游戏,利用LLM的通用能力增强极小极大树搜索。具体地,LLM取代了三个核心模块:(1)玩家动作采样,(2)对手建模,(3)价值函数估计,使代理能有效利用对战历史与人类知识,缩小搜索空间并缓解部分可观测性问题。本框架无需额外的LLM训练。我们在流行的第九世代OU规则下评估该模型。当使用GPT-4o时,其对现有最佳基于LLM的机器人胜率为76%,对最强规则基机器人胜率为84%;即使采用开源的80亿参数Llama 3.1模型,也能以64%胜率超越此前最佳的基于GPT-4o的机器人Pokéllmon。PokéChamp在宝可梦对战平台线上段位上预计拥有1300–1500的Elo分,位列人类玩家前30%至10%。此外,本工作构建了目前最大的真实玩家宝可梦对战数据集,包含超过300万场对战,其中高分段对战超50万场。基于此数据集,我们建立了系列对战评测基准与谜题,用于评估特定战斗技能。我们还对本地游戏引擎进行了关键更新。希望本工作推动更多研究将宝可梦对战作为基准,融合大模型与博弈算法解决通用多智能体问题。视频、代码与数据集见https://sites.google.com/view/pokechamp-llm。
原文摘要 · Abstract (English)
We introduce PokéChamp, a minimax agent powered by Large Language Models (LLMs) for Pokémon battles. Built on a general framework for two-player competitive games, PokéChamp leverages the generalist capabilities of LLMs to enhance minimax tree search. Specifically, LLMs replace three key modules: (1) player action sampling, (2) opponent modeling, and (3) value function estimation, enabling the agent to effectively utilize gameplay history and human knowledge to reduce the search space and address partial observability. Notably, our framework requires no additional LLM training. We evaluate PokéChamp in the popular Gen 9 OU format. When powered by GPT-4o, it achieves a win rate of 76% against the best existing LLM-based bot and 84% against the strongest rule-based bot, demonstrating its superior performance. Even with an open-source 8-billion-parameter Llama 3.1 model, PokéChamp consistently outperforms the previous best LLM-based bot, Pokéllmon powered by GPT-4o, with a 64% win rate. PokéChamp attains a projected Elo of 1300-1500 on the Pokémon Showdown online ladder, placing it among the top 30%-10% of human players. In addition, this work compiles the largest real-player Pokémon battle dataset, featuring over 3 million games, including more than 500k high-Elo matches. Based on this dataset, we establish a series of battle benchmarks and puzzles to evaluate specific battling skills. We further provide key updates to the local game engine. We hope this work fosters further research that leverage Pokémon battle as benchmark to integrate LLM technologies with game-theoretic algorithms addressing general multiagent problems. Videos, code, and dataset available at https://sites.google.com/view/pokechamp-llm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。