用大模型训练扑克机器人，三玩家对局表现接近专业水平。
SpinGPT: A Large-Language-Model Approach to Playing Poker Correctly
- 分两阶段训练：先学专家决策，再通过强化学习优化策略。
- 在30000手对局中，胜率13.4 BB/100，与顶尖程序差距小。
- 首次为三玩家快速扑克设计大模型，适合多玩家博弈研究者。
CFR算法及其变体使扑克机器人在单挑现金局中战胜顶级人类选手，并在六人局中具备竞争力。然而，随着玩家数量增加，CFR的计算复杂度呈指数上升；且在三人及以上游戏中，遵循纳什均衡不再能保证不亏损。这些限制使其难以应用于最受欢迎的锦标赛形式。受大语言模型在国际象棋和外交游戏中的成功启发，我们提出SpinGPT，首个专为流行三玩家在线扑克格式Spin & Go设计的大语言模型。SpinGPT采用两阶段训练：(1) 在32万条高筹码专家决策上进行监督微调；(2) 在27万条求解器生成的手牌上进行强化学习。结果显示，其动作与求解器一致率达78%（容忍准确率）。结合简单深堆策略，在单挑对局中对战Slumbot 30000手，胜率为13.4 ± 12.9 BB/100（95%置信区间）。结果表明，大语言模型或可成为应对多玩家不完美信息博弈的新路径。
原文摘要 · Abstract (English)
The Counterfactual Regret Minimization (CFR) algorithm and its variants have enabled the development of pokerbots capable of beating the best human players in heads-up (1v1) cash games and competing with them in six-player formats. However, CFR's computational complexity rises exponentially with the number of players. Furthermore, in games with three or more players, following Nash equilibrium no longer guarantees a non-losing outcome. These limitations, along with others, significantly restrict the applicability of CFR to the most popular formats: tournaments. Motivated by the recent success of Large Language Models (LLM) in chess and Diplomacy, we present SpinGPT, the first LLM tailored to Spin & Go, a popular three-player online poker format. SpinGPT is trained in two stages: (1) Supervised Fine-Tuning on 320k high-stakes expert decisions; (2) Reinforcement Learning on 270k solver-generated hands. Our results show that SpinGPT matches the solver's actions in 78% of decisions (tolerant accuracy). With a simple deep-stack heuristic, it achieves 13.4 +/- 12.9 BB/100 versus Slumbot in heads-up over 30,000 hands (95% CI). These results suggest that LLMs could be a new way to deal with multi-player imperfect-information games like poker.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。