提出MAPLE方法，让AlphaZero在信息不全游戏中更强且更高效。
MAPLE: Multi-State Aggregated Policy Evaluation for AlphaZero in Imperfect-Information Games
- 在搜索树中聚合多个世界状态的策略与价值评估
- 在幻影围棋和暗黑西洋棋上分别提升291和136 Elo
- 适合研究信息不完全博弈与强化学习的学者
不完全信息博弈中，玩家无法观测完整游戏状态，决策难度高。尽管AlphaZero在完美信息博弈中表现卓越，但扩展到不完全信息博弈仍具挑战。现有基于搜索的方法如PIMC存在策略融合问题，而结合神经网络的IS-MCTS计算开销大。本文提出多状态聚合策略评估（MAPLE），在单个搜索树中聚合多个采样世界状态的策略与价值评估，兼顾PIMC与IS-MCTS优势，并控制计算成本。进一步引入双胞胎采样策略，从信息集中选择有信息量的世界状态。在幻影围棋和暗黑西洋棋上的实验表明，MAPLE显著优于基于PIMC的AlphaZero基线，分别获得291和136 Elo提升。结果证明MAPLE是实现不完全信息博弈中AlphaZero式学习的有效方法。
原文摘要 · Abstract (English)
Imperfect-information games (IIGs) are challenging, as players must make decisions without fully observing the true game state. While AlphaZero has achieved remarkable success in perfect-information games, extending it to IIGs remains difficult. Existing search-based approaches, such as Perfect Information Monte Carlo (PIMC), suffer from strategy fusion, while Information Set Monte Carlo Tree Search (IS-MCTS) incurs high computational cost when combined with neural networks. In this paper, we propose Multi-State Aggregated PoLicy Evaluation (MAPLE), a tree search method that aggregates policy and value evaluations from multiple sampled world states within a single search tree, combining the advantages of PIMC and IS-MCTS while maintaining a controllable computational cost. We further incorporate a Siamese-based sampling strategy to select informative world states from the information set. Experiments on Phantom Go and Dark Hex show that MAPLE significantly outperforms the PIMC-based AlphaZero baseline, achieving Elo improvements of 291 and 136, respectively. These results demonstrate that MAPLE is an effective approach for AlphaZero-style learning in imperfect-information games.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。