让AI在信息不全的博弈中学会预判，提升决策能力
Look-ahead Reasoning with a Learned Model in Imperfect Information Games
- 通过交互学习博弈抽象模型，替代复杂环境建模
- 在大型不完美信息博弈中显著提升预训练代理表现
- 适合需要高效推理的复杂策略游戏研究者
测试时推理能显著提升预训练AI代理的表现。但其依赖显式环境模型，而现实场景中模型往往不可用或过于复杂。尽管MuZero可在完美信息博弈中有效学习模型用于搜索，但将其推广至不完美信息博弈面临巨大挑战，因需更精细的预判机制且单次决策涉及状态众多。本文提出算法LAMIR，直接从代理与环境的交互中学习不完美信息博弈的抽象模型。测试时，利用该训练好的模型进行预判推理。所学抽象将每个子博弈规模压缩至可处理范围，使理论上严谨的预判推理在以往方法无法扩展的游戏中也变得可行。实验证明，当模型容量充足时，LAMIR能学习到精确的底层博弈结构；容量有限时，仍能学习到有价值的抽象，显著提升预训练代理在大型博弈中的表现。
原文摘要 · Abstract (English)
Test-time reasoning significantly enhances pre-trained AI agents' performance. However, it requires an explicit environment model, often unavailable or overly complex in real-world scenarios. While MuZero enables effective model learning for search in perfect information games, extending this paradigm to imperfect information games presents substantial challenges due to more nuanced look-ahead reasoning techniques and large number of states relevant for individual decisions. This paper introduces an algorithm LAMIR that learns an abstracted model of an imperfect information game directly from the agent-environment interaction. During test time, this trained model is used to perform look-ahead reasoning. The learned abstraction limits the size of each subgame to a manageable size, making theoretically principled look-ahead reasoning tractable even in games where previous methods could not scale. We empirically demonstrate that with sufficient capacity, LAMIR learns the exact underlying game structure, and with limited capacity, it still learns a valuable abstraction, which improves game playing performance of the pre-trained agents even in large games.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。