在卡牌游戏中用非完备搜索提升智能体胜率，且更抗攻击。
Unsound Search with Policy and Value Networks in Legends of Code and Magic
- 基于对手构组数据采样世界，用策略与价值网络进行非完备搜索。
- 无搜索时胜率仅26.8%，加搜索后升至51.35%，提升24.6个百分点。
- 该方法对抗最佳响应攻击表现优于冠军模型，适合高对抗性游戏场景。
在可枚举信念状态的完美与不完美信息博弈中，决策时搜索是有效的游戏AI方法。集换式卡牌游戏属于不完美信息博弈，其信念状态规模巨大，例如《代码与魔法传奇》（LoCM）中达到 $2^{101}$。该赛事冠军模型 ByteRL 未使用任何搜索；此前研究认为基于枚举的搜索在此类游戏中不可行。我们测量了三个可预测理论不完善信息蒙特卡洛缺陷成本的属性，发现 LoCM 处于有利区域。以亚军模型 NeteaseOPD 的模仿学习策略为基础，构建了策略与价值前馈网络。智能体从基于亚军构组数据的先验分布中采样世界进行搜索。在严格配置下，战斗阶段对战 ByteRL 的胜率为 51.35%（95% 置信区间 [50.37, 52.33]），基于官方裁判和时间限制，在超过 10,000 场预注册比赛中验证。搜索显著影响胜负：移除搜索后胜率降至 26.8%，增加搜索带来 +24.6 个百分点的提升。非完备搜索虽可能存在可利用性，但我们复现了针对 ByteRL 的已发表最优响应攻击，并将其应用于本模型的两个搜索配置，结果表明其在每轮迭代中均比 ByteRL 更具抗性。在 LoCM 中，非完备搜索可带来更强且更稳健的智能体。
原文摘要 · Abstract (English)
Decision-time search in perfect and imperfect information games with enumerable belief states are effective methods for game AI. Collectible card games are imperfect information games with large belief states. Legends of Code and Magic is a collectible card game competition where the belief states are $2^{101}$. The Legends of Code and Magic (LoCM) champion, ByteRL, plays with no search. Other works claim sound enumeration-based search is unusable in the genre due to the number of belief states. We measured three previously defined properties that predict where theoretically unsound perfect information Monte Carlo's defects are cheap and found LoCM sits in the favorable region. Starting with imitation learning of the runner-up policy, NeteaseOPD, we created a policy and value feed-forward network. Our agent searches over worlds sampled from a prior over the opponent's deck built from the runner-up's drafts. Using our strictest configuration in the battle phase we beat ByteRL with a win percentage of 51.35% 95% CI [50.37, 52.33], over 10,000 pre-registered games using the LoCM official referee and time limit. Search is not a minor factor on the matchup between our agent and ByteRL. Without search this agent scores 26.8% and adding search adds +24.6 points. Unsound search in imperfect information games could be exploitable. We replicate a published best-response attack against ByteRL. We then apply the same attack protocol to two search configurations of our agent, and each one resists it better than ByteRL at every iteration. In LoCM unsound search gives us a stronger and more resilient agent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。