用浅层网络训练游戏智能体，击败强搜索对手
From Imitation to Interaction: Mastering Game of Schnapsen with Shallow Reinforcement Learning
- 用监督学习和强化学习对比训练浅层神经网络
- 强化学习使智能体胜率显著高于监督学习
- 结合深度搜索可进一步提升对强手的胜率
本文研究浅层神经网络智能体能否掌握纸牌游戏Schnapsen，并挑战基于搜索的强基准RdeepBot（使用蒙特卡洛采样与前瞻搜索）。通过逐步复杂的实验设计，我们首先评估了在回放数据上训练的监督学习智能体（MLPBot），随后训练了具有相同浅层结构的强化学习智能体（RLBot），采用异步蒙特卡洛更新与经验回放。结果表明，监督模仿无法有效泛化以击败强对手RdeepBot，而强化学习生成的智能体表现显著更强。在聚焦RdeepBot深度参数的设置中，将学习到的价值函数与更深的前瞻搜索结合，使RLBot在统计上显著提高对最强评估版本RdeepBot的胜率。在基于采样的设置中，性能提升更依赖条件：最佳表现出现在相对较低的训练样本数（num_samples）而非随采样增强而单调提升。
原文摘要 · Abstract (English)
This paper investigates whether shallow neural network agents can master the card game Schnapsen and challenge a strong search-based baseline, RdeepBot, which uses Monte Carlo sampling and lookahead search. Guided by a progressively more complex experimental design, we first evaluate a supervised learning agent (MLPBot) trained on replay data and then a reinforcement learning agent (RLBot) with the same shallow architecture trained through asynchronous Monte Carlo updates and experience replay. The results show that supervised imitation does not generalize well enough to defeat strong RdeepBot opponents, whereas reinforcement learning produces substantially stronger agents. In the setting that focuses on the depth parameter of RdeepBot, the best performance is achieved when the learned value function is combined with deeper lookahead during gameplay, allowing RLBot to achieve statistically significant higher winning rates against the strongest evaluated RdeepBot baseline. In the sample-based setting, the gains are more conditional: the strongest performance appears at a relatively lower training num_samples parameter rather than increasing uniformly with stronger sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。