在四人纸牌游戏Big 2中，强化学习模型通过自对弈实现高效训练。
Self-Play Reinforcement Learning under Imperfect Information in Big 2
- 采用自对弈框架，统一环境与训练条件对比不同强化学习算法
- PPO算法在随机、贪心和启发式对手上均优于Q类方法
- 适度熵正则化提升性能，当前策略自对弈优于固定对手训练
不完全信息多人博弈考验智能体在隐藏信息、稀疏奖励和非平稳对手下的决策能力。本文研究四人不完全信息纸牌游戏Big 2中的挑战。提出一种适用于Big 2的自对弈强化学习框架，实现策略梯度与价值近似算法的可控对比。在统一环境、输入表示、训练预算与评估协议下，PPO在面对随机、贪心及启发式对手时表现优于蒙特卡洛Q近似、SARSA与Q-learning。进一步发现，适度熵正则化可防止策略过度确定化，从而提升PPO性能；而基于当前策略的自对弈相比检查点自对弈或固定对手训练，能提供更优的有限预算课程。这些结果表明，Big 2是研究深度强化学习在不完全信息、多人交互、延迟奖励与可变动作集场景下的理想受控实验平台。
原文摘要 · Abstract (English)
Imperfect-information multiplayer games test whether agents can act under hidden information, sparse rewards, and non-stationary opponents. We study these challenges in Big 2, a four-player imperfect-information card game. We develop a self-play RL framework for Big 2 that enables controlled comparisons between policy-gradient and value-approximating agents. Under a common environment, input representation, training budget, and evaluation protocol, PPO outperforms Monte Carlo Q approximation, SARSA, and Q-learning against random, greedy, and heuristic Big 2 opponents. We further find that moderate entropy regularization improves PPO by preventing the policy from becoming overly deterministic, and that current-policy self-play provides a stronger finite-budget curriculum than checkpoint self-play or fixed-opponent training. Together, these results show that Big 2 is a useful controlled setting for studying deep RL under imperfect information, multiplayer interaction, delayed rewards, and variable action sets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。