对比DQN与CFR在德州扑克中是否会虚张声势，发现两者都骗人但方式不同。
Analysis of Bluffing by DQN and CFR in Leduc Hold'em Poker
- 用实验记录算法对弈时的动作，分析其虚张声势行为。
- 两种算法虚张声势成功率相近，约30%对手会弃牌。
- 适合研究博弈策略或强化学习行为的读者参考。
在扑克游戏中，不可预测性（即虚张声势）是一项关键技能。人类玩家会虚张声势，但多数计算机扑克研究只关注胜率等性能指标，忽视了虚张声势行为。本文研究两种主流算法——基于强化学习的DQN和基于博弈论的CFR——在简化版扑克游戏Leduc Hold'em中是否表现出虚张声势行为。我们设计实验，让DQN与CFR对战并记录其行动。结果表明，两者均表现出虚张声势行为，但方式不同。尽管虚张声势频率各异，但成功虚张声势的比例（即对手弃牌率）大致相同，约为30%。这说明虚张声势是游戏本质特征，而非算法特有。未来工作应探索不同虚张声势风格及完整扑克游戏。代码见：https://github.com/TarikZ03/Bluffing-by-DQN-and-CFR-in-Leduc-Hold-em-Poker-Codebase。
原文摘要 · Abstract (English)
In the game of poker, being unpredictable, or bluffing, is an essential skill. When humans play poker, they bluff. However, most works on computer-poker focus on performance metrics such as win rates, while bluffing is overlooked. In this paper we study whether two popular algorithms, DQN (based on reinforcement learning) and CFR (based on game theory), exhibit bluffing behavior in Leduc Hold'em, a simplified version of poker. We designed an experiment where we let the DQN and CFR agent play against each other while we log their actions. We find that both DQN and CFR exhibit bluffing behavior, but they do so in different ways. Although both attempt to perform bluffs at different rates, the percentage of successful bluffs (where the opponent folds) is roughly the same. This suggests that bluffing is an essential aspect of the game, not of the algorithm. Future work should look at different bluffing styles and at the full game of poker. Code at https://github.com/TarikZ03/Bluffing-by-DQN-and-CFR-in-Leduc-Hold-em-Poker-Codebase.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。