arXiv:2410.05347cs.LGcs.AI2024-10IJCAI被引 1

用混合残差与Transformer结构,提升棋类模型全局感知能力

Bridging Local and Global Knowledge via Transformer in Board Games

  • 将残差块与Transformer块交替设计,融合局部与全局信息
  • 9×9/19×19围棋和19×19哈克棋胜率均超AlphaZero,最高达60.9%
  • 显著提升对循环型、长串定式等复杂模式的识别准确率

尽管AlphaZero在棋类游戏中已达到超人水平,但其在需要整体棋盘理解的场景中仍存局限,例如围棋中的长序列模式识别。为此,我们提出ResTNet,一种将残差块与Transformer块交错设计的网络架构,以弥合局部与全局知识的差距。ResTNet在多个棋类游戏中提升表现:9×9围棋胜率从54.6%升至60.8%,19×19围棋从53.6%升至60.9%,19×19哈克棋从50.4%升至58.0%。该模型有效处理全局信息,成功解决19×19围棋中的两种长序列模式——环形模式与长串定式。环形模式识别的均方误差由2.58降至1.07,对抗程序的攻击成功率从70.44%降至23.91%;长串定式识别准确率由59.15%提升至80.01%。通过注意力图可视化,ResTNet展现出对围棋与哈克棋关键概念的捕捉能力,揭示了AlphaZero决策机制。整体上,ResTNet为融合局部与全局知识提供了可行路径,推动基于AlphaZero算法在棋类游戏中的进一步发展。代码已公开于https://rlg.iis.sinica.edu.tw/papers/restnet。

原文摘要 · Abstract (English)

Although AlphaZero has achieved superhuman performance in board games, recent studies reveal its limitations in handling scenarios requiring a comprehensive understanding of the entire board, such as recognizing long-sequence patterns in Go. To address this challenge, we propose ResTNet, a network that interleaves residual and Transformer blocks to bridge local and global knowledge. ResTNet improves playing strength across multiple board games, increasing win rate from 54.6% to 60.8% in 9x9 Go, 53.6% to 60.9% in 19x19 Go, and 50.4% to 58.0% in 19x19 Hex. In addition, ResTNet effectively processes global information and tackles two long-sequence patterns in 19x19 Go, including circular pattern and ladder pattern. It reduces the mean square error for circular pattern recognition from 2.58 to 1.07 and lowers the attack probability against an adversary program from 70.44% to 23.91%. ResTNet also improves ladder pattern recognition accuracy from 59.15% to 80.01%. By visualizing attention maps, we demonstrate that ResTNet captures critical game concepts in both Go and Hex, offering insights into AlphaZero's decision-making process. Overall, ResTNet shows a promising approach to integrating local and global knowledge, paving the way for more effective AlphaZero-based algorithms in board games. Our code is available at https://rlg.iis.sinica.edu.tw/papers/restnet.

棋类博弈Transformer强化学习注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。