用图神经网络重构国际象棋状态表示,提升模型泛化与训练效率。
Enhancing Chess Reinforcement Learning with Graph Representation
- 用图结构替代网格表示棋局,适配不同规模棋盘
- 小规模模型实现比之前快一个数量级的棋力提升
- 在5×5棋盘训练后可快速适配标准8×8棋盘
掌握游戏是复杂任务,因游戏结构差异大且规则各异。尽管AlphaZero能在围棋、国际象棋等多类游戏中表现出色,但其依赖大量算力及固定卷积神经网络架构,限制了可扩展性与适应性。例如,19×19围棋模型无法直接用于13×13变体。本文聚焦国际象棋,提出使用图结构表示游戏状态,取代传统网格方式,构建基于图神经网络(GNN)的通用架构,并扩展经典图注意力网络(GAT)以支持边特征,自然输出通用策略。实验表明,该架构在参数量相近的小模型上优于先前方法,棋力提升速度加快一个数量级。此外,模型在5×5象棋变体上训练后,可快速微调至标准8×8象棋,展现出良好泛化能力。代码已公开于https://github.com/akulen/AlphaGateau。
原文摘要 · Abstract (English)
Mastering games is a hard task, as games can be extremely complex, and still fundamentally different in structure from one another. While the AlphaZero algorithm has demonstrated an impressive ability to learn the rules and strategy of a large variety of games, ranging from Go and Chess, to Atari games, its reliance on extensive computational resources and rigid Convolutional Neural Network (CNN) architecture limits its adaptability and scalability. A model trained to play on a $19\times 19$ Go board cannot be used to play on a smaller $13\times 13$ board, despite the similarity between the two Go variants. In this paper, we focus on Chess, and explore using a more generic Graph-based Representation of a game state, rather than a grid-based one, to introduce a more general architecture based on Graph Neural Networks (GNN). We also expand the classical Graph Attention Network (GAT) layer to incorporate edge-features, to naturally provide a generic policy output format. Our experiments, performed on smaller networks than the initial AlphaZero paper, show that this new architecture outperforms previous architectures with a similar number of parameters, being able to increase playing strength an order of magnitude faster. We also show that the model, when trained on a smaller $5\times 5$ variant of chess, is able to be quickly fine-tuned to play on regular $8\times 8$ chess, suggesting that this approach yields promising generalization abilities. Our code is available at https://github.com/akulen/AlphaGateau.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。