用改进的表示方法让Transformer以更少计算量下出更强国际象棋。
Mastering Chess with a Transformer Model
- 设计新位置编码，让Transformer更好理解棋盘局势。
- 8倍少计算量超越AlphaZero，30倍少算力媲美顶尖棋手模型。
- 能发现传统引擎难识别的复杂局面，如困子、要塞等高层次策略。
Transformer模型在大规模训练下展现出强大能力，尤其在需要复杂推理和理性决策的认知任务中表现突出。本文探索将Transformer应用于国际象棋，重点研究注意力机制中的位置表示作用。我们发现，只要具备足够表达力的位置表示，Transformer就能以极低的计算成本达到现有棋类模型水平。所提出的Chessformer架构，在计算量仅为8倍的情况下超越AlphaZero的对弈实力与解题能力，并在仅需30分之一计算量时达到此前顶级棋手级变压器模型的表现。此外，这些模型展现出与顶尖传统引擎不同的棋感，能够识别出如被围困棋子、坚固防御等高层位置特征，而这些是传统引擎难以捕捉的。本工作表明，领域特化增强可大幅减少对模型规模的依赖，同时证明深度学习仍可在以搜索为主导的领域取得进展。
原文摘要 · Abstract (English)
Transformer models have demonstrated impressive capabilities when trained at scale, excelling at difficult cognitive tasks requiring complex reasoning and rational decision-making. In this paper, we explore the application of transformers to chess, focusing on the critical role of the position representation within the attention mechanism. We show that transformers endowed with a sufficiently expressive position representation can match existing chess-playing models at a fraction of the computational cost. Our architecture, which we call the Chessformer, significantly outperforms AlphaZero in both playing strength and puzzle solving ability with 8x less computation and matches prior grandmaster-level transformer-based agents in those metrics with 30x less computation. Our models also display an understanding of chess dissimilar and orthogonal to that of top traditional engines, detecting high-level positional features like trapped pieces and fortresses that those engines struggle with. This work demonstrates that domain-specific enhancements can in large part replace the need for model scale, while also highlighting that deep learning can make strides even in areas dominated by search-based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。