不依赖搜索的象棋AI，性能达人类顶尖水平且速度超千倍。
Mastering Chinese Chess AI (Xiangqi) Without Search
- 用监督学习加强化学习训练，摒弃传统搜索机制。
- 每秒响应超千次，比MCTS快上千倍，比AlphaBeta快百倍。
- 适合追求高效推理的AI系统开发者参考。
我们开发了一种高性能象棋人工智能，无需依赖搜索算法即可达到人类前0.1%玩家的水平。通过消除传统系统中的搜索过程，该AI的每秒查询量（QPS）超过基于蒙特卡洛树搜索（MCTS）的系统一千倍以上，也超过基于阿尔法-贝塔剪枝（AlphaBeta）的系统一百倍以上。训练系统由两部分组成：监督学习生成初始类人象棋模型，强化学习在此基础上显著提升整体强度。通过充分的消融实验发现：1. 相同参数量下，Transformer架构在象棋任务中表现优于卷积神经网络（CNN）；2. 将双方可行动作作为特征能显著加速训练；3. 采用选择性对手池比纯自对弈训练具有更快的提升曲线和更高的强度上限；4. 值估计截断（VECT）改进了原始PPO算法的训练过程，本文将予以解释。
原文摘要 · Abstract (English)
We have developed a high-performance Chinese Chess AI that operates without reliance on search algorithms. This AI has demonstrated the capability to compete at a level commensurate with the top 0.1\% of human players. By eliminating the search process typically associated with such systems, this AI achieves a Queries Per Second (QPS) rate that exceeds those of systems based on the Monte Carlo Tree Search (MCTS) algorithm by over a thousandfold and surpasses those based on the AlphaBeta pruning algorithm by more than a hundredfold. The AI training system consists of two parts: supervised learning and reinforcement learning. Supervised learning provides an initial human-like Chinese chess AI, while reinforcement learning, based on supervised learning, elevates the strength of the entire AI to a new level. Based on this training system, we carried out enough ablation experiments and discovered that 1. The same parameter amount of Transformer architecture has a higher performance than CNN on Chinese chess; 2. Possible moves of both sides as features can greatly improve the training process; 3. Selective opponent pool, compared to pure self-play training, results in a faster improvement curve and a higher strength limit. 4. Value Estimation with Cutoff(VECT) improves the original PPO algorithm training process and we will give the explanation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。