提出新搜索策略，让自博弈训练更省算力、更快出成果
Search-contempt: a hybrid MCTS algorithm for training AlphaZero-like engines with better computational efficiency
- 用混合搜索机制调整自博弈生成棋局分布，偏向困难局面
- 训练游戏量从数百万降至数十万，成本由千万级降至数万级
- 适合资源有限的开发者在消费级显卡上训练强大棋类引擎
AlphaZero于2017年通过自我对弈掌握了国际象棋等游戏，但需数百万局训练和数千万美元算力。本文提出一种名为search-contempt的新混合蒙特卡洛树搜索（MCTS）算法，从根本上改变自博弈中生成棋局的分布，更偏好复杂局面。该方法在让子棋（Odds Chess）中显著提升引擎强度，更重要的是，使基于自博弈的引擎训练可在仅数万局、数万美元预算下完成，远低于AlphaZero所需的千万级训练量与百万级成本。这意味着，如今甚至可在普通消费级显卡上，以有限算力、成本和时间，从零开始训练高性能棋类程序。
原文摘要 · Abstract (English)
AlphaZero in 2017 was able to master chess and other games without human knowledge by playing millions of games against itself (self-play), with a computation budget running in the tens of millions of dollars. It used a variant of the Monte Carlo Tree Search (MCTS) algorithm, known as PUCT. This paper introduces search-contempt, a novel hybrid variant of the MCTS algorithm that fundamentally alters the distribution of positions generated in self-play, preferring more challenging positions. In addition, search-contempt has been shown to give a big boost in strength for engines in Odds Chess (where one side receives an unfavorable position from the start). More significantly, it opens up the possibility of training a self-play based engine, in a much more computationally efficient manner with the number of training games running into hundreds of thousands, costing tens of thousands of dollars (instead of tens of millions of training games costing millions of dollars required by AlphaZero). This means that it may finally be possible to train such a program from zero on a standard consumer GPU even with a very limited compute, cost, or time budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。