纯自对弈强化学习实现无需搜索的西洋双陆棋强手，速度更快、水平更高。
PureTD: Reinforcement Learning for Backgammon Money Games with No Evaluation-time Search
- 完全通过自对弈强化学习训练，不依赖专家特征或搜索算法。
- 在有翻倍棋子的金钱局中，性能超越使用单步搜索的开源引擎。
- 适合研究无搜索强化学习或对弈游戏智能体的开发者参考。
我们重新审视了Tesauro的TD-Gammon，在无需评估时搜索的设定下训练西洋双陆棋模型。棋子走法和翻倍决策均通过自对弈强化学习从零开始学习，仅需极少手工设计逻辑且不使用专家特征。在此设定下，我们证明纯自对弈强化学习足以训练出接近当前顶尖水平的模型。具体而言，在含翻倍棋子的金钱局中，该无搜索模型不仅评估速度更快，且显著强于运行单步（1-ply）前瞻搜索的开源引擎GNU Backgammon与Open Sage。
原文摘要 · Abstract (English)
We revisit Tesauro's TD-Gammon for backgammon money games in the setting of no evaluation-time search. Both checker play and cube action (use of the doubling cube) are learned from scratch via self-play reinforcement learning (RL), with minimal hand-coded logic and no expert features. In this setting, we demonstrate that pure self-play RL suffices to train models that reach near-state-of-the-art playing strength. Specifically, for cubeful money games, our search-free model evaluates faster and is substantially stronger than the open-source engines GNU Backgammon and Open Sage running a one-move (1-ply) look-ahead search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。