arXiv:2608.02379cs.AI2026-08

用深度强化学习模拟冰壶战术决策,实现无需人工标注的自主策略学习。

Chess on Ice: Curling Tactical Decision-Making via Backward Induction and Deep Reinforcement Learning

论文配图:Chess on Ice: Curling Tactical Decision-Making via Backward Induction and Deep Reinforcement Learning
图 1 · 摘自论文原文
  • 基于深度确定性策略梯度,结合回溯推理建模冰壶连续动作与随机结果。
  • 在四石简化版中,智能体策略媲美专家手写规则,达成与先手优势持平的表现。
  • 可量化评估战术选项,适合赛事复盘与运动员训练辅助。

冰壶常被称为‘冰上国际象棋’,因其战术决策复杂。然而与国际象棋不同,冰壶在机器学习领域仍鲜有研究,以往工作多局限于统计方法。本文提出一种强化学习框架,可定量评估和比较冰壶中的战术选择。游戏建模面临多重挑战:连续状态与动作空间、反映选手技能差异的随机动作结果,以及动作微小偏差即引发剧烈状态变化。为此,我们采用改进的深度确定性策略梯度(DDPG)算法,利用游戏的有限时域结构进行优化。实验表明,所学智能体可在完全自监督下习得有效策略:在四石简化版本中,其表现与接近最优的手动设计启发式规则相当,且与该版本固有的先手优势相匹敌。除最终策略外,学习到的评判器还对整个连续动作空间提供密集价值估计,支持战术方案的量化对比,可用于赛后分析与运动员备战决策支持。

原文摘要 · Abstract (English)

Curling is often referred to as "Chess on Ice", owing to the tactical complexity of its decision-making process. Yet unlike chess, curling remains largely underexplored from a machine learning perspective, with prior work confined mainly to statistical approaches. We propose a reinforcement learning framework capable of quantitatively evaluating and comparing tactical options in curling. The game poses several modeling challenges: continuous state and action spaces, stochastic action outcomes reflecting player skill variability, and state transitions that are highly sensitive to small perturbations in the executed action. To address them, we employ the Deep Deterministic Policy Gradient actor-critic algorithm, adapted to exploit the finite-horizon structure of the game. Our experiments show that effective curling strategies can be acquired in a fully self-supervised manner, without any human-annotated data: on a reduced four-rock variant, the learned agent matches a hand-crafted expert heuristic in a regime where that heuristic is close to optimal, a parity we quantify against the intrinsic hammer advantage of the variant. Beyond the resulting policy, the learned critic provides a dense value estimate over the entire continuous action space, enabling the quantitative comparison of tactical alternatives for applications such as post-game performance analysis and decision support during athlete preparation.

冰壶强化学习策略评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。