MCTS本质是用搜索语言表达的MC控制，核心机制一致。
Is Monte Carlo Tree Search Just Every-Visit Monte Carlo Control?
- 将树策略与回溯策略视为同一演化策略的已学与未学部分
- 扩展操作对应首次访问初始化，回溯即标准的每访问更新
- 适合想理解两者本质关系的强化学习研究者
蒙特卡洛树搜索（MCTS）和每访问蒙特卡洛（MC）控制通常被视作不同方法。前者以搜索术语描述（选择、扩展、模拟、回溯），后者以强化学习术语描述（轨迹采样、回报估计、动作值更新、策略改进）。本文指出，在轨迹生成与动作值更新层面，二者差异主要为术语不同。树策略与回溯策略可视为单一演化策略中已学习与未学习的部分；扩展对应首次访问初始化；回溯即常规的每访问蒙特卡洛更新。在此视角下，MCTS的四个阶段可简化为两个基本操作：在当前策略下采样轨迹，以及每访问蒙特卡洛更新。因此，MCTS本质上是用搜索语言表达的每访问蒙特卡洛控制。本文旨在阐明此等价性，使其更易识别。
原文摘要 · Abstract (English)
Monte Carlo Tree Search (MCTS) and every-visit Monte Carlo (MC) control are usually presented as different methods. MCTS is described in the language of search (selection, expansion, simulation, and backup), whereas MC control is described in the language of reinforcement learning (trajectory sampling, return estimation, action-value updating, and policy improvement). This note argues that, at the level of trajectory generation and action-value updating, the distinction is largely terminological. The tree policy and rollout policy can be viewed as the learned and not-yet-learned parts of a single evolving policy; expansion corresponds to first visit and initialization; and backup is the ordinary every-visit Monte Carlo update. Under this interpretation, the four stages of MCTS reduce to two basic operations: trajectory sampling under the current policy and every-visit Monte Carlo updating. In this sense, MCTS is simply every-visit Monte Carlo control expressed in the language and data structure of search. The purpose of this note is expository: to make this equivalence explicit and easier to recognize.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。