基于奖励分布的自动动作抽象，让MCTS更高效地发现对称动作。
AUPO -- Abstracted Until Proven Otherwise: A Reward Distribution Based Abstraction Algorithm
- 利用MCTS中的奖励分布统计自动抽象动作，无需转移概率或有向无环图。
- 在多个IPPC基准问题上表现优于传统MCTS，尤其擅长发现远距离状态的对称动作。
- 可与其它抽象方法共存，不干扰树搜索过程，适合强化学习框架集成。
我们提出一种名为AUPO的新算法，作为蒙特卡洛树搜索（MCTS）决策策略的即插即用改进。基于一系列IPPC基准问题的对比实验表明，AUPO明显优于标准MCTS。AUPO是一种仅依赖MCTS过程中获取的奖励分布统计信息的自动动作抽象算法。与现有自动抽象方法不同，它无需访问转移概率，也无需构建有向无环搜索图，因此能有效识别出状态空间中相距较远的对称动作，而当前主流框架如ASAP在此类情况下表现不佳。此外，由于AUPO仅影响决策策略，它与其他仅作用于树搜索的抽象技术具有兼容性，可协同使用。
原文摘要 · Abstract (English)
We introduce a novel, drop-in modification to Monte Carlo Tree Search's (MCTS) decision policy that we call AUPO. Comparisons based on a range of IPPC benchmark problems show that AUPO clearly outperforms MCTS. AUPO is an automatic action abstraction algorithm that solely relies on reward distribution statistics acquired during the MCTS. Thus, unlike other automatic abstraction algorithms, AUPO requires neither access to transition probabilities nor does AUPO require a directed acyclic search graph to build its abstraction, allowing AUPO to detect symmetric actions that state-of-the-art frameworks like ASAP struggle with when the resulting symmetric states are far apart in state space. Furthermore, as AUPO only affects the decision policy, it is not mutually exclusive with other abstraction techniques that only affect the tree search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。