用分层强化学习让AI自动解复杂推箱子谜题，无需人工设计规则。
Solving Sokoban using Hierarchical Reinforcement Learning with Landmarks
- 构建六层分层策略，上层生成子目标指导下层行动。
- 从零开始端到端训练，单次调用可生成长序列动作。
- 首次证明深度递归目标分解能自发涌现并有效解决难题。
我们提出一种新颖的分层强化学习（HRL）框架，通过学习到的子目标实现自顶向下的递归规划，成功应用于复杂的组合谜题游戏推箱子（Sokoban）。该方法构建了六层策略层级，每一层高层策略为下层生成子目标。所有子目标与策略均从零开始端到端学习，不依赖任何领域知识。实验表明，智能体能仅通过一次高层调用生成长序列动作。尽管先前工作探索过2-3层的分层结构和基于子目标的启发式规划，但本文首次证明，深度递归目标分解可完全由学习过程自发产生，且此类层次结构能在困难谜题领域中有效扩展。
原文摘要 · Abstract (English)
We introduce a novel hierarchical reinforcement learning (HRL) framework that performs top-down recursive planning via learned subgoals, successfully applied to the complex combinatorial puzzle game Sokoban. Our approach constructs a six-level policy hierarchy, where each higher-level policy generates subgoals for the level below. All subgoals and policies are learned end-to-end from scratch, without any domain knowledge. Our results show that the agent can generate long action sequences from a single high-level call. While prior work has explored 2-3 level hierarchies and subgoal-based planning heuristics, we demonstrate that deep recursive goal decomposition can emerge purely from learning, and that such hierarchies can scale effectively to hard puzzle domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。