用子目标提升搜索效率，让智能体更快学会解决问题。
Subgoal-Guided Policy Heuristic Search with Learned Subgoals
- 从搜索过程中的成功与失败树中学习子目标和条件策略
- 在在线设置下显著提升策略学习的样本效率
- 适合需要高效探索的强化学习或规划任务
策略树搜索是一类利用策略引导搜索的算法，其求解所需扩展次数的保证依赖于策略质量。尽管表现良好，但传统方法需完整求解轨迹来训练策略，而这些轨迹来自试错搜索过程。当问题实例较难时，从随机初始化策略开始学习成本极高，大量搜索样本浪费在失败尝试上。本文提出一种新方法：从求解过程中展开的搜索树（包括失败案例）中学习子目标及条件策略。实验表明，该策略形式与训练方法在在线设置下显著提升了策略与启发函数的学习样本效率。
原文摘要 · Abstract (English)
Policy tree search is a family of tree search algorithms that use a policy to guide the search. These algorithms provide guarantees on the number of expansions required to solve a given problem that are based on the quality of the policy. While these algorithms have shown promising results, the process in which they are trained requires complete solution trajectories to train the policy. Search trajectories are obtained during a trial-and-error search process. When the training problem instances are hard, learning can be prohibitively costly, especially when starting from a randomly initialized policy. As a result, search samples are wasted in failed attempts to solve these hard instances. This paper introduces a novel method for learning subgoal-based policies for policy tree search algorithms. The subgoals and policies conditioned on subgoals are learned from the trees that the search expands while attempting to solve problems, including the search trees of failed attempts. We empirically show that our policy formulation and training method improve the sample efficiency of learning a policy and heuristic function in this online setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。