用可学习的重根器隐式分解任务,提升搜索效率与可扩展性。
Structure-Induced Information for Rerooting Levin Tree Search

- 通过学习重根器隐式划分问题为软子任务,避免显式生成子目标
- 三种设计在复杂环境上实现最优在线训练效率,超越传统子目标搜索
- 适合需要高效搜索与大规模问题求解的研究者与应用
基于子目标的策略树搜索在复杂单智能体确定性问题中表现优异,但通常依赖显式子目标生成,带来高昂开销并限制可扩展性。本文提出使用近期提出的√LTS算法训练一个可学习的‘重根器’,以隐式分解问题为软子任务。针对已有工作集中于给定或人工设计的重根器形式化保证,本文提出三种重根器设计:(i) 基于聚类的重根器,利用全局状态空间结构;(ii) 基于启发式的重根器,利用学习到的代价至终点估计;(iii) 二者结合的混合方法。该框架无需显式重构与推理生成的子目标,从而实现可扩展的搜索努力分配,显著降低计算开销。实验表明,所提方法能处理传统子目标策略树搜索失效的复杂环境,并在测试领域达到最先进的在线训练效率。
原文摘要 · Abstract (English)
Subgoal-based policy tree search, which uses a policy to guide search, is effective for complex single-agent deterministic problems but often relies on explicit subgoal generation that can incur substantial overhead and hinders scalability. In this paper, we overcome these limitations by using a learned ``rerooter'' through the recently-introduced $\sqrt{\text{LTS}}$ algorithm. A rerooter implicitly decomposes the problem into soft subtasks. While previous work focused on the formal guarantees for given or handcrafted rerooters, in this work we propose three rerooter designs: (i) a clustering-based rerooter that exploits global state-space structure, (ii) a heuristic-based rerooter that leverages learned cost-to-go estimates, and (iii) a hybrid that combines both signals. Our framework avoids having to explicitly reconstruct and reason over generated subgoals, thereby enabling scalable allocation of search effort with significantly lower computational overhead. Empirically, our rerooting-based methods scale to complex environments where subgoal-based policy tree search fails, and achieve state-of-the-art online training efficiency on the domains tested.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。