让机器人通过中间目标精准规划动作路径，提升离线学习时的长程控制能力。
PathBridger: Subgoal Bridges for Offline Goal-Conditioned Reinforcement Learning
- 用逆动力学模型将中间目标显式转为可执行动作块
- 在多物体积木任务上性能提升显著，优于现有方法
- 适合需要长程规划的离线强化学习场景
离线目标条件强化学习（GCRL）旨在仅从固定轨迹数据中学习达到多种目标的策略。长时程离线GCRL仍具挑战性，因稀疏的目标达成信号需跨多步传播，且无法通过额外环境交互纠正执行错误。现有方法通过改进长程价值估计或利用子目标、选项和动作分块减少有效决策时长来应对。然而，在多种分层方法中，选定的子目标仅指明目的地，而中间状态空间路径仍隐含于端点条件的低层策略中。为此，我们提出PathBridger，一种显式连接子目标选择与短时程执行的分层离线GCRL方法。PathBridger构建通往所选中间终点的状态空间桥梁，并使用逆动力学模型将其解码为短时可执行的动作块。在OGBench任务上的实验表明，该方法整体表现强劲，尤其在多物体积木操作任务上取得显著提升。
原文摘要 · Abstract (English)
Offline goal-conditioned reinforcement learning (GCRL) aims to learn policies for reaching diverse goals entirely from fixed trajectory data. Long-horizon offline GCRL remains challenging because sparse goal-reaching signals must be propagated over many steps, while execution errors cannot be corrected through additional environment interaction. Existing methods address these challenges by improving long-range value estimation or reducing the effective decision horizon through subgoals, options, and action chunks. In several hierarchical methods, however, a selected subgoal specifies where to go, while the intervening state-space path remains implicit in an endpoint-conditioned low-level policy. To address this interface, we propose PathBridger, a hierarchical offline GCRL method that explicitly connects subgoal selection to short-horizon execution. PathBridger constructs a state-space bridge toward the selected intermediate endpoint and decodes it into a short executable action chunk using an inverse dynamics model. Experiments across the evaluated OGBench tasks demonstrate strong aggregate performance, with particularly large gains on the multi-object Cube manipulation tasks. Code: https://github.com/SChoish/PathBridger
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。