用能量场优化轨迹，让机器自动规划并验证动作可行性。
Planning as Descent: Goal-Conditioned Latent Trajectory Synthesis in Learned Energy Landscapes
- 构建目标相关的能量函数，通过梯度下降优化轨迹
- 在窄幅专家数据上达95%成功率，超越此前最高68%
- 适合无奖励信号的离线规划场景，尤其擅长处理噪声数据
我们提出规划即下降（PaD），一种离线目标条件强化学习框架，将轨迹生成建立在可验证性基础上。不直接学习策略或显式规划器，而是学习整个潜在轨迹上的目标条件能量函数，使符合目标且可行的未来具有低能量。规划通过该能量场中的梯度优化实现，训练与推理使用相同计算流程，减少解耦建模中常见的训练-测试偏差。PaD采用自监督事后目标重标注进行训练，塑造围绕规划动态的能量场。推理时，多个轨迹候选在不同时间假设下被优化，最终选择能量低、兼顾可行性和效率的方案。我们在OGBench立方体操作任务上评估了PaD。仅用狭窄专家示范训练时，成功率达95%，显著优于此前最高68%的方法。值得注意的是，使用噪声且次优的数据训练反而进一步提升了成功率和计划效率，凸显验证驱动规划的优势。结果表明，学习评估与优化轨迹为无奖励、离线规划提供了一种稳健替代方案。
原文摘要 · Abstract (English)
We present Planning as Descent (PaD), a framework for offline goal-conditioned reinforcement learning that grounds trajectory synthesis in verification. Instead of learning a policy or explicit planner, PaD learns a goal-conditioned energy function over entire latent trajectories, assigning low energy to feasible, goal-consistent futures. Planning is realized as gradient-based refinement in this energy landscape, using identical computation during training and inference to reduce train-test mismatch common in decoupled modeling pipelines. PaD is trained via self-supervised hindsight goal relabeling, shaping the energy landscape around the planning dynamics. At inference, multiple trajectory candidates are refined under different temporal hypotheses, and low-energy plans balancing feasibility and efficiency are selected. We evaluate PaD on OGBench cube manipulation tasks. When trained on narrow expert demonstrations, PaD achieves state-of-the-art 95\% success, strongly outperforming prior methods that peak at 68\%. Remarkably, training on noisy, suboptimal data further improves success and plan efficiency, highlighting the benefits of verification-driven planning. Our results suggest learning to evaluate and refine trajectories provides a robust alternative to direct policy learning for offline, reward-free planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。