提出一种新方法,让稀疏目标规划更可靠地找到最优路径。
Planner-Admissible Graph-PDE Value Extensions for Sparse Goal-Conditioned Planning
- 用图-偏微分方程的极值法扩展价值函数,确保贪婪策略有效。
- 在120个迷宫布局上,新方法成功率高达97%,远超传统方法的58%。
- 适合做强化学习中的高效规划,尤其适用于标签稀缺场景。
稀疏目标条件规划可视为图-偏微分方程的狄利克雷延拓问题:将仅在目标相关边界上的稀疏代价标签扩展至未标记的图顶点,使贪婪回溯能抵达目标。本文研究在操作性argmin-Q规划器下哪些值扩展是规划可接受的。核心结果为局部动作差距证书:若回溯路径上的代理值误差低于真实动作差距的一半,则贪婪回溯可达目标。绝对最小李普希茨延拓(AMLE)通过比较原理的填充距离界实现该证书,而调和延拓因反映的是边界到达概率而非最短路径贪婪顺序,可能误排序局部动作。在120个基于AntMaze布局生成的图配置中,调和延拓的平均回溯成功率为0.584,而AMLE达到0.970。高阶有限p方法也进入高成功率区间,分别为p=4时0.903,p=8时0.973,固定预算的p=16求解器达0.982(但因求解器未完全认证,不作为收敛端点排名)。机制审计显示,多数回溯决策发生在AMLE兼容但调和不兼容的局部几何结构中,且AMLE修正了绝大多数调和错误排序。
原文摘要 · Abstract (English)
Sparse goal-conditioned planning with few cost-to-go labels can be viewed as a graph-PDE Dirichlet extension problem: extend sparse labels on a goal-dependent boundary to unlabelled graph vertices so that greedy rollouts reach the goal. We study which graph value extensions are planner-admissible under the operational argmin-Q planner. Our main result is a local action-gap certificate: if the surrogate value error along the rollout stays below half the true action gap, then the greedy rollout reaches the goal. Absolutely Minimal Lipschitz Extension (AMLE), the p=infinity endpoint of the graph p-Laplacian family, instantiates this certificate through a comparison-principle fill-distance bound. Harmonic extension, by contrast, can mis-rank local actions because its values reflect boundary hitting probabilities rather than shortest-path greedy order. On 120 AntMaze layout-derived graph configurations, harmonic extension achieves 0.584 aggregate rollout success, while AMLE reaches 0.970. Finite high-p methods also enter a high-success regime, with success 0.903 for p=4, 0.973 for p=8, and 0.982 for a fixed-budget p=16 solver, though the p=16 row is not used as a converged endpoint ranking due to incomplete solver certification. Mechanism audits show that many rollout decisions occur in AMLE-compatible but harmonic-incompatible local geometry, and that AMLE corrects most harmonic inversions on the rollout-weighted decision scope.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。