提升潜空间模型的可规划性,让预测更符合实际可达性。
Predictive but Not Plannable: RC-aux for Latent World Models

- 引入可达性修正辅助目标,增强潜空间时间与空间一致性。
- 在多个任务上实现更好规划性能,额外开销小。
- 适合需要长程规划的强化学习与机器人控制场景。
潜空间世界模型虽能准确进行短期预测,但其潜空间与规划需求不匹配。核心问题是时空错位:模型通常以局部预测监督训练,却用于长程目标导向搜索,而欧氏距离未必反映有限动作预算下的可达性。本文提出轻量级可达性修正辅助目标(RC-aux),在无需重构的前提下,沿时间轴和空间轴双重优化。时间轴上,多时域开环预测提升长期一致性;空间轴上,基于预算的可达性监督与时间硬负样本,使潜空间能区分可达与不可达状态。测试时,可达性信号可被规划器利用,优先选择既目标导向又可实现的轨迹。我们在LeWorldModel上验证,无论续训还是从头训练,均在像素控制任务与LIBERO-Goal扩展中显著提升规划效果,且开销低。结果表明,潜空间规划不仅依赖预测精度,更需编码时间与几何结构。代码开源:https://github.com/Guang000/RC-aux。
原文摘要 · Abstract (English)
A latent world model may achieve accurate short-horizon prediction while still inducing a latent space that is poorly aligned with planning. A key issue is spatiotemporal mismatch: these models are often trained with local predictive supervision, but deployed for long-horizon goal-directed search in latent spaces where Euclidean distance may not reflect what is reachable within a finite action budget. We present the Reachability-Correction auxiliary objective (RC-aux), a lightweight correction for this mismatch in reconstruction-free latent world models. RC-aux keeps the world-model backbone unchanged and adds planning-aligned supervision along two axes. Along the time axis, multi-horizon open-loop prediction trains the model beyond one-step consistency. Along the space axis, budget-conditioned reachability supervision, together with temporal hard negatives, encourages the latent space to distinguish states that are eventually reachable from those reachable within the current planning horizon. At test time, the learned reachability signal can also be used by a reachability-aware planner to favor trajectories that are both goal-directed and attainable under the available budget. We instantiate RC-aux on LeWorldModel and evaluate it under both continuation-training and matched-from-scratch settings. Across goal-conditioned pixel-control tasks and a LIBERO-Goal extension, RC-aux improves LeWM-style planning with modest additional cost. These results suggest that planning with latent world models depends not only on predictive accuracy, but also on whether the learned representation encodes the temporal and geometric structure required by downstream search. The code is available at https://github.com/Guang000/RC-aux.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。