让模型直接预测多步动作后的状态,提升长期规划精度。
Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning

- 用可变长度动作序列训练,直接预测长期状态变化。
- 在多个数据集上比当前最佳方法平均提升13%。
- 适合需要长程规划的强化学习任务,如机器人控制。
近期,世界模型作为构建智能体的有前景范式,通过学习条件于观测和动作的未来环境状态预测模型实现。特别是基于JEPA的潜在世界模型,在紧凑表示空间中学习动作条件的动力学,相比像素空间预测更高效。然而,现有潜在世界模型通常依赖单步预测,需递归滚动以进行长期规划,导致误差累积,并与下游规划目标不匹配。为此,本文提出可变长度潜在世界模型(VLWM),学习根据不同长度的动作序列预测未来潜在状态。该模型不再仅训练单步转移,而是直接建模时间扩展的动力学,使同一预测器能评估不同时间跨度的动作计划。我们进一步引入课程训练策略,逐步扩大动作时域,从短程动力学到长程预测稳定优化。测试时,设计了适配VLWM的规划方法,更好利用其可变长度预测能力。在长期控制任务上的实验表明,VLWM显著提升潜在空间世界模型性能,在不同数据集上比最先进的LeWM平均提高13%,尤其在需要长时间规划的任务中表现突出。结果表明,VLWM为改进潜在世界模型的长期预测与规划提供了一种简单而有效的新范式。
原文摘要 · Abstract (English)
Recently, world models have emerged as a promising paradigm for building intelligent agents by learning predictive models that estimate future environment states conditioned on observations and actions. In particular, JEPA-style latent world models provide an efficient alternative to pixel space prediction by learning action-conditioned dynamics in compact representation spaces. However, existing latent world models typically rely on one-step prediction and must be recursively rolled out for long-horizon planning, which leads to compounding errors and a mismatch between training objectives and downstream planning tasks. To address this limitation, we propose Variable-length Latent World Models (VLWMs), a framework that learns to predict future latent states conditioned on action sequences of variable lengths. Instead of training only on one-step transitions, VLWMs directly model temporally extended dynamics, allowing the same predictor to evaluate action plans over different horizons. We further introduce a curriculum training strategy that progressively expands the action horizon, stabilizing optimization from short-range dynamics to long-range prediction. At test time, we design planning methods tailored to VLWMs to better exploit their variable-length predictive capabilities. Experiments on long-horizon control tasks show that VLWMs significantly improve latent space world models, achieving 13\% average improvement over the state-of-the-art LeWM across different datasets, with especially large gains on tasks requiring extended planning. These results suggest that VLWM provides a simple yet effective paradigm for improving long-horizon prediction and planning in latent world models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。