不靠测试时预测未来,也能实现高效智能体控制。
Fast-WAM: Do World Action Models Need Test-time Future Imagination?

- 训练时保留视频建模,测试时跳过未来预测
- 实测延迟仅190ms,速度超现有方法4倍以上
- 适合追求实时响应的机器人应用
世界动作模型(WAMs)作为具身控制的新兴方案,通过显式建模视觉观测在动作下的演化过程,展现出优于视觉-语言-动作模型的潜力。现有WAM大多采用‘想象后执行’范式,依赖迭代视频去噪,导致测试时延迟高,但其未来想象是否真正必要仍不明确。本文提出Fast-WAM,保留训练阶段的视频联合训练,但在测试阶段跳过未来预测,从而分离训练视频建模与推理时未来生成的作用。通过多个变体对比发现,移除训练中的视频共训练会导致性能大幅下降,而跳过测试时预测仍能保持竞争力。Fast-WAM在模拟基准(LIBERO、RoboTwin)和真实任务中表现媲美最先进方法,无需具身预训练,且运行延迟仅为190ms,比现有方法快逾4倍。结果表明,视频预测在WAM中的核心价值更在于提升世界表征能力,而非测试时生成未来图像。
原文摘要 · Abstract (English)
World Action Models (WAMs) have emerged as a promising alternative to Vision-Language-Action (VLA) models for embodied control because they explicitly model how visual observations may evolve under action. Most existing WAMs follow an imagine-then-execute paradigm, incurring substantial test-time latency from iterative video denoising, yet it remains unclear whether explicit future imagination is actually necessary for strong action performance. In this paper, we ask whether WAMs need explicit future imagination at test time, or whether their benefit comes primarily from video modeling during training. We disentangle the role of video modeling during training from explicit future generation during inference by proposing \textbf{Fast-WAM}, a WAM architecture that retains video co-training during training but skips future prediction at test time. We further instantiate several Fast-WAM variants to enable a controlled comparison of these two factors. Across these variants, we find that Fast-WAM remains competitive with imagine-then-execute variants, while removing video co-training causes a much larger performance drop. Empirically, Fast-WAM achieves competitive results with state-of-the-art methods both on simulation benchmarks (LIBERO and RoboTwin) and real-world tasks, without embodied pretraining. It runs in real time with 190ms latency, over 4$\times$ faster than existing imagine-then-execute WAMs. These results suggest that the main value of video prediction in WAMs may lie in improving world representations during training rather than generating future observations at test time. Project page: https://yuantianyuan01.github.io/FastWAM/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。