arXiv:2608.07468cs.CV2026-08被引 1

SimWAM通过训练时预测未来视频,实现高效端到端自动驾驶。

SimWAM: A Simple World Action Model for End-to-End Autonomous Driving

论文配图:SimWAM: A Simple World Action Model for End-to-End Autonomous Driving
图 1 · 摘自论文原文
  • 用未来视频预测作为训练信号,联合优化视频与动作专家
  • 推理时无需生成未来帧,延迟显著低于现有方法
  • 支持零样本迁移,适合追求低延迟的自动驾驶系统

世界-动作模型(WAM)通过传递视频动态先验提升端到端自动驾驶性能,但现有方法在测试时需进行昂贵的未来想象。本文提出SimWAM,一种简单有效的WAM,利用未来视频预测作为训练阶段的监督信号。它通过联合流匹配共同训练一个预训练视频专家和一个轻量级动作专家。隔离注意力掩码使动作预测独立于未来帧,从而在推理时无需显式生成未来帧即可完成轨迹预测。由于两个专家无参数共享且仅通过统一注意力接口交互,视频主干可独立替换,动作专家可独立扩展,无需修改学习目标或推理流程。我们进一步引入强化学习,以优化超越轨迹模仿的组合式驾驶奖励。SimWAM在NAVSIM上取得91.5 PDMS,优于现有基于WAM的规划器且延迟大幅降低,并实现零样本迁移至nuScenes。该结果表明SimWAM是一个简单而稳健的基线,可受益于视频生成技术的进步,实现更高效的自动驾驶。代码与模型权重见https://github.com/H-EmbodVis/SimWAM/。

原文摘要 · Abstract (English)

World-Action Models (WAMs) improve end-to-end autonomous driving by transferring video dynamics priors to action prediction, but existing methods incur costly test-time future imagination. We present SimWAM, a simple yet effective WAM that leverages future-video prediction as a training-time supervision signal. It co-trains a pretrained video expert and a lightweight action expert with joint flow matching. An isolated attention mask keeps action prediction independent of future frames, allowing trajectory prediction without explicit future-frame generation at inference. Since the two experts share no parameters and interact only through a unified attention interface, the video backbone could be replaced and the action expert scaled independently without modifying the learning objective or inference pipeline. We further apply reinforcement learning to optimize a compositional driving reward beyond trajectory imitation. Our SimWAM achieves 91.5 PDMS on NAVSIM, surpasses state-of-the-art WAM-based planners with substantially lower latency, and transfers zero-shot to nuScenes. These results position SimWAM as a simple yet solid baseline that could readily benefit from advances in video generation for efficient autonomous driving. The code and model weights are available at https://github.com/H-EmbodVis/SimWAM/.

端到端驾驶世界模型低延迟强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。