arXiv:2609.03602cs.CVcs.RO2026-09

用六摄像头高效预测驾驶动作,兼顾安全与实时性。

SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving

论文配图:SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
图 1 · 摘自论文原文
  • 共享生成模型中用未来视频监督动作学习,避免推理时生成视频
  • 动作中心的因果掩码阻止动作关注未来画面,实现部署时仅输出动作
  • 引入可微的可行驶区域约束,提升规划安全性与边界感知能力

世界模型(WM)在端到端自动驾驶中展现出强大潜力,通过学习未来场景动态的预测表征。然而,推理时生成未来视频带来巨大计算开销,导致许多近期驾驶世界模型采用单前视摄像头以实现高效部署,限制了变道、汇入、转弯等关键操作的视野覆盖。为此,我们提出SV-WAM——一种保留完整六摄像头输入的全景视角世界-动作模型(WAM),同时保持高效推理。SV-WAM将未来视频预测作为联合动作-视频去噪中的密集训练监督,而非推理输出。核心在于动作中心的因果掩码,防止动作标记在联合去噪过程中访问未来视频标记。因此,推理阶段可直接丢弃视频分支,仅输出动作进行高效规划。此外,我们引入可微的可行驶区域合规正则化器,惩罚车辆轮廓角接近或跨越可行驶边界的行为,显著提升规划安全性与边界意识。在闭环NAVSIMv2基准和开环nuScenes基准上的大量实验表明,SV-WAM在低推理延迟下达到最先进规划性能,并具备有竞争力的零样本迁移能力。

原文摘要 · Abstract (English)

World models (WMs) have demonstrated strong potential for end-to-end autonomous driving by learning predictive representations of future scene dynamics. However, generating future videos during inference introduces substantial computational overhead, leading many recent driving WMs to adopt a single front camera as input for efficient deployment. This design restricts spatial coverage in safety-critical maneuvers such as lane changes, merges, and turns. To address this limitation, we propose SV-WAM, a surround-view world-action model (WAM) that preserves full six-camera observations while maintaining efficient inference. SV-WAM leverages future-video prediction as dense training supervision for action learning within a shared generative model, rather than as an inference-time output. At the core of this design is an action-centered causal mask that prevents action tokens from attending to future-video tokens during joint action-video denoising. Consequently, the video branch can be discarded at deployment, enabling efficient action-only planning. Furthermore, we introduce a differentiable drivable-area compliance regularizer that penalizes vehicle-footprint corners approaching or crossing drivable boundaries, improving planning safety and boundary awareness. Extensive experiments on the closed-loop NAVSIMv2 benchmark and the open-loop nuScenes benchmark demonstrate that SV-WAM achieves state-of-the-art planning performance with low inference latency and competitive zero-shot transfer capability.

自动驾驶世界模型多视角高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。