arXiv:2507.00603cs.CV2025-07ICCV被引 113

用视觉大模型构建隐式世界模型,实现无需标注的端到端自动驾驶。

World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model

  • 基于视觉大模型生成带语义的空间隐状态,融合驾驶意图进行轨迹预测。
  • 在nuScenes和NavSim上实现18.1%的L2误差降低,碰撞率下降46.7%。
  • 适合追求无监督感知与高效训练的自动驾驶研究者。

端到端自动驾驶直接从原始传感器数据生成规划轨迹,但通常依赖昂贵的感知标注提取场景信息。核心挑战在于:如何构建一个信息丰富的驾驶世界模型,实现无需感知标注的端到端规划。本文提出World4Drive框架,利用视觉基础模型构建隐式世界模型,用于生成和评估多模态规划轨迹。首先,从输入中提取包含驾驶意图和富含空间-语义先验的隐式世界表示;随后,基于当前场景特征与驾驶意图生成多模态轨迹,并在隐空间内预测多个意图驱动的未来状态;最后通过世界模型选择模块评估并选出最优轨迹。通过实际未来观测与隐空间重构预测之间的自监督对齐,实现无感知标注的端到端规划。在开放环nuScenes和闭合环NavSim基准上均达到当前最佳性能,相比基线实现18.1%的相对L2误差降低、46.7%的碰撞率下降,以及3.75倍更快的训练收敛速度。

原文摘要 · Abstract (English)

End-to-end autonomous driving directly generates planning trajectories from raw sensor data, yet it typically relies on costly perception supervision to extract scene information. A critical research challenge arises: constructing an informative driving world model to enable perception annotation-free, end-to-end planning via self-supervised learning. In this paper, we present World4Drive, an end-to-end autonomous driving framework that employs vision foundation models to build latent world models for generating and evaluating multi-modal planning trajectories. Specifically, World4Drive first extracts scene features, including driving intention and world latent representations enriched with spatial-semantic priors provided by vision foundation models. It then generates multi-modal planning trajectories based on current scene features and driving intentions and predicts multiple intention-driven future states within the latent space. Finally, it introduces a world model selector module to evaluate and select the best trajectory. We achieve perception annotation-free, end-to-end planning through self-supervised alignment between actual future observations and predicted observations reconstructed from the latent space. World4Drive achieves state-of-the-art performance without manual perception annotations on both the open-loop nuScenes and closed-loop NavSim benchmarks, demonstrating an 18.1\% relative reduction in L2 error, 46.7% lower collision rate, and 3.75 faster training convergence. Codes will be accessed at https://github.com/ucaszyp/World4Drive.

自动驾驶隐空间建模自监督学习视觉大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。