统一视觉语言模型驱动的自动驾驶世界模型,实现感知、规划与图像生成协同优化。
UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving
- 用统一架构联合完成场景理解、轨迹规划与未来图像生成。
- 在Bench2Drive上使轨迹误差降低7.3%,碰撞率下降10.4%。
- 适合关注端到端自动驾驶与生成式世界建模的研究者。
世界模型在自动驾驶中日益关键,准确的场景理解与未来预测对安全控制至关重要。现有方法通常将感知、预测与规划分模块处理,而本文提出UniDrive-WM,一种基于视觉语言模型(VLM)的统一世界模型,能在单一架构中联合执行驾驶场景理解、轨迹规划与条件化未来图像生成。其轨迹规划器生成未来轨迹,用于引导VLM图像生成器合成合理未来帧,这些生成结果作为额外监督信号,反向提升场景理解并迭代优化轨迹生成。我们进一步对比了离散与连续输出表示对未来图像预测的影响,分析其对下游驾驶性能的作用。在挑战性基准Bench2Drive上的实验表明,UniDrive-WM能生成高保真未来图像,并使轨迹L2误差降低7.3%,碰撞率下降10.4%,优于此前最优方法。结果证明,紧密集成VLM驱动的推理、规划与生成式建模在自动驾驶中的优势。项目页面见https://unidrive-wm.github.io/UniDrive-WM。
原文摘要 · Abstract (English)
World models have become central to autonomous driving, where accurate scene understanding and future prediction are crucial for safe control. Recent work has explored using vision-language models (VLMs) for planning, yet existing approaches typically treat perception, prediction, and planning as separate modules. We propose UniDrive-WM, a unified VLM-based world model that jointly performs driving-scene understanding, trajectory planning, and trajectory-conditioned future image generation within a single architecture. UniDrive-WM's trajectory planner predicts a future trajectory, which conditions a VLM-based image generator to produce plausible future frames. These predictions provide additional supervisory signals that enhance scene understanding and iteratively refine trajectory generation. We further compare discrete and continuous output representations for future image prediction, analyzing their influence on downstream driving performance. Experiments on the challenging Bench2Drive benchmark show that UniDrive-WM produces high-fidelity future images and improves planning performance by 7.3% in L2 trajectory error and 10.4% in collision rate over the previous best method. These results demonstrate the advantages of tightly integrating VLM-driven reasoning, planning, and generative world modeling for autonomous driving. The project page is available at https://unidrive-wm.github.io/UniDrive-WM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。