arXiv:2505.22421cs.CVcs.RO2025-05被引 19

GeoDrive通过3D几何约束提升自动驾驶场景建模精度与动作控制能力。

GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control

  • 引入3D几何信息增强世界模型的空间一致性
  • 在动态编辑中实现车辆位置精准调整,提升轨迹预测准确率
  • 支持新轨迹泛化与交互式场景编辑,适合高阶自动驾驶研发

近期世界模型进展显著提升了动态环境仿真能力,使系统能够预判未来状态并评估潜在动作。在自动驾驶中,这有助于车辆预判其他道路使用者行为、进行风险感知规划、加速仿真训练并适应新场景,从而提升安全性和可靠性。现有方法在保持3D几何一致性或处理遮挡时存在缺陷,影响自动驾驶任务中的可靠安全评估。为此,我们提出GeoDrive,显式将鲁棒3D几何条件融入驾驶世界模型,以增强空间理解与动作可控性。具体而言,首先从输入帧提取3D表示,并基于用户指定的本车轨迹生成其2D渲染图。为实现动态建模,我们在训练中引入动态编辑模块,通过修改车辆位置来优化渲染效果。大量实验表明,该方法在动作准确性与3D空间感知上显著优于现有模型,实现更真实、可适应、可靠的场景建模,提升自动驾驶安全性。此外,模型可泛化至新轨迹,并支持交互式场景编辑,如物体编辑与轨迹控制。

原文摘要 · Abstract (English)

Recent advancements in world models have revolutionized dynamic environment simulation, allowing systems to foresee future states and assess potential actions. In autonomous driving, these capabilities help vehicles anticipate the behavior of other road users, perform risk-aware planning, accelerate training in simulation, and adapt to novel scenarios, thereby enhancing safety and reliability. Current approaches exhibit deficiencies in maintaining robust 3D geometric consistency or accumulating artifacts during occlusion handling, both critical for reliable safety assessment in autonomous navigation tasks. To address this, we introduce GeoDrive, which explicitly integrates robust 3D geometry conditions into driving world models to enhance spatial understanding and action controllability. Specifically, we first extract a 3D representation from the input frame and then obtain its 2D rendering based on the user-specified ego-car trajectory. To enable dynamic modeling, we propose a dynamic editing module during training to enhance the renderings by editing the positions of the vehicles. Extensive experiments demonstrate that our method significantly outperforms existing models in both action accuracy and 3D spatial awareness, leading to more realistic, adaptable, and reliable scene modeling for safer autonomous driving. Additionally, our model can generalize to novel trajectories and offers interactive scene editing capabilities, such as object editing and object trajectory control.

自动驾驶世界模型3D建模动作控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。