arXiv:2512.12751cs.CV2025-12被引 7

用4D占据栅格引导视频生成,让自动驾驶模拟更物理合理且高效。

GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generation

  • 先生成4D占据栅格,再用其指导视频合成,保证物理一致性。
  • 推理速度达41 FPS,mIoU提升7.2%,仅用347万参数。
  • 支持多视角一致、可控制的驾驶视频生成,适合仿真与评估。

物理感知的驾驶世界模型对路径规划、分布外数据合成和闭环评估至关重要。现有方法常依赖单一扩散模型直接将驾驶动作映射为视频,导致学习困难且输出物理不一致。为此,我们提出GenieDrive,一种新型物理感知驾驶视频生成框架。该方法首先生成4D占据栅格,作为后续视频生成的物理基础。4D占据包含高分辨率3D结构与动态信息。为高效压缩高分辨率占据,我们设计了一个将占据编码为潜在三平面表示的VAE,使潜空间大小仅为之前方法的58%。我们引入互控注意力(MCA)以精确建模控制对占据演化的影响,并端到端联合训练VAE与预测模块以最大化预测精度。上述设计共同实现推理速度41 FPS、mIoU提升7.2%,仅需3.47M参数。此外,在视频生成模型中引入归一化多视图注意力,结合4D占据生成多视角驾驶视频,显著提升质量,FVD降低20.7%。实验表明,GenieDrive可实现高度可控、多视角一致且物理合理的驾驶视频生成。

原文摘要 · Abstract (English)

Physics-aware driving world model is essential for drive planning, out-of-distribution data synthesis, and closed-loop evaluation. However, existing methods often rely on a single diffusion model to directly map driving actions to videos, which makes learning difficult and leads to physically inconsistent outputs. To overcome these challenges, we propose GenieDrive, a novel framework designed for physics-aware driving video generation. Our approach starts by generating 4D occupancy, which serves as a physics-informed foundation for subsequent video generation. 4D occupancy contains rich physical information, including high-resolution 3D structures and dynamics. To facilitate effective compression of such high-resolution occupancy, we propose a VAE that encodes occupancy into a latent tri-plane representation, reducing the latent size to only 58% of that used in previous methods. We further introduce Mutual Control Attention (MCA) to accurately model the influence of control on occupancy evolution, and we jointly train the VAE and the subsequent prediction module in an end-to-end manner to maximize forecasting accuracy. Together, these designs yield a 7.2% improvement in forecasting mIoU at an inference speed of 41 FPS, while using only 3.47 M parameters. Additionally, a Normalized Multi-View Attention is introduced in the video generation model to generate multi-view driving videos with guidance from our 4D occupancy, significantly improving video quality with a 20.7% reduction in FVD. Experiments demonstrate that GenieDrive enables highly controllable, multi-view consistent, and physics-aware driving video generation.

自动驾驶4D占据视频生成物理建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。