让自动驾驶视频模型在极端轨迹下仍保持物理一致性。
Toward Physically Consistent Driving Video World Models under Challenging Trajectories
- 用物理条件生成器修正不合理驾驶轨迹,使其可执行。
- 在CARLA模拟器生成的挑战性场景上训练,提升极端情况下的视频真实性。
- 适合需要高可靠仿真系统的自动驾驶研发团队使用。
视频生成模型在自动驾驶仿真中展现出作为世界模型的巨大潜力。然而,现有方法主要基于真实驾驶数据训练,多包含自然安全场景,导致在处理模拟器或规划系统生成的挑战性、非事实性轨迹时,常产生严重物理不一致和伪影。为此,我们提出PhyGenesis,一种能生成高视觉保真度且强物理一致性的驾驶视频世界模型。其框架包含两个核心组件:(1) 物理条件生成器,将潜在无效的轨迹输入转化为物理可行的条件;(2) 物理增强型视频生成器,在这些条件下生成多视角高质量驾驶视频。为有效训练,我们构建了一个大规模、富含物理信息的异构数据集,除真实驾驶视频外,还利用CARLA模拟器生成多样化的挑战性驾驶场景,并从中提取监督信号,引导模型学习极端条件下的物理驱动动态。该挑战性轨迹学习策略实现轨迹修正并促进物理一致性视频生成。大量实验表明,PhyGenesis在挑战性轨迹上持续优于当前最先进方法。
原文摘要 · Abstract (English)
Video generation models have shown strong potential as world models for autonomous driving simulation. However, existing approaches are primarily trained on real-world driving datasets, which mostly contain natural and safe driving scenarios. As a result, current models often fail when conditioned on challenging or counterfactual trajectories-such as imperfect trajectories generated by simulators or planning systems-producing videos with severe physical inconsistencies and artifacts. To address this limitation, we propose PhyGenesis, a world model designed to generate driving videos with high visual fidelity and strong physical consistency. Our framework consists of two key components: (1) a physical condition generator that transforms potentially invalid trajectory inputs into physically plausible conditions, and (2) a physics-enhanced video generator that produces high-fidelity multi-view driving videos under these conditions. To effectively train these components, we construct a large-scale, physics-rich heterogeneous dataset. Specifically, in addition to real-world driving videos, we generate diverse challenging driving scenarios using the CARLA simulator, from which we derive supervision signals that guide the model to learn physically grounded dynamics under extreme conditions. This challenging-trajectory learning strategy enables trajectory correction and promotes physically consistent video generation. Extensive experiments demonstrate that PhyGenesis consistently outperforms state-of-the-art methods, especially on challenging trajectories. Our project page is available at: https://wm-research.github.io/PhyGenesis/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。