让自动驾驶仿真更可靠,能模拟危险驾驶行为。
ReSim: Reliable World Simulation for Autonomous Driving
- 用真实数据+模拟器数据混合训练可控世界模型
- 视觉保真度提升44%,非专家行为控制力增50%以上
- 适合评估自动驾驶策略与规划算法的场景
如何在多种车辆行为下可靠地模拟未来驾驶场景?现有基于真实驾驶数据(以安全专家轨迹为主)的世界模型难以模拟罕见的危险或非专家行为,限制了其在策略评估等任务中的应用。本文通过引入来自CARLA等驾驶模拟器的多样化非专家数据,丰富真实人类示范数据,并在此异构数据集上训练可控世界模型。基于扩散变换器架构的视频生成器,我们设计了多种条件信号融合策略,显著提升预测可控性与保真度。所提出的ReSim模型可实现对各类动作(包括危险非专家行为)下多样开放世界驾驶场景的可靠模拟。为进一步弥合高保真仿真与需奖励信号评估行为的任务之间的差距,我们引入Video2Reward模块,从模拟未来视频中估计奖励。ReSim在视觉保真度上提升最高达44%,对专家与非专家行为的控制力均提高超50%,并在NAVSIM上使规划与策略选择性能分别提升2%和25%。
原文摘要 · Abstract (English)
How can we reliably simulate future driving scenarios under a wide range of ego driving behaviors? Recent driving world models, developed exclusively on real-world driving data composed mainly of safe expert trajectories, struggle to follow hazardous or non-expert behaviors, which are rare in such data. This limitation restricts their applicability to tasks such as policy evaluation. In this work, we address this challenge by enriching real-world human demonstrations with diverse non-expert data collected from a driving simulator (e.g., CARLA), and building a controllable world model trained on this heterogeneous corpus. Starting with a video generator featuring a diffusion transformer architecture, we devise several strategies to effectively integrate conditioning signals and improve prediction controllability and fidelity. The resulting model, ReSim, enables Reliable Simulation of diverse open-world driving scenarios under various actions, including hazardous non-expert ones. To close the gap between high-fidelity simulation and applications that require reward signals to judge different actions, we introduce a Video2Reward module that estimates a reward from ReSim's simulated future. Our ReSim paradigm achieves up to 44% higher visual fidelity, improves controllability for both expert and non-expert actions by over 50%, and boosts planning and policy selection performance on NAVSIM by 2% and 25%, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。