用端到端驾驶模型评估生成视频的真实性,提升自动驾驶泛化能力。
Drive&Gen: Co-Evaluating End-to-End Driving and Video Generation Models
- 用驾驶模型作为评判标准,检验生成视频是否真实可控。
- 合成数据可有效提升驾驶模型在新场景下的泛化性能。
- 为自动驾驶测试提供低成本、高效率的虚拟数据方案。
生成模型的发展为自动驾驶领域带来新可能,视频生成模型可作为可控的虚拟测试环境。与此同时,端到端(E2E)驾驶模型因其简洁性和可扩展性,正替代传统模块化系统。然而,这些技术用于仿真与规划时面临关键问题:第一,生成视频能否忠实遵循指定条件并足够真实以供E2E规划器评估?第二,如何通过数据深入理解并改进E2E模型对分布外场景的泛化能力?本文提出Drive&Gen框架,将驾驶模型与生成世界模型协同评估。我们设计新的统计指标,利用E2E驾驶员评估生成视频的现实性;通过可控生成实验,揭示影响E2E规划器性能的分布差距;最终证明,由视频生成模型产生的合成数据能有效替代真实数据采集,显著提升E2E模型在现有运行域之外的泛化能力,推动自动驾驶服务拓展至新运营场景。
原文摘要 · Abstract (English)
Recent advances in generative models have sparked exciting new possibilities in the field of autonomous vehicles. Specifically, video generation models are now being explored as controllable virtual testing environments. Simultaneously, end-to-end (E2E) driving models have emerged as a streamlined alternative to conventional modular autonomous driving systems, gaining popularity for their simplicity and scalability. However, the application of these techniques to simulation and planning raises important questions. First, while video generation models can generate increasingly realistic videos, can these videos faithfully adhere to the specified conditions and be realistic enough for E2E autonomous planner evaluation? Second, given that data is crucial for understanding and controlling E2E planners, how can we gain deeper insights into their biases and improve their ability to generalize to out-of-distribution scenarios? In this work, we bridge the gap between the driving models and generative world models (Drive&Gen) to address these questions. We propose novel statistical measures leveraging E2E drivers to evaluate the realism of generated videos. By exploiting the controllability of the video generation model, we conduct targeted experiments to investigate distribution gaps affecting E2E planner performance. Finally, we show that synthetic data produced by the video generation model offers a cost-effective alternative to real-world data collection. This synthetic data effectively improves E2E model generalization beyond existing Operational Design Domains, facilitating the expansion of autonomous vehicle services into new operational contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。