用视频自动生成自动驾驶测试场景,高效还原真实事故。
From Dashcam Videos to Driving Simulations: Stress Testing Automated Vehicles against Rare Events
- 用提示工程的视觉语言模型将行车记录仪视频转为CARLA仿真脚本。
- 分钟级完成转换,无需人工干预,仿真结果与原视频行为高度一致。
- 支持天气道路参数灵活调整,适合做极端场景压力测试。
在仿真中测试自动驾驶系统(ADS)的性能至关重要,但将真实驾驶视频转化为仿真场景面临巨大挑战,主要源于高维视频数据的复杂性及手动重建耗时。本文提出一种新框架,可自动将真实车祸视频转化为详细的仿真测试场景。该方法利用提示工程的视觉语言模型(VLM),将行车记录仪视频转化为SCENIC脚本,用于定义CARLA仿真器中的环境与驾驶行为,从而生成逼真仿真场景。不同于追求完全复现,本框架聚焦于捕捉原始视频中的核心驾驶行为,并支持天气、道路条件等参数灵活调整,便于开展基于搜索的测试。此外,我们引入一种相似性度量,通过对比真实与仿真视频中的关键驾驶行为特征,实现迭代优化。初步结果显示,整个转换过程仅需数分钟即可完成,全程自动化,无须人工介入,且保持了与原始事件的高度保真度。
原文摘要 · Abstract (English)
Testing Automated Driving Systems (ADS) in simulation with realistic driving scenarios is important for verifying their performance. However, converting real-world driving videos into simulation scenarios is a significant challenge due to the complexity of interpreting high-dimensional video data and the time-consuming nature of precise manual scenario reconstruction. In this work, we propose a novel framework that automates the conversion of real-world car crash videos into detailed simulation scenarios for ADS testing. Our approach leverages prompt-engineered Video Language Models(VLM) to transform dashcam footage into SCENIC scripts, which define the environment and driving behaviors in the CARLA simulator, enabling the generation of realistic simulation scenarios. Importantly, rather than solely aiming for one-to-one scenario reconstruction, our framework focuses on capturing the essential driving behaviors from the original video while offering flexibility in parameters such as weather or road conditions to facilitate search-based testing. Additionally, we introduce a similarity metric that helps iteratively refine the generated scenario through feedback by comparing key features of driving behaviors between the real and simulated videos. Our preliminary results demonstrate substantial time efficiency, finishing the real-to-sim conversion in minutes with full automation and no human intervention, while maintaining high fidelity to the original driving events.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。