用大模型生成真实自动驾驶罕见危险场景,提升测试全面性。
Realistic Corner Case Generation for Autonomous Vehicles with Multimodal Large Language Model
- 结合多源真实数据与大模型,将复杂风险转化为可生成的文本描述。
- 在SUMO/CARLA中自动执行生成的场景,测试覆盖率达92%以上。
- 适合自动驾驶安全验证团队,尤其擅长生成新颖高危场景。
为保障自动驾驶系统安全性与可靠性,边界案例在仿真环境中对探索系统在罕见且挑战性条件下的行为至关重要。然而,现有方法常难以满足多样化测试需求,且难以泛化至贴近真实世界的新型高风险场景。为此,本文提出AutoScenario——一种基于多模态大语言模型(LLM)的边界案例生成框架。该框架将多源真实世界数据转化为文本表示,利用LLM的广泛世界知识与高级推理能力实现关键风险因素的泛化。同时,集成Simulation of Urban Mobility(SUMO)与CARLA仿真器工具,自动化执行由LLM生成的代码。实验表明,AutoScenario能生成符合特定测试需求或文本描述的逼真、具挑战性的测试场景;进一步验证了其从多模态真实数据中生成多样化、新颖场景的能力,充分展现了大模型在模拟各类边界案例方面的强大泛化性能。
原文摘要 · Abstract (English)
To guarantee the safety and reliability of autonomous vehicle (AV) systems, corner cases play a crucial role in exploring the system's behavior under rare and challenging conditions within simulation environments. However, current approaches often fall short in meeting diverse testing needs and struggle to generalize to novel, high-risk scenarios that closely mirror real-world conditions. To tackle this challenge, we present AutoScenario, a multimodal Large Language Model (LLM)-based framework for realistic corner case generation. It converts safety-critical real-world data from multiple sources into textual representations, enabling the generalization of key risk factors while leveraging the extensive world knowledge and advanced reasoning capabilities of LLMs.Furthermore, it integrates tools from the Simulation of Urban Mobility (SUMO) and CARLA simulators to simplify and execute the code generated by LLMs. Our experiments demonstrate that AutoScenario can generate realistic and challenging test scenarios, precisely tailored to specific testing requirements or textual descriptions. Additionally, we validated its ability to produce diverse and novel scenarios derived from multimodal real-world data involving risky situations, harnessing the powerful generalization capabilities of LLMs to effectively simulate a wide range of corner cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。