用大模型理解刹车原因,挖掘驾驶数据中的危险场景
Why Braking? Scenario Extraction and Reasoning Utilizing LLM
- 用大模型将车辆信号转为自然语言,实现场景理解
- 在Argoverse 2数据集上优于传统规则方法,能识别未知危险场景
- 适合自动驾驶安全研究者和数据工程师参考
越来越多配备ADAS的车辆产生了海量驾驶数据,但其中多数是常规行驶行为。如何从这些数据中识别并理解安全关键的极端场景仍是重大挑战。刹车事件尤其能反映潜在危险情况,因此我们提出核心问题:车辆为何刹车?现有方法主要依赖预设规则筛选目标场景,在高速公路等简单环境有效,但在复杂城市环境中泛化能力不足。本文提出一种基于大语言模型(LLM)的新框架,用于场景理解和推理。该方法连接低层数值信号与自然语言描述,使LLM能够解析和分类驾驶场景。我们设计双路径检索机制:支持已知场景的类别搜索,也支持未知分布外(OOD)场景的嵌入检索。为便于评估,我们在Argoverse 2 Sensor Dataset上构建了场景标注。实验表明,本方法优于基于规则的基线,并在处理OOD场景时表现良好。
原文摘要 · Abstract (English)
The growing number of ADAS-equipped vehicles has led to a dramatic increase in driving data, yet most of them capture routine driving behavior. Identifying and understanding safety-critical corner cases within this vast dataset remains a significant challenge. Braking events are particularly indicative of potentially hazardous situations, motivating the central question of our research: Why does a vehicle brake? Existing approaches primarily rely on rule-based heuristics to retrieve target scenarios using predefined condition filters. While effective in simple environments such as highways, these methods lack generalization in complex urban settings. In this paper, we propose a novel framework that leverages Large Language Model (LLM) for scenario understanding and reasoning. Our method bridges the gap between low-level numerical signals and natural language descriptions, enabling LLM to interpret and classify driving scenarios. We propose a dual-path scenario retrieval that supports both category-based search for known scenarios and embedding-based retrieval for unknown Out-of-Distribution (OOD) scenarios. To facilitate evaluation, we curate scenario annotations on the Argoverse 2 Sensor Dataset. Experimental results show that our method outperforms rule-based baselines and generalizes well to OOD scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。