arXiv:2503.07020cs.ROcs.AI2025-03被引 1

用大模型让自动驾驶在感知缺失时更聪明地避险,不盲目急停。

Driving Through Uncertainty: Risk-Averse Control with LLM Commonsense for Autonomous Driving under Perception Deficits

  • 引入大模型理解驾驶常识,动态判断风险程度
  • 在CARLA模拟中实现主动避险,减少无意义急停
  • 适合研究自动驾驶安全与决策的学者参考

部分感知缺陷会破坏自动驾驶对环境的理解,威胁行车安全。现有方法通常采用完全规避风险的策略,如立即停车,虽安全但影响通行效率,且难以应对罕见场景。本文提出LLM-RCO框架,利用大语言模型(LLMs)融合人类驾驶常识,提升系统在感知缺陷下的决策能力。该框架包含四个核心模块:危险推断、短期运动规划、动作条件验证和安全约束生成,可实现动态响应。为训练模型,我们构建了包含53,895段视频的DriveLM-Deficit数据集,涵盖关键物体感知缺失场景,用于标注危险检测与路径规划。在CARLA仿真器中的大量实验表明,相较于纯规避策略,LLM-RCO能更主动地执行合理避险动作,显著增强自动驾驶系统在感知退化情况下的鲁棒性。

原文摘要 · Abstract (English)

Partial perception deficits can compromise autonomous vehicle safety by disrupting environmental understanding. Existing protocols typically default to entirely risk-avoidant actions such as immediate stops, which are detrimental to navigation goals and lack flexibility for rare driving scenarios. Yet, in cases of minor risk, halting the vehicle may be unnecessary, and more adaptive responses are preferable. In this paper, we propose LLM-RCO, a risk-averse framework leveraging large language models (LLMs) to integrate human-like driving commonsense into autonomous systems facing perception deficits. LLM-RCO features four key modules interacting with the dynamic driving environment: hazard inference, short-term motion planner, action condition verifier, and safety constraint generator, enabling proactive and context-aware actions in such challenging conditions. To enhance the driving decision-making of LLMs, we construct DriveLM-Deficit, a dataset of 53,895 video clips featuring deficits of safety-critical objects, annotated for LLM fine-tuning in hazard detection and motion planning. Extensive experiments in adverse driving conditions with the CARLA simulator demonstrate that LLM-RCO promotes proactive maneuvers over purely risk-averse actions in perception deficit scenarios, underscoring its value for boosting autonomous driving resilience against perception loss challenges.

自动驾驶大模型风险控制感知缺陷

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。