arXiv:2412.04209cs.RO2024-12被引 13

用大模型生成多决策并评估信心,让自动驾驶更安全智能

CALMM-Drive: Confidence-Aware Autonomous Driving with Large Multimodal Model

  • 用链式思考+置信度筛选生成多个驾驶决策
  • 融合扩散模型与分层优化,选出最优轨迹
  • 兼顾短期平滑与长期策略,适合复杂路况

决策与运动规划是保障自动驾驶车辆安全高效的核心环节。现有方法多采用‘先决策后规划’或‘先生成后评分’的范式,前者常因决策与规划不一致导致风险,后者难以平衡短期操作指标(如运动平顺性)与长期战术目标(如路线效率),表现出短视或过度保守行为。为此,我们提出CALMM-Drive——一种基于大多模态模型(LMM)的自信感知自动驾驶框架。该方法引入面向驾驶任务的链式思维(CoT)推理与Top-K置信度提取,生成多个候选决策及其置信度。进一步设计新型规划模块,结合扩散模型进行轨迹生成,并通过分层精炼过程筛选最优轨迹。该框架在选择轨迹时同时考虑低层解质量与高层战术置信度,避免单次决策风险,突破短视评分机制局限。在nuPlan闭环仿真环境中全面评估显示,CALMM-Drive在常规与长尾场景下均表现优异,显著推进了大模型赋能自动驾驶中不确定性建模的集成水平。代码将在论文接收后公开。

原文摘要 · Abstract (English)

Decision-making and motion planning constitute critical components for ensuring the safety and efficiency of autonomous vehicles (AVs). Existing methodologies typically adopt two paradigms: decision then planning or generation then scoring. However, the former architecture often suffers from decision-planning misalignment that incurs risky situations. Meanwhile, the latter struggles to balance short-term operational metrics (e.g., immediate motion smoothness) with long-term tactical goals (e.g., route efficiency), resulting in myopic or overly conservative behaviors. To address these issues, we introduce CALMM-Drive, a novel Confidence-Aware Large Multimodal Model (LMM) empowered Autonomous Driving framework. Our approach integrates driving task-oriented Chain-of-Thought (CoT) reasoning coupled with Top-K confidence elicitation, which facilitates high-level reasoning to generate multiple candidate decisions with their confidence levels. Furthermore, we propose a novel planning module that integrates a diffusion model for trajectory generation and a hierarchical refinement process to find the optimal trajectory. This framework enables the selection over trajectory candidates accounting for both low-level solution quality and high-level tactical confidence, which avoids the risks within one-shot decisions and overcomes the limitations in short-sighted scoring mechanisms. Comprehensive evaluations in nuPlan closed-loop simulation environments demonstrate the competitive performance of CALMM-Drive across both common and long-tail benchmarks, showcasing a significant advancement in the integration of uncertainty in LMM-empowered AVs. The code will be released upon acceptance.

自动驾驶大模型决策规划置信度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。