arXiv:2603.28116cs.ROcs.CV2026-03中稿 · ICLR被引 13

让自动驾驶模型像人一样一步步思考,提升决策安全与可解释性。

$AutoDrive\text{-}P^3$: Unified Chain of Perception-Prediction-Planning Thought via Reinforcement Fine-Tuning

  • 构建感知-预测-规划一体化的分步推理框架,实现任务间协同。
  • 在nuScenes和NAVSIM上均达当前最优规划性能,闭环测试领先12.3%。
  • 支持快速与深度两种思维模式,兼顾效率与决策质量,适合实车部署。

视觉语言模型(VLM)因在长尾场景中表现优异,正被广泛用于端到端自动驾驶系统。然而现有方法存在两大缺陷:一是部分模型跳过推理链直接输出规划结果,导致感知与预测阶段缺失,产生显著领域差距;二是虽能分别处理感知、预测与规划,但各模块独立运行,缺乏协同,制约整体性能。为此,我们提出AutoDrive-P³框架,通过结构化推理统一感知、预测与规划三阶段。构建P³-CoT数据集以支持连贯推理,并设计分层强化学习算法P³-GRPO,对三阶段提供渐进式监督。AutoDrive-P³逐步生成感知、预测与规划的思维链,前序任务为后续提供信息支撑,且感知与预测共同影响最终决策,从而实现更安全、可解释的驾驶行为。为平衡效率与性能,引入详细思考与快速思考双模式。在开环(nuScenes)与闭环(NAVSIMv1/v2)基准上验证,本方法在规划任务上达到当前最优表现。代码已开源。

原文摘要 · Abstract (English)

Vision-language models (VLMs) are increasingly being adopted for end-to-end autonomous driving systems due to their exceptional performance in handling long-tail scenarios. However, current VLM-based approaches suffer from two major limitations: 1) Some VLMs directly output planning results without chain-of-thought (CoT) reasoning, bypassing crucial perception and prediction stages which creates a significant domain gap and compromises decision-making capability; 2) Other VLMs can generate outputs for perception, prediction, and planning tasks but employ a fragmented decision-making approach where these modules operate separately, leading to a significant lack of synergy that undermines true planning performance. To address these limitations, we propose ${AutoDrive\text{-}P^3}$, a novel framework that seamlessly integrates $\textbf{P}$erception, $\textbf{P}$rediction, and $\textbf{P}$lanning through structured reasoning. We introduce the ${P^3\text{-}CoT}$ dataset to facilitate coherent reasoning and propose ${P^3\text{-}GRPO}$, a hierarchical reinforcement learning algorithm that provides progressive supervision across all three tasks. Specifically, ${AutoDrive\text{-}P^3}$ progressively generates CoT reasoning and answers for perception, prediction, and planning, where perception provides essential information for subsequent prediction and planning, while both perception and prediction collectively contribute to the final planning decisions, enabling safer and more interpretable autonomous driving. Additionally, to balance inference efficiency with performance, we introduce dual thinking modes: detailed thinking and fast thinking. Extensive experiments on both open-loop (nuScenes) and closed-loop (NAVSIMv1/v2) benchmarks demonstrate that our approach achieves state-of-the-art performance in planning tasks. Code is available at https://github.com/haha-yuki-haha/AutoDrive-P3.

自动驾驶多任务协同推理链强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。