arXiv:2506.08052cs.CVcs.RO2025-06被引 169

用认知框架统一自动驾驶理解与规划,提升安全性与稳定性。

ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving

  • 分三阶段模拟人类驾驶认知,注入先验知识增强理解
  • 融合扩散模型生成连续稳定轨迹,避免语言动作错位
  • 引入强化学习优化安全与舒适性,适合复杂场景应用

近期研究尝试利用视觉语言模型(VLMs)的世界知识与认知能力解决端到端自动驾驶中的长尾问题。然而,现有方法通常将轨迹规划建模为语言建模任务,导致动作输出格式错误、不可行且推理速度慢。本文提出ReCogDrive,一种新型强化认知框架,通过自回归模型与扩散规划器的结合,统一驾驶理解与规划。首先,设计分层数据管道,模仿人类驾驶员的生成、精炼与质量控制三阶段认知过程,将人类驾驶认知注入VLM。其次,通过将VLM学习到的驾驶先验注入扩散规划器,高效生成连续稳定的轨迹,缓解语言-动作不匹配问题。此外,引入扩散组相对策略优化(DiffGRPO)阶段,增强规划器的安全性与舒适性。在NAVSIM与Bench2Drive基准上的大量实验表明,ReCogDrive达到当前最优性能。定性结果显示其在多样化驾驶场景中具备出色的场景理解能力。所有代码、模型权重与数据集将公开,以推动后续研究。

原文摘要 · Abstract (English)

Recent studies have explored leveraging the world knowledge and cognitive capabilities of Vision-Language Models (VLMs) to address the long-tail problem in end-to-end autonomous driving. However, existing methods typically formulate trajectory planning as a language modeling task, where physical actions are output in the language space, potentially leading to issues such as format-violating outputs, infeasible actions, and slow inference speeds. In this paper, we propose ReCogDrive, a novel Reinforced Cognitive framework for end-to-end autonomous Driving, unifying driving understanding and planning by integrating an autoregressive model with a diffusion planner. First, to instill human driving cognition into the VLM, we introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers through three stages: generation, refinement, and quality control. Building on this cognitive foundation, we then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner to efficiently generate continuous and stable trajectories. Furthermore, to enhance driving safety and reduce collisions, we introduce a Diffusion Group Relative Policy Optimization (DiffGRPO) stage, reinforcing the planner for enhanced safety and comfort. Extensive experiments on the NAVSIM and Bench2Drive benchmarks demonstrate that ReCogDrive achieves state-of-the-art performance. Additionally, qualitative results across diverse driving scenarios and DriveBench highlight the model's scene comprehension. All code, model weights, and datasets will be made publicly available to facilitate subsequent research.

自动驾驶视觉语言模型扩散模型强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。