arXiv:2505.20665cs.CV2025-05被引 6

用视觉语言模型实现自动驾驶多阶段推理,提升复杂场景下的决策能力。

DriveRX: A Vision-Language Reasoning Model for Cross-Task Autonomous Driving

  • 将驾驶任务拆解为感知、预测、规划、行为四阶段,每阶段作为视觉语言问答问题建模。
  • 在公开基准上表现优于GPT-4o,复杂或受损条件下仍保持鲁棒性。
  • 生成结构化推理链,可用于标注与下游控制模型的监督信号。

高效自动驾驶依赖于感知、预测、规划与行为之间的稳健推理。然而,传统端到端模型因缺乏结构化推理,在复杂场景下泛化能力差。尽管近期视觉语言模型(VLMs)被应用于驾驶任务,但通常依赖孤立模块和静态监督,难以支持多阶段决策。我们提出AutoDriveRL统一训练框架,将自动驾驶建模为对四个核心任务的结构化推理过程。每个任务独立建模为视觉语言问答问题,并通过任务特异性奖励模型优化,实现不同推理阶段的细粒度强化信号。在此框架下,我们训练出DriveRX——一个面向多阶段决策的跨任务推理VLM。DriveRX在公开基准上表现优异,行为推理性能超越GPT-4o,且在复杂或受干扰驾驶条件下依然稳健。其输出为高层语义推理主干,生成结构化的分阶段推理链,提升决策一致性,同时为标注与下游规划/控制模型提供高质量监督信号。我们已开源AutoDriveRL框架与DriveRX,以支持后续研究。

原文摘要 · Abstract (English)

Effective autonomous driving hinges on robust reasoning across perception, prediction, planning, and behavior. However, conventional end-to-end models fail to generalize in complex scenarios due to the lack of structured reasoning. While recent vision-language models (VLMs) have been applied to driving tasks, they typically rely on isolated modules and static supervision, limiting their ability to support multi-stage decision-making. We present AutoDriveRL, a unified training framework that formulates autonomous driving as a structured reasoning process over four core tasks. Each task is independently modeled as a vision-language QA problem and optimized using task-specific reward models, enabling fine-grained reinforcement signals at different reasoning stages. Within this framework, we train DriveRX, a cross-task reasoning VLM designed for multi-stage decision-making. DriveRX achieves strong performance on the public benchmark, outperforming GPT-4o in behavior reasoning and demonstrating robustness under complex or corrupted driving conditions. DriveRX serves as a high-level semantic reasoning backbone, producing structured stage-wise reasoning chains that enhance decision consistency. These outputs also provide high-quality supervisory signals for annotation and downstream planning/control models. We release the AutoDriveRL framework and DriveRX to support future research.

自动驾驶视觉语言模型多阶段推理强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。