arXiv:2506.11234cs.ROcs.CV2025-06被引 12

用视觉语言轨迹预训练+轻量强化学习,实现更可靠的端到端自动驾驶。

Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Driving

  • 用视觉、语言、轨迹三模态自监督预训练,构建基础驾驶能力。
  • 仅用少量人类偏好标注数据微调,测试集RFS达7.99,排名第一。
  • 无需定制模型结构,可复用现成大模型,适合快速部署和研究。

在分布外场景下保持良好驾驶行为仍是自动驾驶的核心挑战。一种有前景的方向是利用大语言模型的通用知识与推理能力,将异常驾驶场景视为逻辑推理任务。本文提出Poutine,一种基于30亿参数现成视觉语言模型(VLM)的方法,通过简单可扩展的训练流程实现鲁棒的端到端自动驾驶。第一阶段,使用自监督的下一个词预测,在视觉、语言和轨迹(VLT)token上训练Poutine-Base,融合正常与长尾驾驶数据。第二阶段,采用小样本人类偏好标注数据,通过组相对策略优化(GRPO)对Poutine-Base进行微调。我们在为长尾场景设计的Waymo端到端驾驶基准上评估,最终模型在测试集上RFS达到7.99,显著领先,位列2025年Waymo基于视觉的端到端驾驶挑战赛第一名。结果表明,以往工作中的手工分词器或定制架构并非必要。本工作强调,可扩展的VLT预训练结合轻量级强化学习微调,即可实现鲁棒且泛化的自动驾驶。

原文摘要 · Abstract (English)

Maintaining good driving behavior in out-of-distribution scenarios remains a critical challenge in autonomous driving. A promising direction is to leverage the generalist knowledge and reasoning capabilities of large-language models by treating unusual driving scenarios as a logical reasoning task. In this work, we present Poutine, a method that uses an off-the-shelf 3B-parameter vision-language model (VLM) - without any additional components - to achieve robust end-to-end autonomous driving via a simple and scalable training recipe. To learn strong base driving capabilities, we first train Poutine-Base using self-supervised next-token prediction over vision, language, and trajectory (VLT) tokens, leveraging both nominal and long-tail driving data. In the second stage, we fine-tune Poutine-Base using Group Relative Policy Optimization (GRPO) with a small set of human preference-labeled examples. We evaluated our approach on the Waymo end-to-end driving benchmark curated for long-tail scenarios. The final Poutine model achieves an RFS of 7.99 on the test set, placing 1st in the 2025 Waymo Vision-Based End-to-End Driving Challenge by a significant margin. Our results suggest that handcrafted tokenizers or custom architectural components added to base VLMs in prior work are not necessary to achieve strong driving performance. Instead, this work highlights the potential of scalable VLT pretraining combined with lightweight RL fine-tuning to enable robust and generalizable autonomous driving.

端到端驾驶多模态预训练强化学习自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。