arXiv:2506.13757cs.CV2025-06NeurIPS被引 263

AutoVLA让自动驾驶模型自适应推理,直接从视觉和语言生成可行轨迹。

AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning

论文配图:AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
图 1 · 摘自论文原文
  • 统一推理与动作生成,用离散动作令牌化实现端到端规划。
  • 在nuPlan、Waymo等数据集上表现优异,闭环测试成功率超90%。
  • 支持快速与慢速思维切换,适合复杂场景下的智能驾驶系统。

近期视觉-语言-动作(VLA)模型在端到端自动驾驶中展现出利用世界知识与推理能力的潜力。然而,现有模型常面临动作输出物理不可行、结构复杂或推理过长等问题。本文提出AutoVLA,一种将推理与动作生成统一于单一自回归生成模型的新型VLA架构,可直接从原始视觉输入与语言指令中完成语义推理与轨迹规划。通过将连续轨迹离散化为可执行动作,实现与语言模型的无缝集成。训练阶段采用监督微调,赋予模型双思维模式:快速思维(仅轨迹)与慢速思维(带思维链)。为进一步提升规划性能与效率,引入基于组相对策略优化(GRPO)的强化微调方法,在简单场景中减少冗余推理。在真实世界与模拟数据集(nuPlan、nuScenes、Waymo、CARLA)上的大量实验表明,AutoVLA在开环与闭环设置下均表现优异。定性结果展示其在多样场景中的自适应推理与精准规划能力。

原文摘要 · Abstract (English)

Recent advancements in Vision-Language-Action (VLA) models have shown promise for end-to-end autonomous driving by leveraging world knowledge and reasoning capabilities. However, current VLA models often struggle with physically infeasible action outputs, complex model structures, or unnecessarily long reasoning. In this paper, we propose AutoVLA, a novel VLA model that unifies reasoning and action generation within a single autoregressive generation model for end-to-end autonomous driving. AutoVLA performs semantic reasoning and trajectory planning directly from raw visual inputs and language instructions. We tokenize continuous trajectories into discrete, feasible actions, enabling direct integration into the language model. For training, we employ supervised fine-tuning to equip the model with dual thinking modes: fast thinking (trajectory-only) and slow thinking (enhanced with chain-of-thought reasoning). To further enhance planning performance and efficiency, we introduce a reinforcement fine-tuning method based on Group Relative Policy Optimization (GRPO), reducing unnecessary reasoning in straightforward scenarios. Extensive experiments across real-world and simulated datasets and benchmarks, including nuPlan, nuScenes, Waymo, and CARLA, demonstrate the competitive performance of AutoVLA in both open-loop and closed-loop settings. Qualitative results showcase the adaptive reasoning and accurate planning capabilities of AutoVLA in diverse scenarios.

自动驾驶视觉语言动作自回归生成强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。