arXiv:2605.12624cs.ROcs.CV2026-05被引 3

首个统一流式架构的视觉语言动作模型,让自动驾驶规划超越人类表现。

MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving

论文配图:MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving
图 1 · 摘自论文原文
  • 统一流式设计,单次前向传播生成语言与连续动作轨迹。
  • 在长尾数据集上首次超越人类驾驶(8.20 RFS vs. 8.13 GT RFS)。
  • 保留自然语言接口,适合人车交互与实时控制场景。

自动驾驶正从模块化流程转向端到端统一,视觉-语言-动作(VLA)模型是这一演进的自然延伸。然而,实际驾驶中的VLA常落后于视觉-动作(VA)模型的规划质量,问题不在于模型规模,而在于语义推理、时序上下文与连续控制的融合方式。我们提出MindVLA-U1,首个面向自动驾驶的统一流式VLA架构。该架构采用统一视觉语言模型主干,在一次前向传播中生成可选的语言令牌与流匹配的连续动作轨迹,保持各模态的自然输出形式。全流式设计以帧为单位处理视频,而非依赖昂贵的时序建模。规划轨迹跨帧平滑演化,学习的流式记忆通道持续传递时序上下文。统一架构支持密集与稀疏多目标跟踪(MoT)主干的快慢系统,并通过无分类器引导(CFG)实现语言预测意图对动作扩散的显式控制,将语言侧意图转化为控制信号。在长尾WOD-E2E基准上,仅用2步扩散即首次超越经验人类司机(8.20 RFS vs. 8.13 GT RFS),规划精度大幅领先现有VA/VLA模型,且延迟仅16 FPS(对比RAP的18 FPS,1B参数量下),同时保留自然语言接口用于人车交互。

原文摘要 · Abstract (English)

Autonomous driving has progressed from modular pipelines toward end-to-end unification, and Vision-Language-Action (VLA) models are a natural extension of this journey beyond Vision-to-Action (VA). In practice, driving VLAs have often trailed VA on planning quality, suggesting that the difficulty is not simply model scale but the interface through which semantic reasoning, temporal context, and continuous control are combined. We argue that this gap reflects how VLA has been built -- as isolated subtask improvements that fail to compose coherent driving capabilities -- rather than what VLA is. We present MindVLA-U1, the first unified streaming VLA architecture for autonomous driving. A unified VLM backbone produces AR language tokens (optional) and flow-matching continuous action trajectories in a single forward pass over one shared representation, preserving the natural output form of each modality. A full streaming design processes the driving video framewise rather than as fixed video-action chunks under costly temporal VLM modeling. Planned trajectories evolve smoothly across frames while a learned streaming memory channel carries temporal context and updates. The unified architecture enables fast/slow systems on dense & sparse MoT backbones via flexible self-attention context management, and exposes a measurable language-control path for action: language-predicted driving intents steers the action diffusion via classifier-free guidance (CFG), turning language-side intent into control signals for continuous action planning. On the long-tail WOD-E2E benchmark, MindVLA-U1 surpasses experienced human drivers for the first time (8.20 RFS vs. 8.13 GT RFS) with 2 diffusion steps, achieves state-of-the-art planning ADEs over prior VA/VLA by large margins, and matches VA latency (16 FPS vs. RAP's 18 FPS at 1B scale) while preserving natural language interfaces for human-vehicle interaction.

自动驾驶视觉语言动作流式架构连续控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。