arXiv:2603.08113cs.CV2026-03被引 4

让自动驾驶模型按路况选专家,更稳更安全。

SAMoE-VLA: A Scene Adaptive Mixture-of-Experts Vision-Language-Action Model for Autonomous Driving

  • 用鸟瞰图特征决定哪个专家负责,而非逐个词判断。
  • 在nuScenes和LangAuto上表现超越现有方法,参数更少。
  • 适合需要可靠决策的自动驾驶系统研发者。

视觉-语言-动作(VLA)模型在自动驾驶中展现出潜力,但直接沿用大语言模型的基于标记的混合专家(MoE)机制会导致性能不稳定与安全性下降,暴露了标记级专家分工与场景级决策之间的不匹配。为此,我们提出SAMoE-VLA,一种基于场景自适应的VLA框架,将专家选择条件化于结构化的场景表征,而非标记嵌入。核心思想是利用鸟瞰图(BEV)特征生成路由信号,捕捉交通场景上下文,实现针对不同驾驶条件的专家加权与融合。此外,为支持跨世界知识、感知、语言和动作的时序一致性推理,我们引入条件交叉模态因果注意力机制,将世界状态、语言意图与动作历史整合进统一的因果推理流程。在nuScenes开放回路规划数据集与LangAuto闭环基准上的大量实验表明,SAMoE-VLA达到当前最佳性能,优于以往VLA及世界模型方法,且参数量更少。代码即将开源。

原文摘要 · Abstract (English)

Recent advances in Vision-Language-Action (VLA) models have shown promising capabilities in autonomous driving by leveraging the understanding and reasoning strengths of Large Language Models(LLMs).However, our empirical analysis reveals that directly applying existing token-level MoE mechanisms--which are inherited from LLM architectures--to VLA models results in unstable performance and safety degradation in autonomous driving, highlighting a misalignment between token-based expert specialization and scene-level decision-making.To address this, we propose SAMoE-VLA, a scene-adaptive Vision-Language-Action framework that conditions expert selection on structured scene representations instead of token embeddings. Our key idea is to derive the MoE routing signal from bird's-eye-view (BEV) features that encapsulates traffic scene context, enabling scenario-dependent expert weighting and merging tailored to distinct driving conditions. Furthermore, to support temporally consistent reasoning across world-knowledge, perception, language, and action, we introduce a Conditional Cross-Modal Causal Attention mechanism that integrates world state, linguistic intent, and action history into a unified causal reasoning process. Extensive experiments on the nuScenes open loop planning dataset and LangAuto closed-loop benchmark demonstrate that SAMoE-VLA achieves state-of-the-art performance, outperforming prior VLA-based and world-model-based approaches with fewer parameters.Our code will be released soon.

自动驾驶混合专家多模态推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。