用离散掩码扩散模型生成自动驾驶轨迹，支持灵活解码和在线优化。
WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Driving
- 采用掩码扩散迭代生成离散轨迹序列，支持非因果灵活解码。
- 在NAVSIM-v1上达91.0 PDMS，NAVSIM-v2上达89.7 EPDMS。
- 结合MoE与在线强化学习，适合需要场景感知的自动驾驶应用。
基于视觉-语言-动作（VLA）模型的端到端自动驾驶系统融合多模态传感器输入与语言指令，生成规划与控制信号。尽管自回归大语言模型和连续扩散策略广泛应用，离散掩码扩散在轨迹生成中的潜力仍待探索。本文提出WAM-Diff，一种利用掩码扩散迭代精炼表示未来车辆轨迹的离散序列的VLA框架。其核心创新包括：系统性适配掩码扩散用于自动驾驶，支持灵活的非因果解码顺序；通过稀疏MoE架构实现可扩展模型容量，并联合训练运动预测与驾驶导向视觉问答任务；采用组序列策略优化（GSPO）的在线强化学习，以优化序列级驾驶奖励。显著地，该模型在NAVSIM-v1上达到91.0 PDMS，NAVSIM-v2上达到89.7 EPDMS，验证了掩码扩散在自动驾驶中的有效性。该方法为自回归与扩散策略提供有前景的替代方案，支持情境感知的轨迹生成解码策略。代码将公开发布于：https://github.com/fudan-generative-vision/WAM-Diff
原文摘要 · Abstract (English)
End-to-end autonomous driving systems based on vision-language-action (VLA) models integrate multimodal sensor inputs and language instructions to generate planning and control signals. While autoregressive large language models and continuous diffusion policies are prevalent, the potential of discrete masked diffusion for trajectory generation remains largely unexplored. This paper presents WAM-Diff, a VLA framework that employs masked diffusion to iteratively refine a discrete sequence representing future ego-trajectories. Our approach features three key innovations: a systematic adaptation of masked diffusion for autonomous driving that supports flexible, non-causal decoding orders; scalable model capacity via a sparse MoE architecture trained jointly on motion prediction and driving-oriented visual question answering (VQA); and online reinforcement learning using Group Sequence Policy Optimization (GSPO) to optimize sequence-level driving rewards. Remarkably, our model achieves 91.0 PDMS on NAVSIM-v1 and 89.7 EPDMS on NAVSIM-v2, demonstrating the effectiveness of masked diffusion for autonomous driving. The approach provides a promising alternative to autoregressive and diffusion-based policies, supporting scenario-aware decoding strategies for trajectory generation. The code for this paper will be released publicly at: https://github.com/fudan-generative-vision/WAM-Diff
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。