将自回归驾驶模型转为并行扩散模型,提速近15倍且不丢性能。
WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA

- 分三阶段逐步迁移架构,解决注意力模式与目标差异问题。
- 在多个驾驶任务上性能持平基线,推理速度提升2.8倍。
- 适合追求高效部署的自动驾驶系统研发人员。
视觉-语言-动作(VLA)模型已成为端到端自动驾驶的主流范式,但其高效部署受限于高计算延迟和序列自回归解码带来的暴露偏差。尽管专用扩散策略可实现低延迟并行执行,但从头训练通常导致能力单一、缺乏全局视觉-语言推理。将预训练的自回归通用模型转化为并行扩散模型,有望融合多任务认知智能与执行效率,但因注意力模式(因果性对双向性)不匹配及优化目标差异,面临巨大架构挑战。为此,我们提出WAM-Diff2,一种基于三阶段层次化蒸馏策略的多任务离散扩散VLA框架。通过逐块适应、逐块蒸馏与跨尺度模型蒸馏,该方法在保留基础模型语义结构的同时显著加速推理。在驾驶理解、感知与规划基准上的广泛评估表明,WAM-Diff2有效缓解暴露偏差,并实现与自回归基线相当的性能。关键的是,自回归到扩散的转换带来2.8倍解码速度提升,结合FlashInfer与CUDA Graphs等系统级优化后,最终实现15.1倍加速。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have emerged as a prominent paradigm for end-to-end autonomous driving; however, their efficient deployment is severely constrained by high computational latency and exposure bias arising from sequential autoregressive decoding. Conversely, while specialized diffusion policies enable low-latency, parallel execution, training them from scratch typically yields narrow, single-task architectures that lack holistic visual-linguistic reasoning. Successfully transforming pre-trained autoregressive generalists into parallel diffusion models could combine multi-task cognitive intelligence with execution efficiency, yet this transition presents a formidable architectural challenge due to mismatched attention patterns (causal versus bidirectional) and divergent optimization objectives. To bridge this divide, we introduce WAM-Diff2, a multi-task discrete diffusion VLA framework powered by a three-stage hierarchical distillation strategy. By structuring the architectural shift through progressive block-wise adaptation, block-wise distillation, and model-wise cross-scale distillation, WAM-Diff2 preserves the underlying semantic foundations of the base model while accelerating inference. Extensive evaluations across driving understanding, perception, and planning benchmarks demonstrate that WAM-Diff2 effectively mitigates exposure bias and achieves performance parity with autoregressive baselines. Crucially, the autoregressive-to-diffusion transition yields a 2.8x decoding speedup, which scales to an ultimate 15.1x acceleration when combined with system-level optimizations including FlashInfer and CUDA Graphs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。