arXiv:2604.16514cs.CVcs.LG2026-04

将自回归视觉语言模型高效转为并行扩散模型,提升推理速度且不损失性能。

BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation

论文配图:BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation
图 1 · 摘自论文原文
  • 通过渐进式块合并与分阶段知识蒸馏,实现自回归到扩散模型的平滑转换。
  • 仅用440万数据即在4B和8B规模上达到同类开源扩散模型新SOTA。
  • 支持大块并行解码,推理速度最高提升3倍,适合长序列多模态应用。

自回归视觉语言模型具备强大的多模态能力,但逐标记解码存在根本性推理瓶颈。扩散模型虽支持更并行的解码方式,但直接将预训练自回归模型转为大块扩散模型(dVLM)常导致性能显著下降。本文提出BARD,一种简单有效的桥梁框架,可将预训练自回归VLM转化为同架构、高效率的dVLM。方法结合渐进式监督块合并(逐步增大解码块大小)与分阶段内部蒸馏(从固定小块扩散锚点恢复大块性能损失)。引入混合噪声调度器以增强去噪鲁棒性和标记修正能力,并采用内存友好训练策略,支持长多模态序列高效训练。关键发现:自回归到扩散的直接蒸馏对齐差,甚至损害性能;而在扩散域内蒸馏则始终有效。实验表明,仅需≤4.4M数据,BARD-VL即可将Qwen3-VL的多模态能力迁移至大块dVLM。令人瞩目的是,在同等规模下,BARD-VL在4B和8B两个尺度上均达到当前可比开源dVLM的新SOTA。同时,相比源模型,解码吞吐量最高提升3倍。代码已开源:https://github.com/fudan-generative-vision/Bard-VL。

原文摘要 · Abstract (English)

Autoregressive vision-language models (VLMs) deliver strong multimodal capability, but their token-by-token decoding imposes a fundamental inference bottleneck. Diffusion VLMs offer a more parallel decoding paradigm, yet directly converting a pretrained autoregressive VLM into a large-block diffusion VLM (dVLM) often leads to substantial quality degradation. In this work, we present BARD, a simple and effective bridging framework that converts a pretrained autoregressive VLM into a same-architecture, decoding-efficient dVLM. Our approach combines progressive supervised block merging, which gradually enlarges the decoding block size, with stage-wise intra-dVLM distillation from a fixed small-block diffusion anchor to recover performance lost at larger blocks. We further incorporate a mixed noise scheduler to improve robustness and token revision during denoising, and memory-friendly training to enable efficient training on long multimodal sequences. A key empirical finding is that direct autoregressive-to-diffusion distillation is poorly aligned and can even hurt performance, whereas distillation within the diffusion regime is consistently effective. Experimental results show that, with $\leq$ 4.4M data, BARD-VL transfers strong multimodal capability from Qwen3-VL to a large-block dVLM. Remarkably, BARD-VL establishes a new SOTA among comparable-scale open dVLMs on our evaluation suite at both 4B and 8B scales. At the same time, BARD-VL achieves up to 3$\times$ decoding throughput speedup compared to the source model. Code is available at https://github.com/fudan-generative-vision/Bard-VL.

视觉语言模型扩散模型模型转换推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。