一种可切换三种生成模式的统一语言模型，提升推理速度与效率。
Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding
- 同一架构融合自回归、扩散与自推测三种生成方式。
- 8B模型每轮推理比Qwen3多6倍token，吞吐量提升4倍。
- 适合追求高并发、低延迟推理的应用场景。
我们提出Nemotron-Labs-Diffusion，一种统一自回归（AR）、扩散与自推测解码的三模态语言模型。通过联合训练AR-扩散目标，该模型可在不同部署环境与并发水平下动态切换模式以保持高吞吐。研究发现：(1) AR与扩散目标互补——扩散增强前瞻规划能力，自回归提供左右顺序的语言先验；(2) 在自推测模式中，扩散生成草稿，自回归验证，其接受率与真实设备效率均优于多标记预测方法；(3) 速度极快分析表明，最优采样器下扩散模式每前向传播可生成最多76.5%更多token。模型规模扩展至3B、8B和14B，包含基础、指令微调及视觉语言版本，在准确率与速度上持续超越现有开源自回归与扩散语言模型。例如，Nemotron-Labs-Diffusion-8B在相同精度下每前向传播解码量为Qwen3-8B的6倍，于SPEED-Bench上使用SGLang在GB200 GPU上实现4倍更高吞吐。
原文摘要 · Abstract (English)
We introduce Nemotron-Labs-Diffusion, a tri-mode language model (LM) that unifies AR, diffusion, and self-speculation decoding within a single architecture. Trained with a joint AR-diffusion objective, Nemotron-Labs-Diffusion can switch modes to sustain high throughput across deployment settings and concurrency levels. Our study shows that (1) AR and diffusion objectives are complementary: diffusion improves lookahead planning, while AR provides left-to-right linguistic priors. (2) In self-speculation mode, diffusion drafts while AR verifies, outperforming multi-token prediction (MTP) methods in both acceptance rate and real-device efficiency. (3) A speed-of-light analysis further demonstrates diffusion's long-term potential, with up to 76.5% more tokens per forward pass than self-speculation under an optimal sampler. Scaling to 3B, 8B, and 14B parameters, our Nemotron-Labs-Diffusion family, including base, instruct, and vision-language models, consistently outperforms state-of-the-art open-source AR and diffusion LMs in both accuracy and speed. For example, Nemotron-Labs-Diffusion-8B decodes 6x more tokens per forward than Qwen3-8B with comparable accuracy, translating to 4x higher throughput on SPEED-Bench with SGLang on a GB200 GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。