提出新采样方法PARD，让块扩散语言模型更快更准地生成文本。
Rethinking the Generation Order of Block Diffusion Language Models
- 基于左到右解码特性设计并行采样方法
- 生成质量优于现有并行采样，速度远超纯自回归
- 无需训练，适合追求效率的生成任务
扩散语言模型支持灵活的任意顺序生成，但现有采样方法主要针对早期掩码扩散模型（MDMs）。本文研究近期块扩散语言模型（BDLMs）的采样策略，实证与分析表明，这些模型天然更契合从左到右的解码方式。基于此，我们提出无需训练的并行自回归解码（PARD）方法，保留从左到右的去遮蔽结构，同时允许并行提交词元。大量实验显示，PARD在生成质量上持续优于现有并行采样器，且相比纯自回归解码实现显著加速，仅伴随微小质量损失。
原文摘要 · Abstract (English)
Diffusion language models enable flexible arbitrary-order generation, but existing sampling methods are mostly designed for early masked diffusion models (MDMs). In this work, we study sampling for recent block diffusion language models (BDLMs). We show empirically and analytically that these models are naturally more aligned with left-to-right decoding than MDMs. Based on this observation, we propose Parallel Autoregressive Decoding (PARD), a simple training-free sampling method that preserves left-to-right unmasking structure while allowing parallel token commitment. Extensive experiments show that PARD consistently outperforms existing parallel samplers in generation quality, while achieving substantial speedups over pure AR decoding with only a small quality gap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。