arXiv:2606.19475cs.AIcs.CL2026-06

对比8种扩散语言模型,揭示生成质量与效率的权衡

Diffusion Language Models: An Experimental Analysis

  • 通过系统实验评估8种扩散语言模型在8个任务上的表现
  • 发现生成步数、上下文长度等设计直接影响性能与效率
  • 适合关注生成模型效率与部署成本的研究者和工程师

大型语言模型(LLMs)通过自回归生成实现了卓越性能。最近,扩散语言模型(DLMs)作为一种新范式出现,通过迭代去噪而非逐词预测生成文本,支持整个序列的并行优化。尽管已有多种基于扩散的架构提出,但评估协议、数据集、推理预算和生成超参数的差异使得难以公平比较其能力。本文对8种前沿DLMs进行了系统性实验分析,覆盖推理、编程、翻译、知识和结构化问题求解等8个基准任务,同时考量生成质量与计算效率。除了下游评估,还分析了关键推理因素的影响:去噪步数、上下文长度、块大小和并行掩码策略,并通过小规模模型在相同条件下的控制实验补充大规模测试。结果表明,DLM的行为强烈受生成时设计选择影响,在不同任务、架构和推理预算下表现出显著的性能-效率权衡。本研究为当代DLMs的能力与部署特性提供了实用洞见。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have revolutionized language modeling through autoregressive generation, enabling strong performance across a wide range of tasks. Recently, Diffusion Language Models (DLMs) have emerged as an alternative paradigm that generates text through iterative denoising rather than next-token prediction, allowing parallel refinement of entire sequences. While numerous diffusion-based architectures have been proposed, differences in evaluation protocols, datasets, inference budgets, and generation hyperparameters make it difficult to compare their capabilities and understand the trade-offs they offer. In this work, we present a systematic experimental analysis of modern DLMs. Specifically, we evaluate eight state-of-the-art DLMs across eight benchmarks spanning reasoning, coding, translation, knowledge, and structured problem solving, while explicitly considering both generation quality and computational efficiency. Beyond downstream evaluation, we analyze the impact of key inference-time factors, including denoising steps, context length, block size, and parallel unmasking strategies, and complement large-scale experiments with controlled comparisons of smaller models trained under identical conditions. Our analysis highlights the strengths and limitations of diffusion-based language modeling across different tasks, architectures, and inference budgets. We show that the behavior of DLMs is strongly influenced by generation-time design choices, leading to distinct trade-offs between performance and computational efficiency. Overall, our study provides practical insights into the capabilities and deployment characteristics of contemporary DLMs.

扩散模型语言建模效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。