无需预设前向过程,用自修正机制提升文本生成质量与速度。
Forward-Free Diffusion Language Models with BPTT-Free Looped Refinement

- 以生成草稿为隐式中间状态,通过递归精炼分布逼近目标分布。
- 在子80亿参数规模下,40亿模型比更大基线模型性能高15%,速度提升1.5-1.8倍。
- 不依赖特定邻域结构,适配多种采样策略,适合追求高效生成的场景。
扩散语言模型通过迭代去噪生成文本,提供不同于自回归生成的强大替代方案。然而,离散语言空间缺乏自然邻域结构,导致前向过程需人为设计扰动,常产生数学便利但与实际生成错误脱节的状态,影响样本质量。为此,我们提出FReDA,一种无需手写前向过程的扩散语言模型。将扩散建模视为递归分布精炼,模型生成的草稿作为隐式中间状态,学习的精炼模型逐步将草稿分布推向目标分布。训练时仅反向传播至最终一步,避免跨迭代的梯度回传(BPTT),同时保留最终Transformer层的标准反向传播。具体而言,FReDA通过提议候选序列,选择直接自精炼或并行候选中选最优(Best-of-N)的方式进行精炼。该设计使FReDA具备邻域无关性、采样自适应性和灵活精炼参数化能力。大规模评估显示,在子80亿参数范围内,FReDA-4B在推理与编码基准上超越更大规模的扩散基线模型,最高获得15%绝对提升,平均速度提升1.5-1.8倍,并能有效利用额外精炼计算资源。
原文摘要 · Abstract (English)
Diffusion language models generate text through iterative denoising, offering a powerful alternative to autoregressive generation. However, discrete language spaces lack a natural neighborhood structure for defining effective perturbations, motivating artificial corruption schemes in the forward process. Such prescribed forward processes often produce states that are mathematically convenient but misaligned with the drafts and errors encountered during generation, resulting in degraded sample quality. To address this limitation, we propose FReDA, a forward-free diffusion language model that eliminates the need for a hand-designed forward process. We formulate diffusion language modeling as recursive distribution refinement, in which model-generated drafts serve as implicit intermediate states and the learned refinement model progressively moves the draft distribution toward the target distribution. Training detaches the preceding refinement passes and backpropagates only through the final pass, avoiding backpropagation through time (BPTT) across refinement iterations while retaining standard backpropagation within the final Transformer pass. Concretely, FReDA refines drafts by proposing candidate sequences and either directly performing self-refinement or selecting among parallel candidates via Best-of-N refinement. With this design, FReDA is neighborhood-agnostic, sampler-adaptive, and compatible with flexible refinement parameterizations. Extensive evaluations in the sub-8B regime show that FReDA-4B outperforms larger diffusion base models on reasoning and coding benchmarks, achieving absolute gains of up to 15%, while reaching a 1.5-1.8x average speedup over diffusion baselines and scaling effectively with additional refinement computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。