解决扩散语言模型预训练与生成不匹配问题,提升续写准确率。
Reducing Pretraining-Generation Mismatch in Diffusion Language Models

- 设计新预训练目标PCD,分治前缀自回归与后缀去噪。
- 在LLaDA2-Mini上提升4.2%(+2.56点),Qwen上提升14.2%(+4.86点)。
- 无需修改推理方式,适合想改进扩散语言模型的开发者。
自回归语言模型在训练和使用时对齐:生成基于干净提示,训练从干净上下文预测未来词元。扩散语言模型(dLLM)支持并行去噪,但原生预训练会随机污染提示与延续部分,削弱了提示条件生成所需的干净前缀接口。本文识别出这一提示延续中的不匹配问题,提出前缀条件扩散(PCD)预训练目标,结合自回归前缀监督与无偏移后缀去噪。在训练目标层面,PCD调整注意力掩码、污染掩码和标签构建;不需自回归解码器、验证器或新推理模式。通过自回归监督干净前缀,仅对未知延续应用扩散,使局部训练接口更接近块扩散模型的评估查询方式。进一步分离样本内前缀条件与样本间目标混合,可独立识别局部对齐信号与可选的批量混合控制。在LLaDA2-Mini与Qwen-1.7B模型上,PCD一致优于同族原生稳定基线,在主任务六基准平均上取得4.2%相对提升(+2.56点),在关键机制对比中达14.2%相对提升(+4.86点)。结果表明,将预训练上下文分布与提示条件生成对齐,可在不改变推理的前提下恢复扩散模型续写能力的部分差距。
原文摘要 · Abstract (English)
Autoregressive language models align training and use: generation conditions on a clean prompt, and training predicts future tokens from clean left context. Diffusion language models offer parallel denoising, but native dLLM pretraining can randomly corrupt prompt and continuation tokens together, weakening the clean-prefix interface needed for prompt-conditioned generation. We identify this mismatch for prompt continuation and propose PCD (Prefix-Conditioned Diffusion), a pretraining objective that combines AR prefix supervision with no-shift suffix denoising. At the training-objective level, PCD changes the attention mask, corruption mask, and label construction in continued pretraining; it does not require an autoregressive decoder, verifier, or new inference mode. By supervising the clean-prefix side autoregressively and applying diffusion only to the unknown continuation, PCD makes the local training interface resemble how block-diffusion models are queried at evaluation time. We further separate intra-sample prefix conditioning from inter-sample objective mixing, allowing us to identify the local alignment signal separately from the optional batch-level mixing knob. Across LLaDA2-Mini and Qwen-1.7B backbones, PCD consistently improves over same-family native dLLM stable baselines, reaching a 4.2% relative gain on the main LLaDA2-Mini six-benchmark average (+2.56 points) and a 14.2% relative gain in the primary Qwen mechanism comparison (+4.86 points). These results suggest that aligning the pretraining context distribution with prompt-conditioned generation can recover a measurable part of the dLLM continuation gap without changing inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。