让扩散语言模型的训练更匹配生成方式，提升文本质量。
Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding
- 按固定块划分输出，每步只随机遮蔽一个块
- 实验显示在同等算力下，准确率显著提升
- 适合研究扩散模型与自回归生成对齐的学者
离散扩散语言模型在文本生成中展现出强大潜力，但标准监督微调（SFT）与其半自回归推理过程存在不匹配：训练时随机遮蔽整个响应的所有标记，而推理则按固定大小分块顺序生成。这种不一致导致前缀噪声和后缀泄露，使梯度偏离期望的分块似然。我们提出分块SFT（Blockwise SFT），将响应划分为固定大小的块，每步仅选择一个活跃块进行随机遮蔽，冻结所有先前标记，并完全隐藏未来标记。损失仅在活跃块上计算，直接对应分块解码流程。在GSM8K、MATH和MetaMathQA上的实验表明，在相同计算量或标记预算下，该方法持续优于传统SFT。分块大小一致性研究与消融实验确认，性能提升源于训练与推理的精准对齐，而非偶然遮蔽效应。结果强调了在扩散语言模型中，监督粒度应与解码过程相匹配的重要性。
原文摘要 · Abstract (English)
Discrete diffusion language models have shown strong potential for text generation, yet standard supervised fine-tuning (SFT) misaligns with their semi-autoregressive inference: training randomly masks tokens across the entire response, while inference generates fixed-size blocks sequentially. This mismatch introduces noisy prefixes and leaky suffixes, biasing gradients away from the desired blockwise likelihood. We propose Blockwise SFT, which partitions responses into fixed-size blocks, selects one active block per step for stochastic masking, freezes all preceding tokens, and fully hides future ones. Loss is computed only over the active block, directly mirroring the blockwise decoding process. Experiments on GSM8K, MATH, and MetaMathQA show consistent gains over classical SFT under equal compute or token budgets. Block size consistency studies and ablations confirm that improvements stem from faithful training-inference alignment rather than incidental masking effects. Our results highlight the importance of matching supervision granularity to the decoding procedure in diffusion-based language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。