arXiv:2601.22947cs.CLcs.LG2026-01

发现掩码扩散语言模型对位置偏移敏感,提出用新方法缓解此问题。

Reconsidering Positional Supervision in Masked Diffusion Language Model Training

  • 引入<slack>令牌和更新的合并映射,使损失函数容忍小位置偏移
  • 在四个生成任务中,新方法显著优于原模型和交叉熵基线
  • 适合关注非自回归生成鲁棒性的研究者与开发者

掩码扩散语言模型(MDLMs)通过并行解码生成文本,其训练方式与非自回归翻译类似,采用逐位置交叉熵(CE)损失。已有研究表明,此类模型对微小的位置偏移敏感,因CE损失会严厉惩罚错位。本文通过干预实验,在LLaDA-8B-Instruct与Arena-Hard数据集上发现,仅1%的生成词元发生一位置偏移,即可显著降低胜率。为此,受连接时序分类(CTC)启发,提出改进的CTC目标:引入特殊<slack>令牌吸收目标与输出间的位置不确定性,并使用更新的合并映射保留目标表面形式。在四个开放式生成基准上,该方法均取得统计显著提升,且优于原始模型与匹配的交叉熵基线。结果表明,训练阶段的对齐灵活性是改进MDLM监督微调的重要设计维度。

原文摘要 · Abstract (English)

Masked diffusion language models (MDLMs) generate text by unmasking tokens in parallel and have recently emerged as alternatives to autoregressive language models. They can be viewed as parallel decoders trained with a position-wise cross-entropy (CE) loss, the same setup as non-autoregressive translation (NAT). In NAT, CE-trained parallel decoders have been argued to be sensitive to small positional shifts, since CE penalizes them harshly. We ask whether CE-trained MDLMs are similarly sensitive to such shifts under iterative decoding. To probe this, we apply a controlled intervention that introduces them during decoding. On LLaDA-8B-Instruct with Arena-Hard, displacing as little as 1% of generated tokens by one position substantially reduces win rates against the unintervened model, showing that MDLMs are sensitive to such small shifts under iterative parallel decoding. Motivated by this, we adapt connectionist temporal classification (CTC), an alignment-flexible objective known to mitigate it there, to MDLM supervised fine-tuning. By relaxing the strict position-wise match that CE imposes, CTC gives the loss room to absorb small positional shifts; concretely, we modified CTC objective to use a special <slack> token that absorbs positional uncertainty between target tokens and output positions, and a updated collapse map that preserves target surface forms. Across four open-ended generation benchmarks, the resulting model consistently improves over both the original model and a matched cross-entropy-trained baseline, with statistically significant gains on all four. These results identify training-side alignment flexibility as a useful design dimension for MDLM SFT, complementary to the inference-time approaches explored in prior work.

扩散模型非自回归位置对齐训练优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。