提出协同提交机制，让扩散语言模型一次生成多个相关词时更准确。
Don't Commit Alone: Joint Token Commitment in Diffusion Language Models
- 用标记门控延迟提交，让模型在写入前协调多个位置的词选择。
- 在七个任务中五项表现超越原版，代码与推理类任务提升最明显。
- 仅需一次部分前向传播，无需额外模型，可直接集成现有架构。
扩散语言模型（dLLMs）在每轮去噪中独立解码多个位置的词，当这些位置存在依赖关系时，这种独立性会引入由条件总相关性捕获的误差，仅靠置信度选择无法察觉。本文提出CoCommit，一种标记门控的协调机制：在常规批量选择后，通过学习到的标记确定提交集合，并对主干网络最后n层进行重用，以协调标记位置后再用贪婪策略写入词元。该方法近似联合模式解码，复用已有权重，仅需一次局部前向传播且无需辅助模型。在使用LoRA适配器的LLaDA 2.1上，采用贪婪推理时，联合提交在七项评估任务中提升了五项，最大增益出现在代码和推理任务，其余任务表现接近原版。
原文摘要 · Abstract (English)
Diffusion language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from a shared context. When these positions are dependent, this factorization introduces an error captured by conditional total correlation, which confidence-based selection cannot infer from marginal probabilities alone. We propose CoCommit, a marker-gated coordination pass that delays commitment. After the usual bundle selection, a learned marker identifies the commit set, and the backbone's last n layers are re-applied to coordinate the marked positions before greedy argmax writes the tokens. This approximates joint-mode decoding while reusing existing weights, requiring only one partial forward pass and no auxiliary model. On LLaDA 2.1 with LoRA adapters and greedy inference, joint commitment improves five of the seven evaluated benchmarks over the released factorized decoder. The largest gains occur on code and reasoning tasks, while the remaining tasks are near parity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。