无需训练，扩散语言模型可自动找到最优补全长度
Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitly
- 利用首步去噪置信度中的信号，动态推断最佳补全长度
- 代码补全任务中效果提升最高达47.7%，文本补全指标增8.5%~9.9%
- 适合希望提升生成质量又不想额外训练的开发者使用
扩散语言模型（DLM）天然适合补全任务，但其性能受限于预设的补全长度。本文揭示，DLM具备发现正确补全长度的内在能力。我们识别出首步去噪置信度中的两个关键统计现象：在真实长度附近出现局部‘最优峰值’，以及常掩盖该信号的‘长度偏差’。通过利用这一信号并校准偏差，我们的无训练方法CAL（校准自适应长度）可在正式解码前高效搜索最优长度。实证结果表明，CAL在代码补全中相比固定长度基线提升Pass@1高达47.7%，相比基于聊天的自适应方法提升40.5%；在文本补全中BLEU-2和ROUGE-L分别提升最多8.5%和9.9%。这些结果证明，CAL为无需专门训练的鲁棒DLM补全提供了新路径。代码已开源。
原文摘要 · Abstract (English)
Diffusion language models (DLMs) provide a bidirectional generation framework naturally suited for infilling, yet their performance is constrained by the pre-specified infilling length. In this paper, we reveal that DLMs possess an inherent ability to discover the correct infilling length. We identify two key statistical phenomena in the first-step denoising confidence: a local \textit{Oracle Peak} that emerges near the ground-truth length and a systematic \textit{Length Bias} that often obscures this signal. By leveraging this signal and calibrating the bias, our training-free method \textbf{CAL} (\textbf{C}alibrated \textbf{A}daptive \textbf{L}ength) enables DLMs to approximate the optimal length through an efficient search before formal decoding. Empirical evaluations demonstrate that CAL improves Pass@1 by up to 47.7\% over fixed-length baselines and 40.5\% over chat-based adaptive methods in code infilling, while boosting BLEU-2 and ROUGE-L by up to 8.5\% and 9.9\% in text infilling. These results demonstrate that CAL paves the way for robust DLM infilling without requiring any specialized training. Code is available at https://github.com/NiuHechang/Calibrated_Adaptive_Length.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。