用注意力机制指导去噪顺序，提升扩散语言模型的推理能力
Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models
- 根据注意力结构决定去噪顺序，更贴合语义依赖
- 关键注意力令牌在微调中受重点优化，推理更稳定
- 数学与编码任务上超越现有最佳方法
扩散大语言模型（dLLMs）通过并行解码提供高效替代自回归模型的方案，但现有后训练方法多依赖随机掩码策略，忽视了内在的词元依赖关系。本文对dLLMs中的注意力机制进行实证分析，发现更强烈关注未掩码上下文的词元具有更高生成稳定性，并在推理中起关键作用。基于此，提出注意力引导的去噪与优化框架AGDO，使训练与优化过程均与注意力衍生的依赖关系对齐。AGDO依据注意力结构确定去噪顺序，并在监督微调和强化学习中强化注意力关键词元。在数学与编码基准测试中，AGDO持续提升推理性能，优于当前最优的dLLM后训练方法。
原文摘要 · Abstract (English)
Diffusion large language models (dLLMs) offer an efficient alternative to autoregressive models through parallel decoding, yet existing post-training methods largely rely on random masking strategies that overlook intrinsic token dependencies. In this work, we present an empirical analysis of attention in dLLMs and show that tokens attending more strongly to unmasked context exhibit greater generation stability and play a critical role in reasoning. Motivated by these findings, we propose AGDO, an attention-guided denoising and optimization framework that aligns both training and optimization with attention-derived dependencies. AGDO determines the denoising order based on attention structure and emphasizes attention-critical tokens during supervised fine-tuning and reinforcement learning. Experiments on mathematical and coding benchmarks demonstrate that AGDO consistently improves reasoning performance, outperforming state-of-the-art post-training methods for dLLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。