用噪声水平控制生成粒度，让扩散语言模型更早规划结构。
When to Plan, When to Polish: Noise Level as a Granularity Axis for Diffusion Language Models
- 以噪声水平为线索，动态调整生成粒度，高噪时用词组，低噪时用单字。
- 在写作任务中提前形成语义骨架，恢复顺序更优，输出更健康。
- 无需额外规划器或分阶段设计，统一架构实现粗细协同生成。
标准的逐标记扩散语言模型在整个去噪过程中始终以标记粒度进行训练和推理。在高噪声阶段，这种机制导致零散的局部片段，难以形成早期连贯的语义结构，而这正是依赖规划的生成所必需的。现有层次化规划方法虽能分离规划与表述，但需额外规划器、块潜变量或两阶段设计。本文提出噪声依赖粒度控制（NDGC），一种单级扩散方法，利用噪声水平作为粒度指示信号。NDGC使训练暴露与推理承诺随去噪进程动态对齐：高噪声步骤使用连贯的词组支持早期意义确定，低噪声步骤则回归到标记级精细优化。该机制实现了无需显式规划器或分层结构的类规划粗到细去噪。在受控测试、消融实验及WritingPrompts数据集上，NDGC均表现出更早的骨架形成、更有序的恢复过程以及更健康的输出质量。
原文摘要 · Abstract (English)
Standard tokenwise diffusion LMs keep training corruption and inference commitment at token granularity throughout denoising. At high noise, this leaves scattered local fragments rather than coherent evidence, making it hard to form early coarse structure, exactly what planning-sensitive generation requires. Hierarchical planning methods add coarse stages to separate planning from wording, but they need extra planners, block latents, or two stage designs. We propose Noise Dependent Granularity Control (NDGC), a single-level diffusion method that uses the noise level as a granularity cue. NDGC aligns training exposure and inference commitment with denoising progress. High noise steps use coherent token groups to support early meaning commitment, while low noise steps return to token level refinement. This creates planning like coarse to fine denoising without an explicit planner or hierarchical architecture. Across controlled tests, ablations, and WritingPrompts, NDGC shows earlier skeleton formation, better ordered recovery, and healthier outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。