arXiv:2602.18647cs.LGcs.AI2026-02被引 2

根据信息量动态调整噪声分布,让扩散模型更高效学习。

Noise Scheduling as Information-Guided Allocation in Diffusion Training

  • 用训练中去噪损失估算条件熵率,自适应调整噪声分配
  • 图像任务上效果持平或超越基线,减少训练轮次
  • 在基因、语言等新场景中提速三倍,适合资源受限研究

我们提出InfoNoise,一种扩散训练中在线自适应的噪声调度方法,将优化精力集中在最具信息量的噪声水平上。结合损失加权,噪声调度决定了去噪问题的有效分配,但传统方法需提前设定,难以预知有效噪声范围。InfoNoise通过训练过程中的去噪损失估计条件熵率曲线,无需辅助模型或离线搜索,实现数据驱动的自适应调整。基于I–MMSE理论,该曲线识别出噪声观测快速降低对干净样本不确定性的重要区间,并指导训练噪声分布的更新。仅改变噪声分布,保持目标函数、加权方式和参数化不变。在图像基准测试中,尽管调度已高度调优,InfoNoise仍可达到或略超强基线,且以更少更新次数达成同等质量。在表示学习、序列与模态迁移任务(包括DNA与语言生成)中,相比固定及自适应基线均有提升,最高可减少3倍训练算力。结果表明,条件熵率曲线是噪声调度设计的数据依赖目标,使在线自适应成为手动调参的实用替代方案。

原文摘要 · Abstract (English)

We introduce InfoNoise, an online adaptive noise schedule for diffusion training that reallocates optimization effort toward noise levels where denoising is most informative. Together with loss weighting, a noise schedule induces an effective allocation across denoising problems, often fixed before informative noise levels are known. InfoNoise makes this allocation data-adaptive by estimating a conditional-entropy-rate profile from denoising losses during training, without auxiliary models or offline search. Through I--MMSE, this profile identifies where noisy observations rapidly reduce uncertainty about the clean sample and guides adaptation of the training noise distribution. It changes only this distribution, keeping the objective, weighting, and parameterization fixed. On image benchmarks, where schedules have been extensively tuned, InfoNoise matches or slightly exceeds strong baselines and can reach the same quality with fewer updates. On representation, sequence, and modality shifts, including DNA and language generation, InfoNoise improves over fixed and adaptive baselines and reaches target quality with up to $3\times$ less training compute. These results establish the conditional-entropy-rate profile as the data-dependent target for noise schedule design and make online adaptation a practical alternative to manual schedule search.

扩散模型噪声调度自适应训练高效学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。