提出自适应最优扩散调度算法，提升采样质量且无需调参。
Score-Optimal Diffusion Schedules
- 基于估计的Stein梯度动态优化扩散步长调度
- 在图像数据集上达到媲美人工设计的FID分数
- 可直接部署于预训练模型，推理时在线调整
去噪扩散模型（DDMs）提供了一种灵活的高维数据分布采样框架。通过逐步向数据注入噪声，构建从参考高斯分布到数据分布的概率路径。为数值模拟采样过程，需选择从参考分布回溯至干净数据的离散化调度。合适的调度对生成高质量样本至关重要。然而，除手工启发式方法外，尚无通用调度选择方法。本文提出一种新算法，可自适应地选择最优离散化调度，该调度以我们推导出的代价函数为依据。该代价衡量模拟过程从扩散路径一节点移动至下一节点所需的工作量。本方法无需超参数调优，能适应扩散路径的动力学与几何特性。算法仅需评估估计的Stein梯度，因此可扩展至现有预训练模型，在推理时或训练中在线应用。实验表明，所学调度能恢复此前仅通过手动搜索发现的高性能调度，并在图像数据集上获得具有竞争力的FID得分。
原文摘要 · Abstract (English)
Denoising diffusion models (DDMs) offer a flexible framework for sampling from high dimensional data distributions. DDMs generate a path of probability distributions interpolating between a reference Gaussian distribution and a data distribution by incrementally injecting noise into the data. To numerically simulate the sampling process, a discretisation schedule from the reference back towards clean data must be chosen. An appropriate discretisation schedule is crucial to obtain high quality samples. However, beyond hand crafted heuristics, a general method for choosing this schedule remains elusive. This paper presents a novel algorithm for adaptively selecting an optimal discretisation schedule with respect to a cost that we derive. Our cost measures the work done by the simulation procedure to transport samples from one point in the diffusion path to the next. Our method does not require hyperparameter tuning and adapts to the dynamics and geometry of the diffusion path. Our algorithm only involves the evaluation of the estimated Stein score, making it scalable to existing pre-trained models at inference time and online during training. We find that our learned schedule recovers performant schedules previously only discovered through manual search and obtains competitive FID scores on image datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。