提出无需几何假设的扩散模型收敛分析与自适应采样调度方法
Entropy-Based Dimension-Free Convergence and Loss-Adaptive Schedules for Diffusion Models
- 基于信息论构建无维度依赖的收敛性分析框架
- 理论证明生成分布与目标分布间KL散度为O(H²/K)，H为熵，K为采样步数
- 设计仅依赖训练损失的轻量级自适应调度，提升采样质量
扩散生成模型通过离散化由学习到的得分函数（或去噪器）驱动的反向时间动态来生成样本。现有扩散模型的收敛性分析通常至少随环境维度线性增长，而更紧的速率往往依赖于目标分布的内在维度假设或其他几何限制。本文提出一种替代性的、基于信息论的无维度依赖收敛性分析方法，避免了任何几何假设。在对目标分布的温和假设下，我们证明了生成分布与目标分布之间的KL散度上界为O(H²/K)（含端点因子），其中H为香农熵，K为采样步数。此外，通过重新表述KL散度，我们提出一种仅依赖训练损失的轻量级损失自适应调度（LAS），用于高效离散化反向SDE，无需后训练的大量计算。实验表明，LAS在采样质量上优于常见启发式调度。
原文摘要 · Abstract (English)
Diffusion generative models synthesize samples by discretizing reverse-time dynamics driven by a learned score (or denoiser). Existing convergence analyses of diffusion models typically scale at least linearly with the ambient dimension, and sharper rates often depend on intrinsic-dimension assumptions or other geometric restrictions on the target distribution. We develop an alternative, information-theoretic approach to dimension-free convergence that avoids any geometric assumptions. Under mild assumptions on the target distribution, we bound KL divergence between the target and generated distributions by $O(H^2/K)$ (up to endpoint factors), where $H$ is the Shannon entropy and $K$ is the number of sampling steps. Moreover, using a reformulation of the KL divergence, we propose a Loss-Adaptive Schedule (LAS) for efficient discretization of reverse SDE which is lightweight and relies only on the training loss, requiring no post-training heavy computation. Empirically, LAS improves sampling quality over common heuristic schedules.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。