提出可解析求解的单步扩散模型，揭示生成质量与数据维度的关系。
A solvable generative model with a linear, one-step denoiser
- 基于线性去噪器构建可解析的单步扩散模型。
- 发现当数据量超过数据维度时，KL散度开始单调下降。
- 解释了大规模模型为何多步扩散能提升生成质量。
我们基于线性去噪器构建了一个可解析处理的单步扩散模型，并给出了生成分布与采样分布（假设为各向同性高斯）之间Kullback-Leibler散度的显式公式，揭示了有限扩散时间和噪声尺度的影响。研究进一步表明，当训练数据集规模达到数据点维度时，KL散度的单调下降阶段开始出现。对于大规模实际扩散模型，我们基于前述理论论证，解释了为何增加扩散步数能提升生成质量。
原文摘要 · Abstract (English)
We develop an analytically tractable single-step diffusion model based on a linear denoiser and present an explicit formula for the Kullback-Leibler divergence between the generated and sampling distribution, taken to be isotropic Gaussian, showing the effect of finite diffusion time and noise scale. Our study further reveals that the monotonic fall phase of Kullback-Leibler divergence begins when the training dataset size reaches the dimension of the data points. Finally, for large-scale practical diffusion models, we explain why a higher number of diffusion steps enhances production quality based on the theoretical arguments presented before.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。