解析扩散模型学习过程，揭示数据量与生成质量关系。
A solvable model of learning generative diffusion: theory and insights
- 用双层自编码器建模扩散生成，分析在线梯度下降训练
- 证明样本分布投影受训练样本数影响，理论可预测偏差
- 解释模式崩溃机制，适合研究生成模型稳定性的学者
本文研究基于双层自编码器参数化的流或扩散生成模型，在高维目标分布具有低维流形结构的条件下，使用在线随机梯度下降进行训练的问题。我们推导出由学习模型生成样本分布的低维投影的紧致渐近表征，特别揭示了其对训练样本数量的依赖性。基于该分析，我们讨论了模式崩溃如何产生，并在生成数据上重新训练时导致模型崩溃。
原文摘要 · Abstract (English)
In this manuscript, we consider the problem of learning a flow or diffusion-based generative model parametrized by a two-layer auto-encoder, trained with online stochastic gradient descent, on a high-dimensional target density with an underlying low-dimensional manifold structure. We derive a tight asymptotic characterization of low-dimensional projections of the distribution of samples generated by the learned model, ascertaining in particular its dependence on the number of training samples. Building on this analysis, we discuss how mode collapse can arise, and lead to model collapse when the generative model is re-trained on generated synthetic data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。