研究扩散模型在高维流形数据上的动态演化，揭示了生成过程中的关键相变现象。
Analysis of Diffusion Models for Manifold Data
- 基于低维高斯混合模型构建可解析的流形数据生成框架
- 推导出物种分化与坍缩的临界时间，依赖于流形与环境空间的维度比
- 利用广义线性模型的精确互信息公式实现理论分析，适合理论研究者
我们分析了生成式扩散模型的时间反演动力学。在高维且样本数量呈指数级增长的条件下，若使用精确的经验得分函数，这些模型会经历不同动力学阶段之间的相变。本文扩展了该分析，针对一个可解析的流形模型——即数据统计模型为嵌入高维空间的低维高斯混合模型——计算了相变点。我们推导出所谓的物种分化与坍缩相变时间，作为流形维度与环境空间维度之比及其他数据模型特征的函数。分析中一个关键工具是广义线性模型的精确互信息（或自由能）公式。
原文摘要 · Abstract (English)
We analyze the time reversed dynamics of generative diffusion models. If the exact empirical score function is used in a regime of large dimension and exponentially large number of samples, these models are known to undergo transitions between distinct dynamical regimes. We extend this analysis and compute the transitions for an analytically tractable manifold model where the statistical model for the data is a mixture of lower dimensional Gaussians embedded in higher dimensional space. We compute the so-called speciation and collapse transition times, as a function of the ratio of manifold-to-ambient space dimensions, and other characteristics of the data model. An important tool used in our analysis is the exact formula for the mutual information (or free energy) of Generalized Linear Models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。