揭示扩散模型生成过程中的信息动态机制。
The Information Dynamics of Generative Diffusion
- 通过信息论视角分析生成过程中的熵变化速率。
- 发现得分函数的发散决定信息流动速度与轨迹分支。
- 适用于理解扩散模型内部机制的研究者。
生成式扩散模型在机器学习中表现强大,但其理论理解仍不完整。本文从信息论、动力学与热力学角度统一分析生成扩散过程。研究发现,生成过程中条件熵的产生速率(即生成带宽)直接由得分函数向量场的期望发散决定。该发散关联轨迹分支与生成分叉,被定义为能量景观中的对称性破缺相变。超越平均统计,路径条件熵的方差峰值揭示了个体轨迹在不确定性消解中的异质性。结果表明,生成扩散是受控的、由噪声诱导的对称性破缺过程,得分函数作为动态非线性滤波器,调控信息从噪声到数据的流动速率与波动性。
原文摘要 · Abstract (English)
Generative diffusion models have emerged as a powerful class of models in machine learning, yet a unified theoretical understanding of their operation is still developing. This paper provides an integrated perspective on generative diffusion by connecting the information-theoretic, dynamical, and thermodynamic aspects. We demonstrate that the rate of conditional entropy production during generation (i.e., the generative bandwidth) is directly governed by the expected divergence of the score function's vector field. This divergence, in turn, is linked to the branching of trajectories and generative bifurcations, which we characterize as symmetry-breaking phase transitions in the energy landscape. Beyond ensemble averages, we demonstrate that symmetry-breaking decisions are revealed by peaks in the variance of pathwise conditional entropy, capturing heterogeneity in how individual trajectories resolve uncertainty. Together, these results establish generative diffusion as a process of controlled, noise-induced symmetry breaking, in which the score function acts as a dynamic nonlinear filter that regulates both the rate and variability of information flow from noise to data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。