揭示扩散模型如何学习非高斯相关性，解析高阶累积量的演化机制。
On learning higher-order cumulants in diffusion models
- 通过推导生成函数，分析前向与反向过程中高阶累积量的行为。
- 无漂移模型中高阶累积量在前向过程保持不变，终点仍存非平凡相关性。
- 证明即使从标准先验开始，反向过程也能学习这些高阶统计特性。
为分析扩散模型如何学习超出高斯分布的相关性，本文研究了高阶累积量（或关联n点函数）在前向与反向过程中的行为。推导出基于初始数据分布和前向过程特性的矩生成泛函与累积量生成泛函的显式表达式。理论证明：在无漂移的模型（如方差扩展方案）中，前向过程中高阶累积量保持不变，因此前向过程终点仍保留非平凡相关性。进一步表明，由于这些相关性编码于得分函数中，反向过程能够学习高阶累积量，即便从标准正态先验开始。通过一个具有非零累积量的可精确求解的玩具模型以及标量格点场论验证了上述结论。
原文摘要 · Abstract (English)
To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the distribution of the initial data and properties of forward process. It is shown analytically that during the forward process higher-order cumulants are conserved in models without a drift, such as the variance-expanding scheme, and that therefore the endpoint of the forward process maintains nontrivial correlations. We demonstrate that since these correlations are encoded in the score function, higher-order cumulants are learnt in the backward process, also when starting from a normal prior. We confirm our analytical results in an exactly solvable toy model with nonzero cumulants and in scalar lattice field theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。