arXiv:2605.12597cond-mat.dis-nncond-mat.stat-mech2026-05被引 1

通过分析扩散模型在临界点附近的采样难题,发现网络深度可显著加速训练。

The critical slowing down in diffusion models

论文配图:The critical slowing down in diffusion models
图 1 · 摘自论文原文
  • 用单层网络训练时出现临界慢化,参数学习变慢
  • 双层网络使训练时间从二次增长变为对数增长
  • 局部得分近似在不增加参数量下实现加速,适合物理模拟

计算采样自20世纪中叶以来一直是科学的核心。尽管基于机器学习的方法近期取得重大进展,但其行为仍缺乏理论理解,难以控制何时何地成功。本文针对扩散模型——一种实践中极为有效的生成方法——在统计场论的$O(n)$模型($n \to \infty$高斯极限)中的应用提供了理论洞察。在该解析可处理的设定下,我们发现:使用与精确解匹配的一层网络训练得分模型时,参数学习出现临界慢化现象,且这一慢化也影响生成过程,表明即使在学习到的生成模型中,临界附近采样困难依然存在。为克服此瓶颈,我们证明了结合架构深度与物理局域性具有强大效果:采用两层架构可大幅减少临界慢化,训练时间随系统规模呈对数而非二次增长。通过引入局部得分近似,我们进一步证明这种加速可在不增加神经网络参数量的前提下实现。这些结果表明,通过合理架构设计,扩散模型可克服临界慢化,并为理解与改进统计物理等领域的学习采样方法建立了可控框架。

原文摘要 · Abstract (English)

Computational sampling has been central to the sciences since the mid-20th century. While machine-learning-based approaches have recently enabled major advances, their behavior remains poorly understood, with limited theoretical control over when and why they succeed. Here we provide such insight for diffusion models-a class of generative schemes highly effective in practice-by analyzing their application to the $O(n)$ model of statistical field theory in the Gaussian limit $n \to \infty$. In this analytically tractable setting, we show that training a score model with a one-layer network architecture matching the exact solution exhibits a form of critical slowing down in parameter learning. This slowing down also impacts the generation process, indicating that the well-known difficulties of sampling near criticality persist even for learned generative models. To overcome this bottleneck, we demonstrate the power of combining architectural depth with physical locality. We find that using a two-layer architecture drastically reduces the critical slowing down, with the training time scaling logarithmically rather than quadratically with system size. By introducing a local score approximation we show that this acceleration in training time can be achieved without increasing the number of neural network parameters. Taken together, these results demonstrate that diffusion models can overcome the critical slowing down through appropriate architectural design, and establish a controlled framework for understanding and improving learned sampling methods in statistical physics and beyond.

扩散模型临界慢化生成模型物理模拟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。