arXiv:2506.21741stat.MLcs.LG2025-06被引 3

提出高阶朗之万动力学的临界阻尼方法,提升生成模型效率与稳定性。

Critically-Damped Higher-Order Langevin Dynamics for Generative Modeling

  • 引入系统分析中的临界阻尼概念,统一优化高阶扩散过程
  • 在CIFAR-10和CelebA-HQ上实现比基准更优的生成质量(FID更低)
  • 提供前向过程均值与协方差闭式解,简化实现且无需调参

去噪扩散概率模型(DDPMs)是生成AI中一类全新方法,基于随机微分方程描述数据到噪声的前向过程与噪声到数据的反向过程。现有方法常引入辅助变量,如速度、加速度等,将数据视为位置变量。本文将临界阻尼思想从二阶与三阶朗之万动力学推广至任意阶次,提出临界阻尼高阶朗之万动力学(Critically-Damped HOLD)。该方法在n维情形下给出最优超参数设置,取代原HOLD需人工指定的问题。同时,推导出前向过程均值与协方差的闭式表达,极大简化实现。在CIFAR-10与CelebA-HQ 256×256数据集上通过FID指标验证,生成效果优于基线。

原文摘要 · Abstract (English)

Denoising diffusion probabilistic models (DDPMs) represent an entirely new class of generative AI methods that have yet to be fully explored. They use Langevin dynamics, represented as stochastic differential equations, to describe a process that transforms data into noise, the forward process, and a process that transforms noise into generated data, the reverse process. Many of these methods utilize auxiliary variables that formulate the data as a ``position" variable, and the auxiliary variables are referred to as ``velocity", ``acceleration", etc. In this sense, it is possible to ``critically damp" the dynamics. Critical damping has been successfully introduced in Critically-Damped Langevin Dynamics (CLD) and Critically-Damped Third-Order Langevin Dynamics (TOLD++), but has not yet been applied to dynamics of arbitrary order. The proposed methodology generalizes Higher-Order Langevin Dynamics (HOLD), a recent state-of-the-art diffusion method, by introducing the concept of critical damping from systems analysis. Similarly to TOLD++, this work proposes an optimal set of hyperparameters in the $n$-dimensional case, where HOLD leaves these to be user defined. Additionally, this work provides closed-form solutions for the mean and covariance of the forward process that greatly simplify its implementation. Experiments are performed on the CIFAR-10 and CelebA-HQ $256 \times 256$ datasets, and validated against the FID metric.

生成模型扩散模型朗之万动力学优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。