提出连续时间框架，为能量模型训练提供稳定误差保证。
Uniform-in-time convergence bounds for Persistent Contrastive Divergence Algorithms
- 将持久对比发散建模为耦合多尺度随机微分方程系统。
- 首次获得随时间不变的误差上界，确保长期训练稳定性。
- 基于S-ROCK积分器实现高效算法，适合高维能量模型训练。
我们提出了持久对比发散（PCD）在最大似然估计（MLE）未归一化密度时的连续时间形式。该方法将PCD表示为一组耦合的多尺度随机微分方程（SDEs），同时完成参数优化与参数化密度采样。基于此新框架，我们推导出PCD迭代值与真实MLE解之间的显式误差界。这一成果得益于对多尺度系统与平均化过程之间矩差的统一时间（UiT）上界分析。我们还引入了一种高效的连续时间方案，采用一类显式、稳定的随机正交龙格-库塔切比雪夫（S-ROCK）积分器，并给出了长时间范围内的明确误差估计。该方法为能量基模型（EBMs）训练提供了带有显式误差保证的新途径。
原文摘要 · Abstract (English)
We propose a continuous-time formulation of persistent contrastive divergence (PCD) for maximum likelihood estimation (MLE) of unnormalised densities. Our approach expresses PCD as a coupled, multiscale system of stochastic differential equations (SDEs), which perform optimisation of the parameter and sampling of the associated parametrised density, simultaneously. From this novel formulation, we are able to derive explicit bounds for the error between the PCD iterates and the MLE solution for the model parameter. This is made possible by deriving uniform-in-time (UiT) bounds for the difference in moments between the multiscale system and the averaged regime. An efficient implementation of the continuous-time scheme is introduced, leveraging a class of explicit, stable intregators, stochastic orthogonal Runge-Kutta Chebyshev (S-ROCK), for which we provide explicit error estimates in the long-time regime. This leads to a novel method for training energy-based models (EBMs) with explicit error guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。