提出基于粒子的扩散优化方法，解决梯度不可计算的问题。
Particle-based Generalised Stochastic Optimisation
- 用粒子系统模拟均场动力学，近似复杂分布下的优化过程
- 证明连续时间粒子系统具有指数收敛性和非渐近误差界
- 适用于生成模型训练、微调等梯度难求场景
我们提出一类基于扩散的随机粒子优化方法，用于处理损失函数梯度不可解析的情况。具体而言，考虑损失梯度为参数依赖分布的积分形式，这涵盖了生成模型训练、微调和隐变量模型学习等问题。引入均场动力学及其相互作用粒子近似，该框架包含多个现有算法作为特例，并为构造新方法提供路径。在适定性与联合压缩性假设下，证明了指数收敛性，并给出连续时间粒子系统的非渐近误差界。通过构建动量型与高阶Langevin变体，在最大边际似然估计和能量模型训练中进行了验证。
原文摘要 · Abstract (English)
We develop a class of diffusion-based stochastic particle optimisation methods for loss functions with intractable gradients. Specifically, we consider problems in which the loss gradient is an integral with respect to a parameter-dependent distribution, a structure that includes training generative models, fine-tuning, and learning latent-variable models. We introduce mean-field dynamics and its interacting-particle approximations, which contain several existing algorithms as special cases and provides a route to constructing new methods. Under well-posedness and joint contractivity assumptions, we prove exponential convergence and show that the continuous-time particle system admits a non-asymptotic error bound. We illustrate it by developing momentum and higher-order Langevin variants and evaluating them on maximum marginal-likelihood estimation and energy-based-model training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。