arXiv:2504.18911stat.COcs.LG2025-04被引 5

用Adam思想改进Langevin采样步长,提升稳定性与收敛速度。

A Langevin sampling algorithm inspired by the Adam optimizer

  • 引入辅助变量实现动态步长调整,基于梯度范数自适应控制。
  • 在尼尔漏斗和MNIST分类任务中,精度与稳定性均优于固定步长方法。
  • 无需修改漂移项,可无缝接入现有Langevin积分器,适合贝叶斯推断场景。

我们提出一种基于时间重参数化Langevin动力学的自适应步长MCMC采样框架,通过额外自由度驱动步长变化。该方法在相空间中增加一个辅助变量,定义时间重参数化,并利用辅助松弛方程积累局部监测函数的移动平均,从而精确控制步长,且无需修改物理系统的漂移项。算法实现简单,可与任意现成的固定步长Langevin积分器结合使用。以监控对数后验梯度范数为例,步长在梯度陡峭区域自动减小,在平坦区域增大,灵感来自Adam优化器,增强了数值稳定性和收敛速度。步长变化依赖于梯度范数的历史信息,相比即时控制更具稳定性与准确性。在尼尔漏斗和基于MNIST数据的贝叶斯神经网络分类实验中,验证了该方法在准确率与稳定性方面的优势。

原文摘要 · Abstract (English)

We present a framework for adaptive-stepsize MCMC sampling based on time-rescaled Langevin dynamics, in which the stepsize variation is dynamically driven by an additional degree of freedom. Our approach augments the phase space by an additional variable which in turn defines a time reparameterization. The use of an auxiliary relaxation equation allows accumulation of a moving average of a local monitor function and provides for precise control of the timestep while circumventing the need to modify the drift term in the physical system. Our algorithm is straightforward to implement and can be readily combined with any off-the-peg fixed-stepsize Langevin integrator. As a particular example, we consider control of the stepsize by monitoring the norm of the log-posterior gradient, which takes inspiration from the Adam optimizer, the stepsize being automatically reduced in regions of steep change of the log posterior and increased on plateaus, improving numerical stability and convergence speed. As in Adam, the stepsize variation depends on the recent history of the gradient norm, which enhances stability and improves accuracy compared to more immediate control approaches. We demonstrate the potential benefit of this method--both in accuracy and in stability--in numerical experiments including Neal's funnel and a Bayesian neural network for classification of MNIST data.

采样算法贝叶斯推断自适应步长Langevin

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。