arXiv:2409.01464stat.MLcs.LG2024-09被引 9

一种新型贝叶斯推断方法,能更高效地逼近后验分布。

Stein transport for Bayesian inference

  • 基于核空间的向量场驱动粒子沿温度曲线演化
  • 在有限时间 t=1 达到后验近似,比 SVGD 更快更准
  • 有效缓解 SVGD 的方差坍缩问题,适合高维推断

我们提出一种名为 Stein transport 的新贝叶斯推断方法,通过预定义的温度分布路径高效推动粒子集演化。向量场来自再生核希尔伯特空间,可通过核岭回归或 Stein 几何下的无穷小最优传输映射获得。其更新公式虽与 Stein 变分梯度下降(SVGD)相似,但引入了随时间变化的得分函数和粒子加权机制。与依赖长期收敛的 SVGD 不同,Stein transport 在有限时间 $t=1$ 即可达到后验近似。通过研究均场极限,我们分析了正则化与有限粒子效应带来的误差,并将其与出生-死亡过程及 Fisher-Rao 梯度流关联。实验表明,相较于 SVGD,Stein transport 通常以更少计算开销获得更高精度的后验近似,并显著缓解了常见的方差坍缩现象。

原文摘要 · Abstract (English)

We introduce $\textit{Stein transport}$, a novel methodology for Bayesian inference designed to efficiently push an ensemble of particles along a predefined curve of tempered probability distributions. The driving vector field is chosen from a reproducing kernel Hilbert space and can be derived either through a suitable kernel ridge regression formulation or as an infinitesimal optimal transport map in the Stein geometry. The update equations of Stein transport resemble those of Stein variational gradient descent (SVGD), but introduce a time-varying score function as well as specific weights attached to the particles. While SVGD relies on convergence in the long-time limit, Stein transport reaches its posterior approximation at finite time $t=1$. Studying the mean-field limit, we discuss the errors incurred by regularisation and finite-particle effects, and we connect Stein transport to birth-death dynamics and Fisher-Rao gradient flows. In a series of experiments, we show that in comparison to SVGD, Stein transport not only often reaches more accurate posterior approximations with a significantly reduced computational budget, but that it also effectively mitigates the variance collapse phenomenon commonly observed in SVGD.

贝叶斯推断粒子演化Stein 方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。