arXiv:2412.03312cs.LGcs.AI2024-12ICML被引 11

用新路径引导粒子采样,提升贝叶斯推断精度与校准能力

Path-Guided Particle-based Sampling

论文配图:Path-Guided Particle-based Sampling
图 1 · 摘自论文原文
  • 设计新型对数加权收缩密度路径,引导粒子从先验向后验演化
  • 在合成与真实任务中,相比SVG、Langevin等方法,误差更小、校准更好
  • 适合需要高精度后验近似的贝叶斯学习场景,如不确定性建模

基于无分区目标分布的粒子化贝叶斯推断方法(如Stein变分梯度下降, SVGD)受到广泛关注。本文提出一种路径引导的粒子采样方法(PGPS),其核心是设计一种新的对数加权收缩(Log-weighted Shrinkage, LwS)密度路径,连接初始分布与目标分布。通过神经网络学习由该密度路径的Fokker-Planck方程启发的向量场,粒子从初始分布出发,沿常微分方程定义的轨迹演化,使粒子分布沿密度路径逐步逼近目标分布。所提的LwS路径能高效探索目标分布的多个模式,而传统方法在此类情形下表现不佳。理论上分析了因近似与离散化误差导致的样本分布与目标分布间的Wasserstein距离。实验表明,所提PGPS-LwS方法在合成与真实世界贝叶斯学习任务中,相较基线方法(如SVG、Langevin动力学)具有更高的推断准确率与更好的校准性能。

原文摘要 · Abstract (English)

Particle-based Bayesian inference methods by sampling from a partition-free target (posterior) distribution, e.g., Stein variational gradient descent (SVGD), have attracted significant attention. We propose a path-guided particle-based sampling~(PGPS) method based on a novel Log-weighted Shrinkage (LwS) density path linking an initial distribution to the target distribution. We propose to utilize a Neural network to learn a vector field motivated by the Fokker-Planck equation of the designed density path. Particles, initiated from the initial distribution, evolve according to the ordinary differential equation defined by the vector field. The distribution of these particles is guided along a density path from the initial distribution to the target distribution. The proposed LwS density path allows for an efficient search of modes of the target distribution while canonical methods fail. We theoretically analyze the Wasserstein distance of the distribution of the PGPS-generated samples and the target distribution due to approximation and discretization errors. Practically, the proposed PGPS-LwS method demonstrates higher Bayesian inference accuracy and better calibration ability in experiments conducted on both synthetic and real-world Bayesian learning tasks, compared to baselines, such as SVGD and Langevin dynamics, etc.

贝叶斯推断粒子采样密度路径

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。