提出多项式斯坦因差异，高效评估贝叶斯采样质量
The Polynomial Stein Discrepancy for Assessing Moment Convergence
- 基于多项式核构造新型差异度量，避免传统方法的高计算开销
- 在高维下仍保持良好性能，对前r阶矩差异检测力强于竞品
- 可帮助优化贝叶斯采样算法超参数，适合大规模推断场景
我们提出一种新方法，用于衡量样本集与目标后验分布之间的差异，以评估贝叶斯推断中的样本质量。经典方法如有效样本量不适用于存在渐近偏差的可扩展采样算法（如随机梯度朗之万动力学）。目前金标准是核斯坦因差异（KSD），但其计算复杂度为样本数的二次方，难以扩展，且易受维度诅咒影响，需大量调参。为此，我们开发了多项式斯坦因差异（PSD）及其配套拟合优度检验。尽管该检验不能完全确定收敛性，但我们证明它能检测高斯目标下前r阶矩的差异。实验显示，该方法在多个案例中功效高于竞品，且计算成本更低。最后，我们验证了PSD能更高效地协助选择贝叶斯采样算法的超参数。
原文摘要 · Abstract (English)
We propose a novel method for measuring the discrepancy between a set of samples and a desired posterior distribution for Bayesian inference. Classical methods for assessing sample quality like the effective sample size are not appropriate for scalable Bayesian sampling algorithms, such as stochastic gradient Langevin dynamics, that are asymptotically biased. Instead, the gold standard is to use the kernel Stein Discrepancy (KSD), which is itself not scalable given its quadratic cost in the number of samples. The KSD and its faster extensions also typically suffer from the curse of dimensionality and can require extensive tuning. To address these limitations, we develop the polynomial Stein discrepancy (PSD) and an associated goodness-of-fit test. While the new test is not fully convergence-determining, we prove that it detects differences in the first r moments for Gaussian targets. We empirically show that the test has higher power than its competitors in several examples, and at a lower computational cost. Finally, we demonstrate that the PSD can assist practitioners to select hyper-parameters of Bayesian sampling algorithms more efficiently than competitors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。