用并行重要性采样加速贝叶斯推断,比现有方法更快更稳定。
Massively Parallel Expectation Maximization For Approximate Posteriors
- 基于并行重要性采样估算后验矩,再用EM算法拟合近似后验
- 在多个数据集上速度比RWS和VI快2~5倍,收敛更稳定
- 适合大规模层次模型,对参数重参数不敏感
层次模型的贝叶斯推断极具挑战性。传统MCMC难以扩展到大规模模型,而变分推断(VI)和重加权唤醒-睡眠(RWS)虽可扩展,但依赖梯度,常需大量迭代。我们发现,现代大规模并行重要性权重方法(Bowyer等, 2024)能快速准确估计后验矩,由此提出基于期望最大化(QEM)的近似后验拟合方法:期望步使用该方法获取高质量后验矩,极大化步则利用这些矩拟合高斯、伽马、贝塔、狄利克雷、二项、多项、分类等简单分布(或其组合)。实验表明,QEM比最先进的并行RWS和VI快2~5倍,且对模型重参数化不变,避免了梯度方法的性能下降。
原文摘要 · Abstract (English)
Bayesian inference for hierarchical models can be very challenging. MCMC methods have difficulty scaling to large models with many observations and latent variables. While variational inference (VI) and reweighted wake-sleep (RWS) can be more scalable, they are gradient-based methods and so often require many iterations to converge. Our key insight was that modern massively parallel importance weighting methods (Bowyer et al., 2024) give fast and accurate posterior moment estimates, and we can use these moment estimates to rapidly learn an approximate posterior. Specifically, we propose using expectation maximization to fit the approximate posterior, which we call QEM. The expectation step involves computing the posterior moments using high-quality massively parallel estimates from Bowyer et al. (2024). The maximization step involves fitting the approximate posterior using these moments, which can be done straightforwardly for simple approximate posteriors such as Gaussian, Gamma, Beta, Dirichlet, Binomial, Multinomial, Categorical, etc. (or combinations thereof). We show that QEM is faster than state-of-the-art, massively parallel variants of RWS and VI, and is invariant to reparameterizations of the model that dramatically slow down gradient based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。