用各向同性高斯混合模型实现高效贝叶斯后验近似
Variational Inference with Mixtures of Isotropic Gaussians
- 采用等方差高斯混合模型，通过梯度下降优化中心位置和方差
- 在保持计算效率的同时，能准确逼近多峰后验分布
- 适合需要快速、低存储的贝叶斯推断场景
变分推断（VI）是贝叶斯推断中一种流行方法，旨在参数族内寻找后验分布的最佳近似，最小化通常为反向KL散度的损失。本文聚焦于各向同性高斯混合模型（即协方差矩阵为单位阵的倍数）且权重均匀的参数族。我们构建了相应的变分框架，并提出适用于该族的高效算法。与通用协方差高斯混合模型相比，该选择在准确逼近多峰后验分布与内存及计算效率之间取得平衡。算法通过梯度下降优化混合成分的位置（高斯均值），并采用（熵正则化）镜像下降或Bures下降优化方差参数。我们在数值实验中展示了算法性能。
原文摘要 · Abstract (English)
Variational inference (VI) is a popular approach in Bayesian inference, that looks for the best approximation of the posterior distribution within a parametric family, minimizing a loss that is typically the (reverse) Kullback-Leibler (KL) divergence. In this paper, we focus on the following parametric family: mixtures of isotropic Gaussians (i.e., with diagonal covariance matrices proportional to the identity) and uniform weights. We develop a variational framework and provide efficient algorithms suited for this family. In contrast with mixtures of Gaussian with generic covariance matrices, this choice presents a balance between accurate approximations of multimodal Bayesian posteriors, while being memory and computationally efficient. Our algorithms implement gradient descent on the location of the mixture components (the modes of the Gaussians), and either (an entropic) Mirror or Bures descent on their variance parameters. We illustrate the performance of our algorithms on numerical experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。