改进黑箱变分推断稳定性，用自适应指数积分提升混合高斯近似效果
Adaptive Exponential Integration for Stable Gaussian Mixture Black-Box Variational Inference
- 采用自然梯度预处理与指数积分，保证协方差矩阵始终正定
- 自适应时间步长使算法在多模态分布和偏微分方程反问题中稳定收敛
- 理论证明噪声无时指数收敛，适合复杂后验分布的高效推断
基于高斯混合族的黑箱变分推断（BBVI）无需目标密度梯度即可灵活逼近复杂后验分布。但标准数值优化常因不稳定和低效而受限。本文提出一个稳定高效的框架，包含三个核心：(1) 通过自然梯度实现仿射不变预处理；(2) 使用指数积分无条件保持协方差矩阵正定性；(3) 自适应时间步长以保障稳定性并适应不同升温与收敛阶段。该方法与流形优化及镜像下降有天然联系。对高斯后验，在无噪声设定下证明指数收敛，在蒙特卡洛估计下实现几乎必然收敛，严格验证了自适应时间步的必要性。在多模态分布、Neal多尺度漏斗模型及基于偏微分方程的达西流贝叶斯反问题上，实验验证了方法的有效性。
原文摘要 · Abstract (English)
Black-box variational inference (BBVI) with Gaussian mixture families offers a flexible approach for approximating complex posterior distributions without requiring gradients of the target density. However, standard numerical optimization methods often suffer from instability and inefficiency. We develop a stable and efficient framework that combines three key components: (1) affine-invariant preconditioning via natural gradient formulations, (2) an exponential integrator that unconditionally preserves the positive definiteness of covariance matrices, and (3) adaptive time stepping to ensure stability and to accommodate distinct warm-up and convergence phases. The proposed approach has natural connections to manifold optimization and mirror descent. For Gaussian posteriors, we prove exponential convergence in the noise-free setting and almost-sure convergence under Monte Carlo estimation, rigorously justifying the necessity of adaptive time stepping. Numerical experiments on multimodal distributions, Neal's multiscale funnel, and a PDE-based Bayesian inverse problem for Darcy flow demonstrate the effectiveness of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。