提出新算法提升大规模变分推断速度与稳定性。
Scalable Mean-Field Variational Inference via Preconditioned Primal-Dual Optimization
- 从对偶视角重构问题，用增广拉格朗日法联合优化全局与局部参数。
- 在真实空间转录组数据上收敛更快，解的质量优于现有方法。
- 自适应调整不同参数块的更新步长，适合大规模复杂模型推断。
本文从随机小批量对偶视角研究大规模均值场变分推断（MFVI）问题。通过将MFVI重述为约束有限和问题，提出一种基于增广拉格朗日公式的新型对偶算法——对偶变分推断（PD-VI），可规模化联合更新证据下界中的全局与局部变分参数。为应对不同参数块间异质损失几何，引入块预条件扩展版本P²D-VI，自适应调整各参数块的对偶更新，提升数值稳定性和实际效率。在固定步长下，建立了PD-VI与P²D-VI的收敛性保证，无需共轭假设或显式有界方差条件。一般情形下证明了$O(1/T)$收敛至驻点，强凸条件下实现线性收敛。在合成数据与真实大规模空间转录组数据上的实验表明，所提方法在收敛速度与解质量上均持续优于现有随机变分推断方法。
原文摘要 · Abstract (English)
In this work, we investigate the large-scale mean-field variational inference (MFVI) problem from a mini-batch primal-dual perspective. By reformulating MFVI as a constrained finite-sum problem, we develop a novel primal-dual algorithm based on an augmented Lagrangian formulation, termed primal-dual variational inference (PD-VI). PD-VI jointly updates global and local variational parameters in the evidence lower bound in a scalable manner. To further account for heterogeneous loss geometry across different variational parameter blocks, we introduce a block-preconditioned extension, P$^2$D-VI, which adapts the primal-dual updates to the geometry of each parameter block and improves both numerical robustness and practical efficiency. We establish convergence guarantees for both PD-VI and P$^2$D-VI under properly chosen constant step size, without relying on conjugacy assumptions or explicit bounded-variance conditions. In particular, we prove $O(1/T)$ convergence to a stationary point in general settings and linear convergence under strong convexity. Numerical experiments on synthetic data and a real large-scale spatial transcriptomics dataset demonstrate that our methods consistently outperform existing stochastic variational inference approaches in terms of convergence speed and solution quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。