arXiv:2504.05431stat.MEcs.LG2025-04

提出新变分推断框架,高效处理强超高斯似然模型

A Generalized Tangent Approximation based Variational Inference Framework for Strongly Super-Gaussian Likelihoods

  • 基于切线近似与凸对偶构造对数似然下界,实现高斯先验共轭
  • 在温和假设下证明算法收敛性,且达到近最优变分风险界
  • 适用于复杂数据结构建模,适合需理论保障的高维贝叶斯推断场景

变分推断作为马尔可夫链蒙特卡洛采样的替代方法,在复杂贝叶斯模型中实现了可扩展计算。然而,现有方法通常依赖于特定模型形式或随机黑箱优化。切线近似是一种利用概率模型几何结构的系统化变分方法,但其应用主要限于逻辑回归等有限场景。本文提出一种基于切线变换的新变分框架,适用于具有强超高斯似然的广泛概率模型。该方法通过凸对偶构建对数似然的切线下界,使在原本不可解的设定下,模型参数的高斯先验仍保持共轭性。在对数据生成机制的温和假设下,我们建立了算法收敛性保证,这一贡献区别于通常缺乏理论支撑的黑箱变分方法。此外,我们推导出变分风险的近最小极大最优界。所提方法在模拟和真实数据场景中表现优异,挑战了现有变分算法在可扩展性与捕捉复杂数据结构方面的能力。

原文摘要 · Abstract (English)

Variational inference, as an alternative to Markov chain Monte Carlo sampling, has played a transformative role in enabling scalable computation for complex Bayesian models. Nevertheless, existing approaches often depend on either rigid model-specific formulations or stochastic black-box optimization routines. Tangent approximation is a principled class of structured variational methods that exploits the geometry of the underlying probability model. However, its utility has largely been confined to logistic regression and related modeling regimes. In this article, we propose a novel variational framework based on tangent transformation for a broad class of probability models characterized by strongly super-Gaussian likelihoods. Our method leverages convex duality to construct tangent minorants of the log-likelihood, thereby inducing conjugacy with Gaussian priors over model parameters in an otherwise intractable setup. Under mild assumptions on the data-generating mechanism, we establish algorithmic convergence guarantees, a contribution that stands in contrast to the limited theoretical assurances typically available for black-box variational methods. Additionally, we derive near-minimax optimal bounds for the variational risk. Superior performance of our proposed methodology is illustrated on simulated and real-data scenarios that challenge state-of-the-art variational algorithms in terms of scalability and their ability to consistently capture complex underlying data structure.

变分推断贝叶斯建模理论保证高斯先验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。