arXiv:2601.12023stat.MLcs.LG2026-01

用核方法解决半隐式变分推断的优化难题,无需额外优化。

A Kernel Approach for Semi-implicit Variational Inference

  • 基于核方法构造显式解,避免传统方法中的低层优化问题。
  • 优化目标简化为核Stein散度,可高效用随机梯度法求解。
  • 理论保证强,适用于需要高表达力又追求计算效率的贝叶斯推断场景。

半隐式变分推断(SIVI)通过分层半隐式分布提升变分族的表达能力,但其密度不可计算导致标准ELBO优化存在偏差。近期基于得分匹配的方法(SIVI-SM)虽通过极小极大公式解决该问题,却引入了额外的低层优化。本文提出核半隐式变分推断(KSIVI),一种原理严谨且可计算的替代方案,通过核方法消除低层优化。我们证明:在再生核希尔伯特空间中优化时,低层问题具有显式解,使目标函数简化为核Stein散度(KSD)。利用半隐式分布的分层结构,该KSD目标可借助随机梯度方法高效优化。我们通过蒙特卡洛梯度估计的方差界建立了优化保证,并推导出统计泛化误差上界为$ ilde{oldsymbol{O}}(1/oldsymbol{ ext{sqrt}}oldsymbol{n})$。进一步提出多层分层扩展,增强表达力同时保持可计算性。在合成数据和真实贝叶斯推断任务上的实验表明KSIVI有效可靠。

原文摘要 · Abstract (English)

Semi-implicit variational inference (SIVI) enhances the expressiveness of variational families through hierarchical semi-implicit distributions, but the intractability of their densities makes standard ELBO-based optimization biased. Recent score-matching approaches to SIVI (SIVI-SM) address this issue via a minimax formulation, at the expense of an additional lower-level optimization problem. In this paper, we propose kernel semi-implicit variational inference (KSIVI), a principled and tractable alternative that eliminates the lower-level optimization by leveraging kernel methods. We show that when optimizing over a reproducing kernel Hilbert space, the lower-level problem admits an explicit solution, reducing the objective to the kernel Stein discrepancy (KSD). Exploiting the hierarchical structure of semi-implicit distributions, the resulting KSD objective can be efficiently optimized using stochastic gradient methods. We establish optimization guarantees via variance bounds on Monte Carlo gradient estimators and derive statistical generalization bounds of order $\tilde{\mathcal{O}}(1/\sqrt{n})$. We further introduce a multi-layer hierarchical extension that improves expressiveness while preserving tractability. Empirical results on synthetic and real-world Bayesian inference tasks demonstrate the effectiveness of KSIVI.

变分推断核方法贝叶斯推理优化理论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。