在对称分布下,变分推断能精确恢复均值与相关矩阵。
Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrix
- 利用位置尺度族与目标分布的对称性,提升变分推断鲁棒性。
- 即使近似分布因子化或尾部行为不同,仍可精确恢复均值和相关矩阵。
- 适用于贝叶斯推断中对称性为合理理想化的场景。
当目标密度 $p$ 不可计算时,变分推断(VI)试图从一个可处理的分布族 $Q$ 中寻找最优近似 $q$,通常通过最小化反向KL散度 $ ext{KL}(q||p)$ 实现。实践中 $Q$ 无法包含 $p$,即便 $q$ 是 $ ext{KL}(q||p)$ 的唯一全局极小值,近似也存在偏差。本文研究当 $p$ 具有特定对称性且 $Q$ 为共享这些对称性的位置尺度族时,VI 对模型误设的鲁棒性。证明了在温和正则条件下,甚至严重误设时仍具有强保证:(i) 若 $p$ 具有偶对称性,则 VI 能精确恢复 $p$ 的均值;(ii) 若 $p$ 还具有椭球对称性,则能精确恢复其相关矩阵。该均值恢复结果在 $q$ 因子化而 $p$ 不是时仍成立,相关矩阵恢复在 $q$ 与 $p$ 尾部行为不同时亦成立。我们分析了多种贝叶斯推断情形下此类对称性的合理性,并实验检验了对称性缺失时的性能表现。
原文摘要 · Abstract (English)
Given an intractable target density $p$, variational inference (VI) attempts to find the best approximation $q$ from a tractable family $Q$. This is typically done by minimizing the exclusive Kullback-Leibler divergence, $\text{KL}(q||p)$. In practice, $Q$ is not rich enough to contain $p$, and the approximation is misspecified even when it is a unique global minimizer of $\text{KL}(q||p)$. In this paper, we analyze the robustness of VI to these misspecifications when $p$ exhibits certain symmetries and $Q$ is a location-scale family that shares these symmetries. We prove strong guarantees for VI not only under mild regularity conditions but also in the face of severe misspecifications. Namely, we show that (i) VI recovers the mean of $p$ when $p$ exhibits an \textit{even} symmetry, and (ii) it recovers the correlation matrix of $p$ when in addition~$p$ exhibits an \textit{elliptical} symmetry. These guarantees hold for the mean even when $q$ is factorized and $p$ is not, and for the correlation matrix even when~$q$ and~$p$ behave differently in their tails. We analyze various regimes of Bayesian inference where these symmetries are useful idealizations, and we also investigate experimentally how VI behaves in their absence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。