提出新型梯度估计器,解决无重参数化时变分推断的信号噪声比下降问题。
Importance Weighted Variational Inference without the Reparameterization Trick
- 基于REINFORCE改进变分推断梯度估计,避免重参数化限制
- 发现现有方法随采样数增加信号噪声比趋近于零
- 新提出的VIMCO-star实现√N级信号噪声比,适合复杂模型
重要性加权变分推断(VI)通过优化蒙特卡洛样本数为N的紧致界来近似归一化常数未知的概率密度。标准优化依赖重参数化梯度,虽理论完善但限制了数据生成过程与变分近似的灵活性。而无需重参数化的REINFORCE梯度虽无此限制,却缺乏严谨理论支撑。本文首次对重要性加权VI中的REINFORCE梯度进行全面分析,揭示并修复当前最先进的变分推断蒙特卡洛目标(VIMCO)梯度估计器的根本缺陷。我们引入广义的VIMCO梯度族,证明现有方法在样本数N增大时信号噪声比(SNR)趋于零,导致优化失效。为此,提出新型VIMCO-star梯度估计器,其可实现√N量级的SNR增长,克服了原有方法的信号崩溃问题。实验表明,在重参数化不可用的挑战性场景下,该方法显著优于现有VIMCO实现。
原文摘要 · Abstract (English)
Importance weighted variational inference (VI) approximates densities known up to a normalizing constant by optimizing bounds that tighten with the number of Monte Carlo samples $N$. Standard optimization relies on reparameterized gradient estimators, which are well-studied theoretically yet restrict both the choice of the data-generating process and the variational approximation. While REINFORCE gradient estimators do not suffer from such restrictions, they lack rigorous theoretical justification. In this paper, we provide the first comprehensive analysis of REINFORCE gradient estimators in importance weighted VI, leveraging this theoretical foundation to diagnose and resolve fundamental deficiencies in current state-of-the-art estimators. Specifically, we introduce and examine a generalized family of variational inference for Monte Carlo objectives (VIMCO) gradient estimators. We prove that state-of-the-art VIMCO gradient estimators exhibit a vanishing signal-to-noise ratio (SNR) as $N$ increases, which prevents effective optimization. To overcome this issue, we propose the novel VIMCO-$\star$ gradient estimator and show that it averts the SNR collapse of existing VIMCO gradient estimators by achieving a $\sqrt{N}$ SNR scaling instead. We demonstrate its superior empirical performance compared to current VIMCO implementations in challenging settings where reparameterized gradients are typically unavailable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。