改进变分推断的收敛性，用平方根参数化提升稳定性
Optimization Guarantees for Square-Root Natural-Gradient Variational Inference
- 采用平方根参数化重构高斯协方差，解决自然梯度收敛难题
- 理论证明在凹对数似然下可实现收敛，连续时间流也成立
- 实验显示优于欧氏与Wasserstein几何方法，适合高维推断
使用自然梯度下降进行变分推断在实践中通常收敛迅速，但其理论收敛性难以建立，即使在最简单的情形——对数似然为凹函数且采用高斯近似时也是如此。本文表明，通过采用高斯协方差的平方根参数化，可克服这一挑战。该方法建立了自然梯度变分高斯推断及其连续时间梯度流的新收敛保证。实验结果验证了自然梯度方法的有效性，并凸显其相较于欧氏或Wasserstein几何算法的优势。
原文摘要 · Abstract (English)
Variational inference with natural-gradient descent often shows fast convergence in practice, but its theoretical convergence guarantees have been challenging to establish. This is true even for the simplest cases that involve concave log-likelihoods and use a Gaussian approximation. We show that the challenge can be circumvented for such cases using a square-root parameterization for the Gaussian covariance. This approach establishes novel convergence guarantees for natural-gradient variational-Gaussian inference and its continuous-time gradient flow. Our experiments demonstrate the effectiveness of natural gradient methods and highlight their advantages over algorithms that use Euclidean or Wasserstein geometries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。