改进稀疏高斯过程的变分下界，提升预测精度与超参数学习稳定性。
New Bounds for Sparse Variational Gaussian Processes
- 引入依赖训练样本数的额外参数，放宽传统变分假设。
- 可解析优化新下界，实现更紧的近似且降低超参数学习偏差。
- 兼容非高斯似然，代码改动小，适合现有稀疏GP框架升级。
稀疏变分高斯过程通过构造可处理的后验近似来实现高斯过程模型。这类方法的核心假设是真实后验分布（训练函数值与诱导变量）被一个包含条件高斯先验 $p({\bf f} | {\bf u})$ 因子分解的变分分布近似。尽管该假设被视为基础，我们发现，在模型训练中可通过引入一个依赖于 $N$ 个额外参数（$N$ 为训练样本数）的更通用变分分布 $q({\bf f} | {\bf u})$ 来放松这一限制。在高斯回归中，可对这些额外参数进行解析优化，并导出一个更紧的、可计算的压缩下界。该新下界也适用于随机优化，其实现只需对现有稀疏GP代码进行微小修改。此外，方法还可扩展至非高斯似然。在多个数据集上的实验表明，该方法能有效降低超参数学习的偏差，并带来更好的预测性能。
原文摘要 · Abstract (English)
Sparse variational Gaussian processes (GPs) construct tractable posterior approximations to GP models. At the core of these methods is the assumption that the true posterior distribution over training function values ${\bf f}$ and inducing variables ${\bf u}$ is approximated by a variational distribution that incorporates the conditional GP prior $p({\bf f} | {\bf u})$ in its factorization. While this assumption is considered as fundamental, we show that for model training we can relax it through the use of a more general variational distribution $q({\bf f} | {\bf u})$ that depends on $N$ extra parameters, where $N$ is the number of training examples. In GP regression, we can analytically optimize the evidence lower bound over the extra parameters and express a tractable collapsed bound that is tighter than the previous bound. The new bound is also amenable to stochastic optimization and its implementation requires minor modifications to existing sparse GP code. Further, we also describe extensions to non-Gaussian likelihoods. On several datasets we demonstrate that our method can reduce bias when learning the hyperparameters and can lead to better predictive performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。