用可学习的推断函数提升深度高斯过程的表达能力与效率
Amortized Variational Inference for Deep Gaussian Processes
- 用可学习的映射函数将观测值直接转为变分参数
- 在相同计算量下比传统方法更准确，且能处理复杂函数
- 适合需要高效精确不确定估计的复杂建模任务
高斯过程（GPs）是用于函数逼近的贝叶斯非参数模型，能提供合理的预测不确定性估计。深度高斯过程（DGPs）是多层推广，可表示复杂的边缘分布和映射关系。由于精确推断在GP及其扩展中要么计算成本过高，要么解析不可行，现有方法常采用变分推断（VI）进行近似。但传统近似模型的表达力严重依赖独立的诱导变量，可能不足以应对某些问题。本文提出针对DGPs的折返式变分推断，学习一个从每个观测值映射到变分参数的推断函数。该方法拥有更丰富的先验（基于更少输入相关的诱导变量），以及灵活的折返边际后验，能更好地建模复杂函数。理论分析与实验表明，本方法在计算成本更低的情况下，表现与甚至优于先前方法。
原文摘要 · Abstract (English)
Gaussian processes (GPs) are Bayesian nonparametric models for function approximation with principled predictive uncertainty estimates. Deep Gaussian processes (DGPs) are multilayer generalizations of GPs that can represent complex marginal densities as well as complex mappings. As exact inference is either computationally prohibitive or analytically intractable in GPs and extensions thereof, some existing methods resort to variational inference (VI) techniques for tractable approximations. However, the expressivity of conventional approximate GP models critically relies on independent inducing variables that might not be informative enough for some problems. In this work we introduce amortized variational inference for DGPs, which learns an inference function that maps each observation to variational parameters. The resulting method enjoys a more expressive prior conditioned on fewer input dependent inducing variables and a flexible amortized marginal posterior that is able to model more complicated functions. We show with theoretical reasoning and experimental results that our method performs similarly or better than previous approaches at less computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。