用历史数据直接学出更灵活的高斯过程先验,提升预测能力。
Empirical Gaussian Processes
- 从历史数据中直接估算均值和协方差函数,替代手工设计核函数
- 在学习曲线外推和时间序列预测上达到竞争性性能
- 适合需要自适应先验、不依赖专家经验的建模场景
高斯过程(GPs)是强大的概率回归模型,但其实际效果常受限于核函数的选择。传统核函数多来自少量标准形式,需专家知识,适应性差,且对假设空间有强假设。本文研究经验高斯过程(Empirical GPs),一种构建灵活、数据驱动的GP先验的合理框架。不依赖参数化核函数,而是从历史观测数据中经验估计均值与协方差函数,使先验能反映数据中复杂的非平凡协方差结构。理论上,该模型收敛至与真实数据生成过程在KL散度意义下最接近的GP。实践中,将从独立数据集学习GP先验的问题建模为似然估计,并推导出具有闭式更新的期望最大化算法,可处理跨数据集异质观测位置。实验表明,经验高斯过程在学习曲线外推和时间序列预测基准任务中表现优异。
原文摘要 · Abstract (English)
Gaussian processes (GPs) are powerful and widely used probabilistic regression models, but their effectiveness in practice is often limited by the choice of kernel function. This kernel function is typically handcrafted from a small set of standard functions, a process that requires expert knowledge, results in limited adaptivity to data, and imposes strong assumptions on the hypothesis space. We study Empirical GPs, a principled framework for constructing flexible, data-driven GP priors that overcome these limitations. Rather than relying on standard parametric kernels, we estimate the mean and covariance functions empirically from a corpus of historical observations, enabling the prior to reflect rich, non-trivial covariance structures present in the data. Theoretically, we show that the resulting model converges to the GP that is closest (in KL-divergence sense) to the real data generating process. Practically, we formulate the problem of learning the GP prior from independent datasets as likelihood estimation and derive an Expectation-Maximization algorithm with closed-form updates, allowing the model handle heterogeneous observation locations across datasets. We demonstrate that Empirical GPs achieve competitive performance on learning curve extrapolation and time series forecasting benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。