arXiv:2608.29349cs.LG2026-08

解决大规模高斯过程模型的不确定性过估计问题

Information-Based Calibration of Uncertainty Quantification in Product-of-Experts Gaussian Process Models

  • 用信息论方法校准局部高斯过程的后验方差
  • 在6个数据集上使校准误差降低12.0%,对数似然提升2.3%
  • 适合需要可靠不确定性的大规模贝叶斯优化场景

单全局高斯过程(GP-glo)回归存在立方级计算开销,难以扩展至大数据集。产品专家高斯过程模型(GP-pro)通过组合局部高斯过程捕捉全局相关性,缓解了这一问题。然而,在不相交数据子集上训练局部专家可能导致后验方差被高估。我们提出GP-pro-c,一种基于信息论方法校准后验方差的产品专家高斯过程模型。该方法利用高斯过程中信息增益的单调性和次模性,定义校准比,以降低各局部高斯过程的后验方差。我们在四个合成函数和六个回归数据集上评估了GP-pro-c,使用负对数似然(NLL)、均方根误差(RMSE)和期望归一化校准误差(ENCE)。实验表明,与未校准的GP-pro模型相比,GP-pro-c平均使NLL降低2.3%,ENCE降低12.0%。所提方法在保持预测精度的同时减轻了后验方差过估计,并降低了计算复杂度。GP-pro-c为可扩展高斯过程模型中的不确定性估计提供了有前景的解决方案,可作为高维大规模数据贝叶斯优化的有用代理模型。

原文摘要 · Abstract (English)

Gaussian process (GP) regression with a single global GP (GP-glo) incurs cubic computational cost, limiting scalability to large datasets. Product-of-experts GP models (GP-pro), which combine local GP models to capture global correlations, alleviate this computational burden. However, training local experts on disjoint data subsets can lead to overestimated posterior variances. We propose GP-pro-c, a product-of-experts GP model that calibrates these variances using an information-based method. The method exploits the monotonicity and submodularity of information gain in GPs to define a calibration ratio that reduces the posterior variance of individual local GP models. We evaluate GP-pro-c using negative log-likelihood (NLL), root mean squared error (RMSE), and expected normalised calibration error (ENCE). Experiments on four synthetic functions and six regression datasets show that GP-pro-c achieves average reductions of 2.3% in NLL and 12.0% in ENCE compared with the uncalibrated GP-pro model. The proposed method mitigates posterior variance overestimation while maintaining predictive accuracy and reducing computational complexity. GP-pro-c provides a promising approach for uncertainty estimation in scalable GP models and may serve as a useful surrogate model for Bayesian optimisation with high-dimensional and large-scale data.

高斯过程不确定性量化可扩展性贝叶斯优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。