让高斯过程在大规模数据上快速选模型并保持不确定性量化能力
Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference
- 设计新损失函数实现线性时间模型选择
- 180万数据点训练仅需数小时，性能优于主流方法
- 适合需要可靠不确定性的大模型应用
高斯过程的模型选择在训练数据规模增大时面临时间和内存的双重瓶颈。尽管已有多种近似方法，但均带来不可避免的误差。近期工作将计算误差建模为计算不确定性，实现了计算与精度的显式权衡，但代价为二次复杂度。本文将其扩展至模型选择，引入线性时间算法，并提出新型超参数优化训练损失。实验表明，该方法在中大型数据集上优于当前最优的SGPR、CGGP和SVGP。在180万数据点上训练的计算感知高斯过程可在单张GPU上数小时内完成模型选择。结果使高斯过程在大规模数据上仍能保持可靠的不确定性量化能力，满足最优决策的基本需求。
原文摘要 · Abstract (English)
Model selection in Gaussian processes scales prohibitively with the size of the training dataset, both in time and memory. While many approximations exist, all incur inevitable approximation error. Recent work accounts for this error in the form of computational uncertainty, which enables -- at the cost of quadratic complexity -- an explicit tradeoff between computation and precision. Here we extend this development to model selection, which requires significant enhancements to the existing approach, including linear-time scaling in the size of the dataset. We propose a novel training loss for hyperparameter optimization and demonstrate empirically that the resulting method can outperform SGPR, CGGP and SVGP, state-of-the-art methods for GP model selection, on medium to large-scale datasets. Our experiments show that model selection for computation-aware GPs trained on 1.8 million data points can be done within a few hours on a single GPU. As a result of this work, Gaussian processes can be trained on large-scale datasets without significantly compromising their ability to quantify uncertainty -- a fundamental prerequisite for optimal decision-making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。