arXiv:2510.12128cs.LGcs.DC2025-10

nuGPR加速高斯过程回归,显著降低训练时间和内存占用。

nuGPR: GPU-Accelerated Gaussian Process Regression with Iterative Algorithms and Low-Rank Approximations

  • 用预条件共轭梯度法加速求解,结合聚类发现协方差矩阵块对角结构。
  • 通过低秩近似减少计算量,实现最高12倍内存降低和2倍训练提速。
  • 采用数值梯度替代反向传播,适合大规模高斯过程建模与实时应用。

高斯过程回归(GPR)是一种具有固有不确定性估计的重要监督学习模型。本文提出nuGPR框架,解决GPR训练中计算成本高的问题。该框架融合数值线性代数技术,优化关键步骤的计算效率。具体包括:利用预条件共轭梯度法加速GPR中的线性求解;通过输入数据聚类识别协方差矩阵的块对角结构,并对非对角块构建低秩近似;进一步采用数值梯度优化超参数,避免反向传播开销;最后基于CUDA Toolkit在NVIDIA GPU上并行化训练流程。实验表明,相较于现有最佳GPU实现,nuGPR在多种合成与真实数据集上可将总训练时间减少最多2倍,峰值内存消耗降低最多12倍。

原文摘要 · Abstract (English)

Gaussian Process Regression (GPR) is an important type of supervised machine learning model with inherent uncertainty measure in its predictions. We propose a new framework, nuGPR, to address the well-known challenge of high computation cost associated with GPR training. Our framework includes several ideas from numerical linear algebra to reduce the amount of computation in key steps of GPR, and we combine them to establish an end-to-end training algorithm. Specifically, we leverage the preconditioned conjugate gradient method to accelerate the convergence of the linear solves required in GPR. We exploit clustering in the input data to identify block-diagonal structure of the covariance matrix and subsequently construct low-rank approximations of the off-diagonal blocks. These enhancements significantly reduce the time and space complexity of our computations. In addition, unlike other frameworks that rely on exact differentiation, we employ numerical gradients to optimize the hyperparameters of our GPR model, further reducing the training cost by eliminating the need for backpropagation. Lastly, we leverage the CUDA Toolkit to efficiently parallelize the training procedure on NVIDIA GPUs. As a result, nuGPR reduces total training time by up to 2x and peak memory consumption by up to 12x on various synthetic and real-world datasets when compared to the best existing GPU-based GPR implementation.

高斯过程GPU加速低秩近似数值优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。