无需矩阵求逆的稀疏高斯过程，训练更快更稳定。
Inverse-Free Sparse Variational Gaussian Processes
- 用仅含矩阵乘法的优化方式替代传统求逆，适配高效硬件。
- 在回归与分类任务中性能接近传统方法，调优后速度更快。
- 提供实用调参技巧，可直接接入现有模型流程。
高斯过程（GPs）具有优异性质，但大规模训练成本高昂。稀疏变分高斯过程（SVGP）虽降低开销，仍依赖核矩阵的Cholesky分解，不适用于低精度、高度并行的硬件。已有方法通过引入辅助矩阵参数构建仅依赖矩阵乘法的变分下界，但用标准一阶优化器优化困难。本文提出更良态的下界，并推导出仅含矩阵乘法的自然梯度更新规则，显著提升优化稳定性与收敛性。进一步设计简单启发式策略，如学习率调度和停止条件，使整体优化流程可无缝嵌入现有工作流。在回归与分类基准测试中，所提方法1）可作为SVGP模型的即插即用替代方案（如深度高斯过程）；2）性能与传统方法相当；3）经良好调优后，速度优于基线。
原文摘要 · Abstract (English)
Gaussian processes (GPs) offer appealing properties but are costly to train at scale. Sparse variational GP (SVGP) approximations reduce cost yet still rely on Cholesky decompositions of kernel matrices, ill-suited to low-precision, massively parallel hardware. While one can construct valid variational bounds that rely only on matrix multiplications (matmuls) via an auxiliary matrix parameter, optimising them with off-the-shelf first-order methods is challenging. We make the inverse-free approach practical by proposing a better-conditioned bound and deriving a matmul-only natural-gradient update for the auxiliary parameter, markedly improving stability and convergence. We further provide simple heuristics, such as step-size schedules and stopping criteria, that make the overall optimisation routine fit seamlessly into existing workflows. Across regression and classification benchmarks, we demonstrate that our method 1) serves as a drop-in replacement in SVGP-based models (e.g., deep GPs), 2) recovers similar performance to traditional methods, and 3) can be faster than baselines when well tuned.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。