arXiv:2507.20678cs.LG2025-07

改进了高斯过程中的乔列斯基分解,提升不确定性减少效率。

Novel Pivoted Cholesky Decompositions for Efficient Gaussian Process Inference

  • 基于熵最大化思想设计新型选主元策略,优化分解顺序。
  • 在稀疏回归和预条件迭代求解中性能优于传统方法,计算开销几乎不变。
  • 适合需要高效不确定性建模的主动学习与实验设计场景。

乔列斯基分解是求解对称正定矩阵线性系统的核心工具,广泛应用于线性代数、优化和机器学习。通过引入选主元策略,可提升其数值稳定性,而选主元顺序直接影响中间分解对原矩阵的重构精度,进而决定算法在提前终止时的效率。标准实现从对角线上选择最大值作为下一主元,这在贝叶斯非参数推断中等价于贪婪熵最大化,常用于主动学习与实验设计。本文深入探讨该联系,推导出新的选主元策略。所提算法能更高效地降低数据集上的不确定性,支持观测信息动态更新,并具备定制化实现优势。在高斯过程的两个关键任务——稀疏回归与预条件迭代求解——上进行基准测试,结果表明新策略性能与传统基线相当或更优,且额外计算开销可忽略不计。

原文摘要 · Abstract (English)

The Cholesky decomposition is a fundamental tool for solving linear systems with symmetric and positive definite matrices which are ubiquitous in linear algebra, optimization, and machine learning. Its numerical stability can be improved by introducing a pivoting strategy that iteratively permutes the rows and columns of the matrix. The order of pivoting indices determines how accurately the intermediate decomposition can reconstruct the original matrix, thus is decisive for the algorithm's efficiency in the case of early termination. Standard implementations select the next pivot from the largest value on the diagonal. In the case of Bayesian nonparametric inference, this strategy corresponds to greedy entropy maximization, which is often used in active learning and design of experiments. We explore this connection in detail and deduce novel pivoting strategies for the Cholesky decomposition. The resulting algorithms are more efficient at reducing the uncertainty over a data set, can be updated to include information about observations, and additionally benefit from a tailored implementation. We benchmark the effectiveness of the new selection strategies on two tasks important to Gaussian processes: sparse regression and inference based on preconditioned iterative solvers. Our results show that the proposed selection strategies are either on par or, in most cases, outperform traditional baselines while requiring a negligible amount of additional computation.

高斯过程乔列斯基分解主动学习优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。