用稀疏基核近似加速高斯过程，实现大模型下的高效不确定性建模。
SIKA-GP: Accelerating Gaussian Process Inference with Sparse Inducing Kernel Approximations for Bayesian Deep Learning
- 基于稀疏激活的基函数构造紧凑核表示，实现低复杂度计算。
- 仅需O(log M)复杂度，支持大规模数据和深度网络中的快速推理。
- 可无缝嵌入贝叶斯神经网络，适合需要不确定性估计的大规模任务。
高斯过程（GPs）提供了不确定性估计的严格贝叶斯框架，但其计算复杂度严重限制了在大数据集上的可扩展性。本文提出SIKA-GP，通过基于双射有序模板基的稀疏诱导核近似加速GP推断，对诱导点数量仅依赖O(log M)复杂度。该方法从稀疏激活的基函数中构建紧凑且表达力强的核表示，支持高效的张量化GPU计算，并可与现代大规模模型无缝集成。SIKA-GP能自然嵌入具有稀疏激活的贝叶斯神经网络（BNNs），在不损失预测性能的前提下显著提升训练与推理速度。该方法还可推广至深度特征学习，解决深度架构与高维特征表示带来的可扩展性挑战。在视觉与基于Transformer的语言基准上的实验证明，该方法持续生成快速且准确的GP模型，为可扩展核学习提供了一条严谨路径。
原文摘要 · Abstract (English)
Gaussian processes (GPs) provide a principled Bayesian framework for uncertainty estimation, but their computational complexity severely limits scalability to large datasets. We propose SIKA-GP, which accelerates GP inference using sparse inducing kernel approximations based on a dyadic ordered template basis, incurring only ${O}(\log M)$ complexity dependence on the number of inducing points. Our approach constructs compact and expressive kernel representations from sparsely activated bases, enabling efficient tensorized GPU computation and seamless integration with modern large-scale models. SIKA-GP can be naturally embedded into Bayesian neural networks (BNNs) with sparse activations, yielding significant speedups in both training and inference without sacrificing predictive performance. The method naturally extends to deep feature learning, addressing the scalability challenges introduced by deep architectures and high-dimensional feature representations. Empirical results on vision and transformer-based language benchmarks demonstrate that our approach consistently delivers fast and accurate GP models, providing a principled path toward scalable kernel learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。