用经验神经正切核实现高效可靠的神经网络不确定性量化
Uncertainty Quantification with the Empirical Neural Tangent Kernel
- 通过线性化网络并采样梯度，构建高效的深度集成模型
- 计算成本降低数倍，同时在回归与分类任务中表现领先
- 适合需要高可信度预测的医疗、自动驾驶等关键场景
尽管神经网络在各类任务中表现出色，但准确量化其预测不确定性对于确保可信性并推动其在关键系统中的应用至关重要。现有贝叶斯不确定性量化方法要么成本低，要么可靠，难以兼顾。本文提出一种训练后、基于采样的不确定性量化方法，适用于过参数化网络。该方法通过对适当线性化的网络进行（随机）梯度下降采样，构建高效且有意义的深度集成。我们证明该方法能有效近似高斯过程的后验分布，使用经验神经正切核（empirical Neural Tangent Kernel）。一系列数值实验表明，该方法不仅在计算效率上显著优于现有方法——常降低成本多个数量级，还在回归和分类任务的多种不确定性量化指标上保持了顶尖性能。
原文摘要 · Abstract (English)
While neural networks have demonstrated impressive performance across various tasks, accurately quantifying uncertainty in their predictions is essential to ensure their trustworthiness and enable widespread adoption in critical systems. Several Bayesian uncertainty quantification (UQ) methods exist that are either cheap or reliable, but not both. We propose a post-hoc, sampling-based UQ method for over-parameterized networks at the end of training. Our approach constructs efficient and meaningful deep ensembles by employing a (stochastic) gradient-descent sampling process on appropriately linearized networks. We demonstrate that our method effectively approximates the posterior of a Gaussian process using the empirical Neural Tangent Kernel. Through a series of numerical experiments, we show that our method not only outperforms competing approaches in computational efficiency-often reducing costs by multiple factors-but also maintains state-of-the-art performance across a variety of UQ metrics for both regression and classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。