arXiv:2602.01279cs.LG2026-02被引 4

提升神经网络不确定性估计,纠正传统方法低估风险的问题。

Richer Bayesian Last Layers with Subsampled NTK Features

  • 用NTK特征投影增强最后层贝叶斯推理,捕捉全网不确定性
  • 后验方差至少与标准方法相当,有效缓解低估风险问题
  • 采样加速计算,适合需要高效可靠不确定性的研究者

贝叶斯最后层(BLLs)为神经网络提供了一种便捷且计算高效的不确定性估计方式。然而,由于仅对最后一层进行贝叶斯处理,忽略了早期层带来的不确定性,导致对认知不确定性(epistemic uncertainty)的估计偏低。本文提出一种新方法,通过将神经正切核(NTK)特征投影到最后一层特征空间,实现对全网络变异性的后验推断,同时保持标准BLL的低计算开销。理论证明,该方法所得后验方差不小于标准BLL,从而纠正其低估认知不确定性的倾向。为进一步降低计算成本,引入均匀采样方案估算投影矩阵并执行后验推断,并推导出两类采样的近似误差界。在UCI回归、上下文老虎机、图像分类及图像与表格数据的分布外检测任务上的实证评估表明,该方法在提升校准度和不确定性估计性能的同时,显著降低计算开销,优于标准BLL与现有竞争基线。

原文摘要 · Abstract (English)

Bayesian Last Layers (BLLs) provide a convenient and computationally efficient way to estimate uncertainty in neural networks. However, they underestimate epistemic uncertainty because they apply a Bayesian treatment only to the final layer, ignoring uncertainty induced by earlier layers. We propose a method that improves BLLs by leveraging a projection of Neural Tangent Kernel (NTK) features onto the space spanned by the last-layer features. This enables posterior inference that accounts for variability of the full network while retaining the low computational cost of inference of a standard BLL. We show that our method yields posterior variances that are provably greater or equal to those of a standard BLL, correcting its tendency to underestimate epistemic uncertainty. To further reduce computational cost, we introduce a uniform subsampling scheme for estimating the projection matrix and for posterior inference. We derive approximation bounds for both types of subsampling. Empirical evaluations on UCI regression, contextual bandits, image classification, and out-of-distribution detection tasks in image and tabular datasets, demonstrate improved calibration and uncertainty estimates compared to standard BLLs and competitive baselines, while reducing computational cost.

贝叶斯深度学习不确定性估计神经正切核高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。