用二次近似改进神经网络的不确定性估计，计算开销小且效果稳定。
Improving the Linearized Laplace Approximation via Quadratic Approximations
- 用幂迭代高效逼近二阶项，构建二次拉普拉斯近似
- 在五个回归数据集上比线性拉普拉斯提升不确定性估计性能
- 适合关注模型置信度校准的研究者或应用者
深度神经网络常对分布外样本产生过度自信的预测，推动了贝叶斯不确定性量化的发展。线性化拉普拉斯近似（LLA）通过线性化DNN并对其应用拉普拉斯推断来实现这一目标，且该线性模型也用于预测。我们认为，后验中的线性化会降低对真实拉普拉斯近似的保真度。为缓解此问题，我们提出二次拉普拉斯近似（QLA），在不显著增加计算成本的前提下，通过高效的幂迭代获取秩一因子，近似近似拉普拉斯对数后验中的每个二阶项。QLA预期能获得更接近完整拉普拉斯的后验精度，而无需构造完整的海森矩阵（通常不可行）。预测时仍使用线性化模型。实验表明，QLA在五个回归数据集上实现了适度但一致的不确定性估计改进。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) often produce overconfident out-of-distribution predictions, motivating Bayesian uncertainty quantification. The Linearized Laplace Approximation (LLA) achieves this by linearizing the DNN and applying Laplace inference to the resulting model. Importantly, the linear model is also used for prediction. We argue this linearization in the posterior may degrade fidelity to the true Laplace approximation. To alleviate this problem, without increasing significantly the computational cost, we propose the Quadratic Laplace Approximation (QLA). QLA approximates each second order factor in the approximate Laplace log-posterior using a rank-one factor obtained via efficient power iterations. QLA is expected to yield a posterior precision closer to that of the full Laplace without forming the full Hessian, which is typically intractable. For prediction, QLA also uses the linearized model. Empirically, QLA yields modest yet consistent uncertainty estimation improvements over LLA on five regression datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。