用学生t分布提升神经网络不确定性估计的可靠性
Function-Space Empirical Bayes Regularisation with Student's t Priors
- 在参数与函数空间同时使用重尾的学生t先验，更符合神经网络输出特性
- 在分布内预测和分布外检测任务上均优于主流变分推断方法
- 适合需要可信置信度的场景，如医疗诊断、自动驾驶
贝叶斯深度学习通过将深度神经网络与贝叶斯推断结合，为不确定性估计提供了严谨框架，但选择信息性强的先验分布仍是关键挑战。现有函数空间变分推断（FSVI）正则化方法通常采用高斯先验，难以捕捉神经网络输出固有的重尾统计特征。本文提出一种新的函数空间经验贝叶斯正则化框架——ST-FS-EB，同时在参数空间和函数空间引入重尾的学生t先验。通过变分推断（VI）近似后验分布，基于蒙特卡洛丢弃（MC dropout）构建证据下界（ELBO）目标。在多种基于变分推断的贝叶斯深度学习基线方法对比中，该方法在分布内预测、分布外检测及分布偏移处理任务上均表现出更强鲁棒性。
原文摘要 · Abstract (English)
Bayesian deep learning (BDL) has emerged as a principled approach to produce reliable uncertainty estimates by integrating deep neural networks with Bayesian inference, and the selection of informative prior distributions remains a significant challenge. Various function-space variational inference (FSVI) regularisation methods have been presented, assigning meaningful priors over model predictions. However, these methods typically rely on a Gaussian prior, which fails to capture the heavy-tailed statistical characteristics inherent in neural network outputs. By contrast, this work proposes a novel function-space empirical Bayes regularisation framework -- termed ST-FS-EB -- which employs heavy-tailed Student's $t$ priors in both parameter and function spaces. Also, we approximate the posterior distribution through variational inference (VI), inducing an evidence lower bound (ELBO) objective based on Monte Carlo (MC) dropout. Furthermore, the proposed method is evaluated against various VI-based BDL baselines, and the results demonstrate its robust performance in in-distribution prediction, out-of-distribution (OOD) detection and handling distribution shifts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。