最后一层线性化可高效实现与全网相当的不确定性量化
Is the Last Layer Sufficient for Uncertainty Quantification?
- 仅对最后一层进行线性化，大幅降低计算开销
- 实验显示其不确定性预测性能与全网线性化相当
- 适合追求高效可靠不确定性的实际应用开发者
深度神经网络的认知不确定性量化（UQ）是确保AI在关键任务中安全应用的必要条件。现有主流方法将DNN线性化为贝叶斯广义线性模型（GLM），通过预测后验分布建模不确定性。通常采用仅线性化网络最后一层的近似方法以降低计算负担，但普遍认为会牺牲性能。本文从理论和实证两方面比较全网与仅最后一层线性化所得的GLM。基于随机矩阵理论的分析表明，全网线性化在UQ能力上并无显著提升。结合跨多种现代机器学习任务的大规模实证评估，结论为：仅用最后一层近似即可获得与全网线性化相当的UQ性能，同时计算效率显著更高。
原文摘要 · Abstract (English)
Epistemic uncertainty quantification (UQ) for deep neural networks (DNNs) is a requirement for safe adoption of AI in mission-critical settings. Several leading methods for UQ linearize DNNs to form Bayesian Generalized Linear Models (GLMs), where epistemic uncertainty is modeled via the predictive posterior distribution. Linearizing around the parameters of the final connected layer of a DNN is a commonly used approximation for reducing the computational burden of such GLMs, though it is often believed to come at the cost of degraded performance. In this work, we compare GLMs arising from full-network and last-layer linearization using both theoretical and empirical approaches. We first employ tools from random matrix theory to conduct a theoretical comparison; this analysis reveals no meaningful improvement in the UQ capabilities of full linearization. Coupled with a large-scale empirical evaluation across a range of modern machine learning tasks, we arrive at the following conclusion: a last-layer approximation yields comparable UQ performance while offering substantially improved computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。