不重训模型,用概率跳跃连接实现精准不确定性估计
Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networks
- 通过神经坍缩检测选敏感平滑层,构建概率跳跃连接
- 单次前向传播即可分离认知与随机不确定性
- 适配已有模型,无需重训练,适合部署场景
深度学习中的确定性不确定性量化旨在通过网络一次前向传播估算不确定性,依赖特征提取器的输出。现有方法要求特征提取器兼具敏感性与平滑性,以确保输入变化能引发特征向量有意义的变化。平滑性支持泛化,敏感性防止特征坍塌(不同输入映射到相同特征)。当前方法常需使用谱归一化重新训练网络。本文提出不修改训练过程,利用神经坍缩度量识别一个既敏感又平滑的中间层,并在其上拟合概率模型,称为概率跳跃连接(PSC)。实证分析表明,谱归一化影响神经坍缩,而PSC能有效解耦认知不确定性与随机不确定性。此外,PSC在单次前向传播下实现的不确定性量化和分布外检测性能,达到或超过需要训练修改的现有方法。通过改造现有模型,PSC可在不重训练前提下实现高质量的不确定性量化与分布外检测能力。
原文摘要 · Abstract (English)
Deterministic uncertainty quantification (UQ) in deep learning aims to estimate uncertainty with a single pass through a network by leveraging outputs from the network's feature extractor. Existing methods require that the feature extractor be both sensitive and smooth, ensuring meaningful input changes produce meaningful changes in feature vectors. Smoothness enables generalization, while sensitivity prevents feature collapse, where distinct inputs are mapped to identical feature vectors. To meet these requirements, current deterministic methods often retrain networks with spectral normalization. Instead of modifying training, we propose using measures of neural collapse to identify an existing intermediate layer that is both sensitive and smooth. We then fit a probabilistic model to the feature vector of this intermediate layer, which we call a probabilistic skip connection (PSC). Through empirical analysis, we explore the impact of spectral normalization on neural collapse and demonstrate that PSCs can effectively disentangle aleatoric and epistemic uncertainty. Additionally, we show that PSCs achieve uncertainty quantification and out-of-distribution (OOD) detection performance that matches or exceeds existing single-pass methods requiring training modifications. By retrofitting existing models, PSCs enable high-quality UQ and OOD capabilities without retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。