不重训练也不采样，用激活空间的高斯过程量化预训练模型不确定性。
Activation-Space Uncertainty Quantification for Pretrained Networks
- 将贝叶斯建模从权重转移到激活，保持原模型预测不变。
- 在多个任务上达到与强基线相当甚至更好的校准和分布外检测性能。
- 单次前向传播即可获得确定性不确定性估计，适合实际部署。
可靠的不确定性估计对部署预训练模型至关重要；然而，许多有效方法需要重训练、蒙特卡洛采样或昂贵的二阶计算，可能改变冻结主干的预测结果。为此，我们提出高斯过程激活（GAPA），一种后处理方法，将贝叶斯建模从权重转移到激活空间。GAPA用高斯过程激活替代标准非线性函数，其后验均值恰好匹配原始激活，从而在构造上保留主干的点预测，同时在激活空间提供闭式认知不确定性方差。为适配现代架构，我们采用基于缓存训练激活的稀疏变分诱导点近似，并结合局部k近邻子集条件化，实现无需采样、反向传播或二阶信息的确定性单次前向传播不确定性推断。在回归、分类、图像分割和语言建模任务中，GAPA在校准和分布外检测方面表现与强基线相当或更优，且推理效率高。
原文摘要 · Abstract (English)
Reliable uncertainty estimates are crucial for deploying pretrained models; yet, many strong methods for quantifying uncertainty require retraining, Monte Carlo sampling, or expensive second-order computations and may alter a frozen backbone's predictions. To address this, we introduce Gaussian Process Activations (GAPA), a post-hoc method that shifts Bayesian modeling from weights to activations. GAPA replaces standard nonlinearities with Gaussian-process activations whose posterior mean exactly matches the original activation, preserving the backbone's point predictions by construction while providing closed-form epistemic variances in activation space. To scale to modern architectures, we use a sparse variational inducing-point approximation over cached training activations, combined with local k-nearest-neighbor subset conditioning, enabling deterministic single-pass uncertainty propagation without sampling, backpropagation, or second-order information. Across regression, classification, image segmentation, and language modeling, GAPA matches or outperforms strong post-hoc baselines in calibration and out-of-distribution detection while remaining efficient at test time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。