给预训练模型加个轻量模块，就能事后估算不确定度。
Evidential Transformation Network: Turning Pretrained Models into Evidential Models for Post-hoc Uncertainty Estimation
- 在logit空间做自适应变换，将输出转为狄利克雷分布参数
- 在图像和语言任务上显著提升不确定性估计效果
- 仅增加少量计算开销，适合实际部署
预训练模型在视觉和语言任务中已成为标准，但通常无法提供可靠的置信度度量。现有不确定性估计方法如深度集成和MC dropout往往计算开销过大，难以实用。证据深度学习（EDL）虽更高效，但要求模型从训练起就输出证据量，这在预训练网络中几乎不可能实现。为此，我们提出轻量级后处理模块Evidential Transformation Network（ETN），可将预训练预测器转化为证据模型。ETN在logit空间学习样本相关的仿射变换，并将变换后的输出解释为狄利克雷分布参数以进行不确定性估计。我们在图像分类和大语言模型问答任务上，在分布内与分布外设置下评估ETN，结果表明其持续优于后处理基线，同时保持准确率并仅引入极小计算开销。
原文摘要 · Abstract (English)
Pretrained models have become standard in both vision and language, yet they typically do not provide reliable measures of confidence. Existing uncertainty estimation methods, such as deep ensembles and MC dropout, are often too computationally expensive to deploy in practice. Evidential Deep Learning (EDL) offers a more efficient alternative, but it requires models to be trained to output evidential quantities from the start, which is rarely true for pretrained networks. To enable EDL-style uncertainty estimation in pretrained models, we propose the Evidential Transformation Network (ETN), a lightweight post-hoc module that converts a pretrained predictor into an evidential model. ETN operates in logit space: it learns a sample-dependent affine transformation of the logits and interprets the transformed outputs as parameters of a Dirichlet distribution for uncertainty estimation. We evaluate ETN on image classification and large language model question-answering benchmarks under both in-distribution and out-of-distribution settings. ETN consistently improves uncertainty estimation over post-hoc baselines while preserving accuracy and adding only minimal computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。