用区间概率压缩深度集成，提升推理效率同时保留不确定性量化能力
Credal Ensemble Distillation for Uncertainty Quantification
- 将深度集成压缩为单模型，输出类别概率区间而非单一值
- 在分布外检测任务上表现优于或相当主流方法，推理速度显著提升
- 适合需要高效可靠不确定估计的部署场景
深度集成（DE）已成为量化预测不确定性并区分随机性与认知性不确定性的有力方法，从而提升模型鲁棒性与可靠性。然而，其推理时的高计算与内存开销严重制约了实际应用。为此，我们提出可信集合蒸馏（CED），一种将深度集成压缩为单个模型CREDIT的新框架，适用于分类任务。CREDIT不再输出单一softmax概率分布，而是预测类别的概率区间，构成一个可信集——即概率分布的凸集，用于不确定性量化。在分布外检测基准上的实证结果表明，CED在不确定性估计方面优于或相当于多个现有基线方法，同时相比深度集成显著降低了推理开销。
原文摘要 · Abstract (English)
Deep ensembles (DE) have emerged as a powerful approach for quantifying predictive uncertainty and distinguishing its aleatoric and epistemic components, thereby enhancing model robustness and reliability. However, their high computational and memory costs during inference pose significant challenges for wide practical deployment. To overcome this issue, we propose credal ensemble distillation (CED), a novel framework that compresses a DE into a single model, CREDIT, for classification tasks. Instead of a single softmax probability distribution, CREDIT predicts class-wise probability intervals that define a credal set, a convex set of probability distributions, for uncertainty quantification. Empirical results on out-of-distribution detection benchmarks demonstrate that CED achieves superior or comparable uncertainty estimation compared to several existing baselines, while substantially reducing inference overhead compared to DE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。