arXiv:2604.06032stat.MLcs.LG2026-04

用集成模型估算不确定性,提升分类可靠性。

Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification

论文配图:Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification
图 1 · 摘自论文原文
  • 通过软输出均值估计狄利克雷参数,避开敏感的损失设计。
  • 在多个数据集上显著降低预测置信度波动,提升稳定性。
  • 适合需要可靠不确定性的下游任务,如拒绝低置信预测。

以交叉熵训练的神经网络分类器虽有高准确率,但缺乏内在的预测不确定性估计,需依赖外部方法。此外,单次训练中真实类别对应的Softmax分数在不同运行间波动大,影响基于不确定性的下游决策可靠性。证据深度学习虽能在单次前向传播中生成不确定性估计,但其训练对损失形式、先验正则化和激活函数等设计高度敏感。为此,本文提出一种基于集成的狄利克雷参数估计策略:对多个模型的Softmax输出取均值,采用矩估计法推导狄利克雷分布参数,并可选地加入最大似然精修步骤。该方法将不确定性估计与脆弱的证据损失设计解耦,同时缓解单次交叉熵训练的不稳定性,生成明确的狄利克雷预测分布。在多个数据集上,这些由集成推导出的狄利克雷估计表现出更强的稳定性和更优的不确定性行为,在预测置信度评分和选择性分类等下游任务中表现更优。

原文摘要 · Abstract (English)

Neural network classifiers trained with cross-entropy loss achieve strong predictive accuracy but lack the capability to provide inherent predictive uncertainty estimates, thus requiring external techniques to obtain these estimates. In addition, softmax scores for the true class can vary substantially across independent training runs, which limits the reliability of uncertainty-based decisions in downstream tasks. Evidential Deep Learning aims to address these limitations by producing uncertainty estimates in a single pass, but evidential training is highly sensitive to design choices including loss formulation, prior regularization, and activation functions. Therefore, this work introduces an alternative Dirichlet parameter estimation strategy by applying a method of moments estimator to ensembles of softmax outputs, with an optional maximum-likelihood refinement step. This ensemble-based construction decouples uncertainty estimation from the fragile evidential loss design while also mitigating the variability of single-run cross-entropy training, producing explicit Dirichlet predictive distributions. Across multiple datasets, we show that the improved stability and predictive uncertainty behavior of these ensemble-derived Dirichlet estimates translate into stronger performance in downstream uncertainty-guided applications such as prediction confidence scoring and selective classification.

不确定性估计集成学习选择性分类狄利克雷分布

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。