arXiv:2410.00393cs.LGcs.AI2024-10TPAMI被引 27

改进不确定性估计方法,让模型更准确地判断自己有多自信。

Revisiting Essential and Nonessential Settings of Evidential Deep Learning

  • 重新设计了证据深度学习的非必要设置,只保留核心的概率投影机制。
  • 在多个数据集上实现更强的不确定性估计性能,避免过度自信问题。
  • 适合关注模型可信度、需要可靠置信度输出的研究者使用。

证据深度学习(EDL)是一种新兴的不确定性估计方法,可在单次前向传播中提供可靠的预测不确定性,受到广泛关注。其基于主观逻辑,通过神经网络生成狄利克雷分布的浓度参数,构建类概率分布。然而,现有方法包含若干非必要设置:在模型构建中,常忽略先验权重参数并固定为类别数,但其值影响证据比例与量级的平衡;在优化过程中,经验风险包含方差最小化项,使分布趋向狄拉克δ函数,可能加剧过自信;此外,结构风险通常包含KL散度最小化正则项,其优化方向偏离初衷且违背常识,削弱了证据量级的信息。为此,本文提出Re-EDL,通过放松非必要设置、仅保留关键的主观逻辑投影概率机制,将先验权重设为可调超参数,并直接优化狄利克雷分布的期望,摒弃方差最小化项与散度正则项。大量实验验证了方法的有效性,达到当前最优性能。源代码已公开于https://github.com/MengyuanChen21/Re-EDL。

原文摘要 · Abstract (English)

Evidential Deep Learning (EDL) is an emerging method for uncertainty estimation that provides reliable predictive uncertainty in a single forward pass, attracting significant attention. Grounded in subjective logic, EDL derives Dirichlet concentration parameters from neural networks to construct a Dirichlet probability density function (PDF), modeling the distribution of class probabilities. Despite its success, EDL incorporates several nonessential settings: In model construction, (1) a commonly ignored prior weight parameter is fixed to the number of classes, while its value actually impacts the balance between the proportion of evidence and its magnitude in deriving predictive scores. In model optimization, (2) the empirical risk features a variance-minimizing optimization term that biases the PDF towards a Dirac delta function, potentially exacerbating overconfidence. (3) Additionally, the structural risk typically includes a KL-divergence-minimizing regularization, whose optimization direction extends beyond the intended purpose and contradicts common sense, diminishing the information carried by the evidence magnitude. Therefore, we propose Re-EDL, a simplified yet more effective variant of EDL, by relaxing the nonessential settings and retaining the essential one, namely, the adoption of projected probability from subjective logic. Specifically, Re-EDL treats the prior weight as an adjustable hyperparameter rather than a fixed scalar, and directly optimizes the expectation of the Dirichlet PDF provided by deprecating both the variance-minimizing optimization term and the divergence regularization term. Extensive experiments and state-of-the-art performance validate the effectiveness of our method. The source code is available at https://github.com/MengyuanChen21/Re-EDL.

不确定性估计深度学习狄利克雷分布

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。