用简单损失函数实现可靠不确定性估计,兼容softmax分类器。
Plug-in Losses for Evidential Deep Learning: A Simplified Framework for Uncertainty Estimation that Includes the Softmax Classifier
- 用狄利克雷分布均值替代复杂目标,简化计算
- 在语音命令数据集上达到与经典EDL相当的准确率和选择性预测性能
- 首次在语音识别中实现覆盖-精度权衡,支持软最大分类器
真实世界的基于传感器的学习系统需要既可靠又计算高效的不确定性估计。证据深度学习(EDL)通过将类别概率建模为狄利克雷分布,实现了单次前向传播的不确定性估计,其中狄利克雷参数由神经网络预测。然而,该方法存在计算挑战,因为狄利克雷期望目标比标准监督学习损失更复杂,增加了分析和实现难度。本文通过在狄利克雷均值处评估的插件损失近似第一阶经验风险最小化问题的目标,解决了这一问题,并证明在温和假设下,近似误差随证据增长而衰减,适用于均方误差和交叉熵等广泛损失函数。作为特例,我们的分析为在不确定性估计中使用softmax提供了理论依据,因为在特定证据到狄利克雷映射下,该框架包含了标准softmax分类器。我们在Google Speech Commands数据集上验证了所提出的简化目标,结果表明其在预测准确率和选择性预测性能上与经典EDL相当,同时使用标准深度学习损失和训练流程更易实现。据我们所知,这是首个在语音识别任务中通过EDL获得覆盖率-准确性权衡的实证分析。
原文摘要 · Abstract (English)
Real-world sensor-based learning systems require uncertainty estimation that is both reliable and computationally efficient. Evidential Deep Learning (EDL) provides single-pass uncertainty estimation by modeling the class probabilities via Dirichlet distributions, where the Dirichlet parameters are predicted by a learned neural network mapping. However, this approach can lead to computational challenges, as Dirichlet expected objectives are more complex than standard supervised learning losses, complicating their analysis and implementation. We address this issue by approximating the objective of the first-order empirical risk minimization problem induced by EDL with a plug-in loss evaluated at the Dirichlet mean and show that, under mild assumptions, the approximation error decays with growing evidence for a broad class of loss functions, including mean-squared error and cross-entropy loss. As a special case, our analysis provides justification for the use of softmax in the context of uncertainty estimation, since under a particular evidence-to-Dirichlet mapping, our framework includes the standard softmax classifier. We validate the proposed simplified objectives on the Google Speech Commands dataset and show that they achieve predictive accuracy and selective prediction performance comparable to classical EDL, while being simpler to implement using standard deep learning losses and training pipelines. To the best of our knowledge, this empirical analysis is the first to obtain coverage-accuracy trade-offs for speech recognition tasks through EDL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。