用可能性理论解决深度学习预测不确定性的过拟合问题
Possibilistic Predictive Uncertainty for Deep Learning
- 基于可能性理论构建参数后验，通过上确界映射到预测空间
- 采用可学习的狄利克雷函数近似后验，训练目标有闭式解
- 在多个基准上表现优于现有方法，兼具理论严谨与计算高效
深度神经网络在各类应用中表现优异，但在未见输入上常表现出过度自信，亟需可靠的认知不确定性建模。现有方法面临根本矛盾：贝叶斯方法虽原理严谨但计算成本高，而高效的二阶预测器缺乏与认知不确定性量化之间的严格关联。为此，我们提出狄利克雷近似可能性后验预测（DAPPr），一个基于可能性理论的原理性框架。通过定义参数上的可能性后验，利用上确界算子将其投影至预测空间，并以可学习的狄利克雷可能性函数近似投影后的后验。该投影-近似策略带来简洁的训练目标和闭式解。尽管结构简单，大量实验表明，DAPPr在多个基准上达到或超过先进二阶预测器的不确定性量化性能，同时保持理论严谨性和计算效率。代码已开源：https://github.com/MaxwellYaoNi/DAPPr。
原文摘要 · Abstract (English)
Deep neural networks achieve impressive results across diverse applications, yet their overconfidence on unseen inputs necessitates reliable epistemic uncertainty modeling. Existing methods for uncertainty modeling face a fundamental dilemma: Bayesian approaches provide principled estimates but remain computationally prohibitive, while efficient second-order predictors lack rigorous connections between their specific objectives and epistemic uncertainty quantification. To resolve this dilemma, we introduce Dirichlet-approximated possibilistic posterior predictions (DAPPr), a principled framework grounded in possibility theory. We define a possibilistic posterior over parameters, project it to the prediction space via supremum operators, and approximate the projected posterior using learnable Dirichlet possibility functions. This projection-and-approximation strategy yields a simple training objective with closed-form solutions. Despite its simplicity, extensive experiments across diverse benchmarks show that DAPPr achieves competitive or superior uncertainty quantification performance over state-of-the-art second-order predictors while maintaining both principled derivation and computational efficiency. Code is available at https://github.com/MaxwellYaoNi/DAPPr.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。