统一评估不确定性的预测模型，可灵活权衡准确与精确。
A Unified Evaluation Framework for Epistemic Predictions
- 设计新评估框架，适配多种不确定性模型
- 在多个数据集上验证指标有效且行为符合预期
- 适合需要定制化精度-准确率权衡的应用场景
不确定性感知模型的预测形式多样，包括单点估计（常对预测样本取平均）、预测分布、集合值或可信集表示。本文提出一种适用于广泛模型类别的统一评估框架，通过设计合适的性能度量，使用户能根据实际需求灵活调整预测的准确率与精确度之间的权衡。该框架支持针对特定应用场景选择最合适的模型。在CIFAR-10、MNIST和CIFAR-100数据集上，对贝叶斯、集成、证据理论、确定性、可信集及信念函数分类器的实验表明，该度量表现符合预期。
原文摘要 · Abstract (English)
Predictions of uncertainty-aware models are diverse, ranging from single point estimates (often averaged over prediction samples) to predictive distributions, to set-valued or credal-set representations. We propose a novel unified evaluation framework for uncertainty-aware classifiers, applicable to a wide range of model classes, which allows users to tailor the trade-off between accuracy and precision of predictions via a suitably designed performance metric. This makes possible the selection of the most suitable model for a particular real-world application as a function of the desired trade-off. Our experiments, concerning Bayesian, ensemble, evidential, deterministic, credal and belief function classifiers on the CIFAR-10, MNIST and CIFAR-100 datasets, show that the metric behaves as desired.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。