用可信集度量预测不确定性,让模型知道它有多不确定。
Quantifying Epistemic Predictive Uncertainty in Conformal Prediction
- 基于共形预测构建可信集,揭示多种可能模型的冲突信息
- 提出最大均值模糊度度量,量化认知不确定性,精度更高
- 适合主动学习与选择性分类场景,提升决策可靠性
本文研究共形预测框架下认知预测不确定性(EPU)的量化问题——即因存在多个合理预测模型而带来的预测时不确定性。通过近期结果可知,在弱假设下,任何完整的共形预测过程都会诱导出一组闭合凸的预测分布,称为可信集。重要的是,共形预测区域(CPR)恰好等于所有分布赋予概率不低于 $1-α$ 的标签集合。本文首次证明该性质在分拆共形预测(split CP)中同样成立。基于此,我们提出一种计算高效且解析可处理的不确定性度量——最大均值模糊度(Maximum Mean Imprecision),用于衡量可信集中信息冲突程度,从而量化EPU。在主动学习和选择性分类实验中,所提度量提供的不确定性评估远比仅依赖CPR大小更细致、更有效。本工作凸显了共形预测作为认知不确定性下决策基础的潜力。
原文摘要 · Abstract (English)
We study the problem of quantifying epistemic predictive uncertainty (EPU) -- that is, uncertainty faced at prediction time due to the existence of multiple plausible predictive models -- within the framework of conformal prediction (CP). To expose the implicit model multiplicity underlying CP, we build on recent results showing that, under a mild assumption, any full CP procedure induces a set of closed and convex predictive distributions, commonly referred to as a credal set. Importantly, the conformal prediction region (CPR) coincides exactly with the set of labels to which all distributions in the induced credal set assign probability at least $1-α$. As our first contribution, we prove that this characterisation also holds in split CP. Building on this connection, we then propose a computationally efficient and analytically tractable uncertainty measure, based on \emph{Maximum Mean Imprecision}, to quantify the EPU by measuring the degree of conflicting information within the induced credal set. Experiments on active learning and selective classification demonstrate that the quantified EPU provides substantially more informative and fine-grained uncertainty assessments than reliance on CPR size alone. More broadly, this work highlights the potential of CP serving as a principled basis for decision-making under epistemic uncertainty.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。