拆解预测置信度，揭示可靠性、信息损失与不确定性来源
Decomposing Probabilistic Scores: Reliability, Information Loss and Uncertainty
- 将任意合理损失分解为可靠性、剩余不确定性和信息损失三项
- 发现特征到得分的映射会引入可量化的信息损失项
- 适用于校准、模型融合和渐进式建模分析，理论严谨
校准是依赖于预测器保留信息程度的条件属性。我们推导出任意合理损失的分解恒等式，明确揭示了这种依赖关系。在任意信息层级 𝒜 下，𝒜-可测预测器的期望损失可分解为合理损失偏差（可靠性）项和条件熵（残余不确定性）项。对于嵌套层级 𝒜⊆ℬ，链式分解可量化从 𝒜 到 ℬ 的信息增益。应用于具有特征 𝑋 和得分 𝑆=𝑠(𝑋) 的分类任务时，得到三项恒等式：偏差校准项、衡量从 𝑋 到 𝑆 信息损失的‘分组’项，以及特征层面的不可约不确定性。该框架可用于分析后处理校准、已校准模型聚合及分阶段/提升构造，对 Brier 损失和对数损失给出了显式表达式。
原文摘要 · Abstract (English)
Calibration is a conditional property that depends on the information retained by a predictor. We develop decomposition identities for arbitrary proper losses that make this dependence explicit. At any information level $\mathcal A$, the expected loss of an $\mathcal A$-measurable predictor splits into a proper-regret (reliability) term and a conditional entropy (residual uncertainty) term. For nested levels $\mathcal A\subseteq\mathcal B$, a chain decomposition quantifies the information gain from $\mathcal A$ to $\mathcal B$. Applied to classification with features $\boldsymbol{X}$ and score $S=s(\boldsymbol{X})$, this yields a three-term identity: miscalibration, a {\em grouping} term measuring information loss from $\boldsymbol{X}$ to $S$, and irreducible uncertainty at the feature level. We leverage the framework to analyze post-hoc recalibration, aggregation of calibrated models, and stagewise/boosting constructions, with explicit forms for Brier and log-loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。