arXiv:2503.02384cs.LG2025-03被引 14

提出新校准度量,同时保证决策合理性和报告真实概率的激励。

Truthfulness of Decision-Theoretic Calibration Measures

  • 引入子采样阶梯校准度量,兼顾决策合理性与真实性。
  • 在光滑设定下,真实度误差控制在√log(1/c)内,优于指数级差距。
  • 证明了非光滑场景下无法同时满足完全性与真理性,揭示理论极限。

校准度量用于衡量预测者偏差程度,要求预测值在给定预测概率下无偏。理想的校准度量需满足两个条件:一是下游决策者最优响应时无后悔(决策论意义),二是预测者通过报告真实概率可近似最小化误差(真实性)。现有度量仅满足其一,无法兼顾。本文提出新度量“子采样阶梯校准”(StepCE^sub),在任意乘积分布下,其真实性误差仅为O(1),而先前决策论型度量存在e^{-Ω(T)}-Ω(√T)的差距;在每个事件条件概率受噪声扰动幅度c>0的光滑设定下,其真实性误差为O(√log(1/c)),而旧方法差距达e^{-Ω(T)}-Ω(T^{1/3})。此外,我们证明了一个普遍不可能性结果:任何完整且具有决策论意义的校准度量,在非光滑设置中必不连续且不真实。

原文摘要 · Abstract (English)

Calibration measures quantify how much a forecaster's predictions violates calibration, which requires that forecasts are unbiased conditioning on the forecasted probabilities. Two important desiderata for a calibration measure are its decision-theoretic implications (i.e., downstream decision-makers that best-respond to the forecasts are always no-regret) and its truthfulness (i.e., a forecaster approximately minimizes error by always reporting the true probabilities). Existing measures satisfy at most one of the properties, but not both. We introduce a new calibration measure termed subsampled step calibration, $\mathsf{StepCE}^{\textsf{sub}}$, that is both decision-theoretic and truthful. In particular, on any product distribution, $\mathsf{StepCE}^{\textsf{sub}}$ is truthful up to an $O(1)$ factor whereas prior decision-theoretic calibration measures suffer from an $e^{-Ω(T)}$-$Ω(\sqrt{T})$ truthfulness gap. Moreover, in any smoothed setting where the conditional probability of each event is perturbed by a noise of magnitude $c > 0$, $\mathsf{StepCE}^{\textsf{sub}}$ is truthful up to an $O(\sqrt{\log(1/c)})$ factor, while prior decision-theoretic measures have an $e^{-Ω(T)}$-$Ω(T^{1/3})$ truthfulness gap. We also prove a general impossibility result for truthful decision-theoretic forecasting: any complete and decision-theoretic calibration measure must be discontinuous and non-truthful in the non-smoothed setting.

校准度量决策理论真实性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。