提出角度校准法,在高维下实现可证明的精准概率输出。
Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling
- 基于估计权重与真实权重的夹角构造校准预测器
- 在高维样本与特征同阶增长时,校准误差趋于零
- 首次证明校准与最优性双重性质,适合高维分类场景
我们研究线性二分类器 $σ(\hat{w}^\top x)$ 的校准问题,其中特征 $x$ 为高斯分布,$σ$ 为链接函数,$\hat{w}$ 是真实权重 $w^\star$ 的估计。通过与一个非信息性的‘随机分类器’插值,构建了一个依赖于 $\hat{w}$ 与 $w^\star$ 夹角 $\angle(\hat{w}, w_\star)$ 的校准预测器。在样本数与特征数同阶发散的高维情形下,该角度校准方法被证明是严格校准的。夹角可一致估计,且所得预测器在合理校准类中唯一达到贝格曼最优(Bregman-optimal),最小化对真实标签分布的贝格曼散度。本工作首次在高维下同时提供可证明的校准与最优性。此外,我们揭示了经典Platt校准在特定条件下收敛至该最优解,因此其在高维下也具备可证明的良好性质。
原文摘要 · Abstract (English)
We study the fundamental problem of calibrating a linear binary classifier of the form $σ(\hat{w}^\top x)$, where the feature vector $x$ is Gaussian, $σ$ is a link function, and $\hat{w}$ is an estimator of the true linear weight $w^\star$. By interpolating with a noninformative $\textit{chance classifier}$, we construct a well-calibrated predictor whose interpolation weight depends on the angle $\angle(\hat{w}, w_\star)$ between the estimator $\hat{w}$ and the true linear weight $w_\star$. We establish that this angular calibration approach is provably well-calibrated in a high-dimensional regime where the number of samples and features both diverge, at a comparable rate. The angle $\angle(\hat{w}, w_\star)$ can be consistently estimated. Furthermore, the resulting predictor is uniquely $\textit{Bregman-optimal}$, minimizing the Bregman divergence to the true label distribution within a suitable class of calibrated predictors. Our work is the first to provide a calibration strategy that satisfies both calibration and optimality properties provably in high dimensions. Additionally, we identify conditions under which a classical Platt-scaling predictor converges to our Bregman-optimal calibrated solution. Thus, Platt-scaling also inherits these desirable properties provably in high dimensions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。