校准能减少分类器预测分歧,提升公平性。
Mitigating the Multiplicity Burden: The Role of Calibration in Reducing Predictive Multiplicity of Classifiers
- 用后处理校准降低多个近优模型间的预测冲突。
- 少数类样本的预测分歧和置信度差异显著更大。
- Platt Scaling 和 Isotonic Regression 效果最好,适合高风险场景。
随着机器学习模型在高风险场景中的广泛应用,确保概率可靠性与预测稳定性至关重要。本文研究分类校准与预测多重性之间的关系——即在同一申请人上,多个近优模型(Rashomon集)产生不同信用结果的现象。基于九个不同的信用风险基准数据集,我们发现预测多重性集中在低置信度区域,且少数类样本承受了不成比例的多重性负担,其预测多重性与置信度差异显著。实验表明,采用后处理校准方法(Platt Scaling、Isotonic Regression、Temperature Scaling)可有效降低Rashomon集内的模糊性。其中,Platt Scaling和Isotonic Regression在减少预测多重性方面表现最稳健。结果表明,校准可作为共识强化层,有助于缓解算法任意性,支持程序公平性。
原文摘要 · Abstract (English)
As machine learning models are increasingly deployed in high-stakes environments, ensuring both probabilistic reliability and prediction stability has become critical. This paper examines the interplay between classification calibration and predictive multiplicity - the phenomenon in which multiple near-optimal models within the Rashomon set yield conflicting credit outcomes for the same applicant. Using nine diverse credit risk benchmark datasets, we investigate whether predictive multiplicity concentrates in regions of low predictive confidence and how post-hoc calibration can mitigate algorithmic arbitrariness. Our empirical analysis reveals that minority class observations bear a disproportionate multiplicity burden, as confirmed by significant disparities in predictive multiplicity and prediction confidence. Furthermore, our empirical comparisons indicate that applying post-hoc calibration methods - specifically Platt Scaling, Isotonic Regression, and Temperature Scaling - is associated with lower obscurity across the Rashomon set. Among the tested techniques, Platt Scaling and Isotonic Regression provide the most robust reduction in predictive multiplicity. These findings suggest that calibration can function as a consensus-enforcing layer and may support procedural fairness by mitigating predictive multiplicity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。