揭示三类任务在数据分布偏移下的等价性，推动跨领域方法融合。
On the Interconnections of Calibration, Quantification, and Classifier Accuracy Prediction under Dataset Shift
- 通过相互归约证明校准、量化与准确率预测三者等价
- 基于跨领域方法迁移，性能常优于专用方法
- 适合关注分布偏移下模型可靠性研究的学者
当训练数据与测试数据的分布不一致（即存在数据集偏移）时，对分类器决策分数进行校准、估计测试样本中正例比例，或预测分类器准确率等经典方法面临严峻挑战。本文系统研究了这三项基础任务在数据集偏移条件下的内在关联。具体而言，我们通过相互归约的方式证明了三者等价——即只要掌握其中任一任务的最优解（预言机），即可解决其余两项。基于此理论结果，我们提出了针对每项任务的新方法，直接借鉴并适配其他领域的成熟技术。实验表明，这些跨领域迁移的方法性能通常具有竞争力，甚至在某些情况下超越各自领域的专用方法。论文旨在促进这三个研究方向之间的交叉融合，推动统一框架的发展，并加强各领域间的协同创新。
原文摘要 · Abstract (English)
When the distribution of the data used to train a classifier differs from that of the test data, i.e., under dataset shift, well-established routines for calibrating the decision scores of the classifier, estimating the proportion of positives in a test sample, or estimating the accuracy of the classifier, become particularly challenging. This paper investigates the interconnections among three fundamental problems, calibration, quantification, and classifier accuracy prediction, under dataset shift conditions. Specifically, we prove their equivalence through mutual reduction, i.e., we show that access to an oracle for any one of these tasks enables the resolution of the other two. Based on these proofs, we propose new methods for each problem based on direct adaptations of well-established methods borrowed from the other disciplines. Our results show such methods are often competitive, and sometimes even surpass the performance of dedicated approaches from each discipline. The main goal of this paper is to fostering cross-fertilization among these research areas, encouraging the development of unified approaches and promoting synergies across the fields.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。