用校准后的预测得分提升小样本半监督均值估计的准确性与效率
Calibeating Prediction-Powered Inference
- 对预测分数进行事后校准,无需重新训练模型
- 校准后预测性能和估计效率显著优于原始分数
- 适合有预训练模型但预测不准的半监督学习场景
在小量标注数据、大量无标注数据及黑箱预测模型存在校准偏差的设定下,研究半监督均值估计问题。标准方法AIPW能抵御模型误设,但在预测分数与结果尺度不匹配时效率较低。本文提出校准预测驱动推断(Calibrated Prediction-Powered Inference),在标注集上事后校准预测分数,无需重训练即可同时提升预测能力与半监督估计效率。研究线性与等距校准,证明等距校准具有一阶最优性:其性能优于原始分数和简单后处理,且进一步后处理无一阶收益。线性校准与PPI++一阶等价。厘清了现有估计器关系:原PPI是AIPW特例,当模型准确时可能低效;而PPI++是经验效率最大化的AIPW。模拟与真实数据实验表明,校准方法常优于PPI,媲美或超越AIPW与PPI++。配套Python包ppi_aipw已发布。
原文摘要 · Abstract (English)
We study semisupervised mean estimation with a small labeled sample, a large unlabeled sample, and a black-box prediction model whose output may be miscalibrated. A standard approach in this setting is augmented inverse-probability weighting (AIPW) [Robins et al., 1994], which protects against prediction-model misspecification but can be inefficient when the prediction score is poorly aligned with the outcome scale. We introduce Calibrated Prediction-Powered Inference, which post-hoc calibrates the prediction score on the labeled sample before using it for semisupervised estimation. This simple step requires no retraining and can improve the original score both as a predictor of the outcome and as a regression adjustment for semisupervised inference. We study both linear and isotonic calibration. For isotonic calibration, we establish first-order optimality guarantees: isotonic post-processing can improve predictive accuracy and estimator efficiency relative to the original score and simpler post-processing rules, while no further post-processing of the fitted isotonic score yields additional first-order gains. For linear calibration, we show first-order equivalence to PPI++. We also clarify the relationship among existing estimators, showing that the original PPI estimator is a special case of AIPW and can be inefficient when the prediction model is accurate, while PPI++ is AIPW with empirical efficiency maximization [Rubin et al., 2008]. In simulations and real-data experiments, our calibrated estimators often outperform PPI and are competitive with, or outperform, AIPW and PPI++. We provide an accompanying Python package, ppi_aipw, at https://larsvanderlaan.github.io/ppi-aipw/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。