用无标签数据提升预测置信集的风险控制精度
Semi-Supervised Risk Control via Prediction-Powered Inference
- 基于预测驱动推断，融合无标签数据优化校准超参数
- 在少样本图像和时间序列分类中显著降低保守性误差率
- 适合需要严格错误率控制的医疗、金融等高风险场景
风险控制预测集（RCPS）框架可将任意机器学习模型输出转化为具有严格错误率控制的预测规则。其核心思想是利用有标签的保留校准数据调整影响错误率的超参数。然而，当保留数据量有限时，该超参数估计噪声大，导致预测规则过于保守。为突破样本量限制，本文提出一种半监督校准方法，通过利用无标签数据严格调节超参数，同时保持统计有效性。该方法基于预测驱动推断框架，针对风险控制任务进行定制化设计。在两个真实数据实验中验证了其有效性：少样本图像分类与早期时间序列分类。
原文摘要 · Abstract (English)
The risk-controlling prediction sets (RCPS) framework is a general tool for transforming the output of any machine learning model to design a predictive rule with rigorous error rate control. The key idea behind this framework is to use labeled hold-out calibration data to tune a hyper-parameter that affects the error rate of the resulting prediction rule. However, the limitation of such a calibration scheme is that with limited hold-out data, the tuned hyper-parameter becomes noisy and leads to a prediction rule with an error rate that is often unnecessarily conservative. To overcome this sample-size barrier, we introduce a semi-supervised calibration procedure that leverages unlabeled data to rigorously tune the hyper-parameter without compromising statistical validity. Our procedure builds upon the prediction-powered inference framework, carefully tailoring it to risk-controlling tasks. We demonstrate the benefits and validity of our proposal through two real-data experiments: few-shot image classification and early time series classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。