用一次校准集多次生成可靠预测，提升置信区间实用性
When Can We Reuse a Calibration Set for Multiple Conformal Predictions?
- 结合霍夫丁不等式设计可复用校准集的修正方法
- 在CIFAR-10上实现高概率覆盖，保证置信度可达95%
- 适合需要频繁预测且资源受限的实际场景
可靠的不确定性量化对机器学习应用的可信性至关重要。归纳式分位数预测（ICP）提供了一种无需分布假设的框架，可生成用户指定置信度的预测集合或区间。然而，标准ICP的保证是边际性的，通常需为每次新预测准备独立的校准集以维持有效性。本文通过将e-分位数预测与霍夫丁不等式结合，证明了可高概率地重复使用单一校准集。以CIFAR-10为例，我们训练深度神经网络并利用校准集估计霍夫丁修正项，进而应用改进的马尔可夫不等式构造具有可量化置信度的预测集。结果表明，在降低校准频率的同时仍能保持可证明的性能，显著提升了分位数预测的实用性。代码已公开。
原文摘要 · Abstract (English)
Reliable uncertainty quantification is crucial for the trustworthiness of machine learning applications. Inductive Conformal Prediction (ICP) offers a distribution-free framework for generating prediction sets or intervals with user-specified confidence. However, standard ICP guarantees are marginal and typically require a fresh calibration set for each new prediction to maintain their validity. This paper addresses this practical limitation by demonstrating how e-conformal prediction, in conjunction with Hoeffding's inequality, can enable the repeated use of a single calibration set with a high probability of preserving the desired coverage. Through a case study on the CIFAR-10 dataset, we train a deep neural network and utilise a calibration set to estimate a Hoeffding correction. This correction allows us to apply a modified Markov's inequality, leading to the construction of prediction sets with quantifiable confidence. Our results illustrate the feasibility of maintaining provable performance in conformal prediction while enhancing its practicality by reducing the need for repeated calibration. The code for this work is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。