让机器学习预测更可靠,缩小不确定区间。
Selective Conformal Risk Control
- 分两阶段筛选可信样本并控制风险,提升预测精度。
- 在两个数据集上达到目标覆盖和风险水平,效果相近。
- 计算高效且保证可靠性,适合高风险应用。
可靠的风险量化对高风险领域部署机器学习系统至关重要。分位数预测可提供无需分布假设的覆盖率保证,但常生成过大的预测集,限制了实际应用。为此,我们提出选择性共形风险控制(SCRC),将共形预测与选择性分类结合,将不确定性控制建模为两阶段问题:第一阶段筛选置信度高的样本,第二阶段在选中的子集上应用共形风险控制以构建校准的预测集。我们提出了两种算法:SCRC-T通过联合计算校准与测试样本的阈值,保持交换性,提供精确的有限样本保证;SCRC-I仅依赖校准,提供类似PAC的概率保证,计算效率更高。在两个公开数据集上的实验表明,两种方法均实现目标覆盖率和风险水平,性能相近;其中SCRC-I风险控制略保守,但计算实用性更优。结果表明,选择性共形风险控制为紧凑、可靠的不确定性量化提供了有效且高效的路径。
原文摘要 · Abstract (English)
Reliable uncertainty quantification is essential for deploying machine learning systems in high-stakes domains. Conformal prediction provides distribution-free coverage guarantees but often produces overly large prediction sets, limiting its practical utility. To address this issue, we propose \textit{Selective Conformal Risk Control} (SCRC), a unified framework that integrates conformal prediction with selective classification. The framework formulates uncertainty control as a two-stage problem: the first stage selects confident samples for prediction, and the second stage applies conformal risk control on the selected subset to construct calibrated prediction sets. We develop two algorithms under this framework. The first, SCRC-T, preserves exchangeability by computing thresholds jointly over calibration and test samples, offering exact finite-sample guarantees. The second, SCRC-I, is a calibration-only variant that provides PAC-style probabilistic guarantees while being more computational efficient. Experiments on two public datasets show that both methods achieve the target coverage and risk levels, with nearly identical performance, while SCRC-I exhibits slightly more conservative risk control but superior computational practicality. Our results demonstrate that selective conformal risk control offers an effective and efficient path toward compact, reliable uncertainty quantification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。