解决在线校准预测在标签噪声下的失效问题,提升实际覆盖率精度。
Exploring the Noise Robustness of Online Conformal Prediction
- 用鲁棒的pinball损失动态调整阈值,无需真实标签。
- 在均匀噪声下实现精确覆盖,误差收敛速度达T^{-1/2}。
- 适合标签不准确场景,如真实世界数据流中的不确定性量化。
校准预测是一种新兴的不确定性量化技术,可构建以预设概率包含真实标签的预测集。近期研究提出了自适应调整预测集的在线校准方法,以应对分布漂移。然而,现有算法通常假设标签完全准确,这在实践中极少成立。本文研究了在已知噪声率的均匀标签噪声下,不同学习率策略(恒定与动态)中在线校准预测的鲁棒性。结果表明,标签噪声会导致实际误覆盖率与目标率α之间存在持续偏差,造成覆盖保证过高或过低。为此,我们提出噪声鲁棒的在线校准预测(NR-OCP),通过新型鲁棒pinball损失更新阈值,无需真实标签即可无偏估计纯净pinball损失。理论分析显示,NR-OCP在两种学习率策略下均消除覆盖偏差,且经验与期望覆盖误差的收敛速度均为$\\(mathcal{O}(T^{-1/2})$。大量实验验证了该方法在精确覆盖与效率上的优越性。
原文摘要 · Abstract (English)
Conformal prediction is an emerging technique for uncertainty quantification that constructs prediction sets guaranteed to contain the true label with a predefined probability. Recent work develops online conformal prediction methods that adaptively construct prediction sets to accommodate distribution shifts. However, existing algorithms typically assume perfect label accuracy which rarely holds in practice. In this work, we investigate the robustness of online conformal prediction under uniform label noise with a known noise rate, in both constant and dynamic learning rate schedules. We show that label noise causes a persistent gap between the actual mis-coverage rate and the desired rate $α$, leading to either overestimated or underestimated coverage guarantees. To address this issue, we propose Noise Robust Online Conformal Prediction (dubbed NR-OCP) by updating the threshold with a novel robust pinball loss, which provides an unbiased estimate of clean pinball loss without requiring ground-truth labels. Our theoretical analysis shows that NR-OCP eliminates the coverage gap in both constant and dynamic learning rate schedules, achieving a convergence rate of $\mathcal{O}(T^{-1/2})$ for both empirical and expected coverage errors under uniform label noise. Extensive experiments demonstrate the effectiveness of our method by achieving both precise coverage and improved efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。