提出新评估方法,提升非平稳数据流的在线预测可靠性。
Optimal training-conditional regret for online conformal prediction
- 用训练条件累积损失评估在线校准,更贴合实际场景
- 针对突变与渐变两种漂移,分别设计自适应校准算法
- 理论证明最优性能,适合动态环境下的在线学习应用
我们研究非平稳数据流下的在线共形预测问题,面对未知分布漂移。不同于以往在对抗设定下以时间平均覆盖率差距评估的方法,本文采用训练条件下的累积后悔值作为评价标准。针对独立生成的数据及两类分布漂移(突变点与平滑漂移),当非一致性评分函数在独立数据集上预训练时,提出一种基于分治风格的算法,结合漂移检测动态更新校准集,可实现最小最大最优后悔率。当非一致性评分在线训练时,开发了一种全共形风格算法,利用漂移检测应对非平稳性;该方法依赖模型拟合算法的稳定性而非置换对称性,更适合演化环境中的在线学习。建立了该算法的非渐近后悔界,在适当条件下达到最小最大下界。数值实验验证了理论结果。
原文摘要 · Abstract (English)
We study online conformal prediction for non-stationary data streams subject to unknown distribution drift. While most prior work studied this problem under adversarial settings and/or assessed performance in terms of gaps of time-averaged marginal coverage, we instead evaluate performance through training-conditional cumulative regret. We specifically focus on independently generated data with two types of distribution shift: abrupt change points and smooth drift. When non-conformity score functions are pretrained on an independent dataset, we propose a split-conformal style algorithm that leverages drift detection to adaptively update calibration sets, which provably achieves minimax-optimal regret. When non-conformity scores are instead trained online, we develop a full-conformal style algorithm that again incorporates drift detection to handle non-stationarity; this approach relies on stability - rather than permutation symmetry - of the model-fitting algorithm, which is often better suited to online learning under evolving environments. We establish non-asymptotic regret guarantees for our online full conformal algorithm, which match the minimax lower bound under appropriate restrictions on the prediction sets. Numerical experiments corroborate our theoretical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。