用核方法提升预测带的自适应性,兼顾覆盖率与计算效率。
Scalable and adaptive prediction bands with kernel sum-of-squares

- 将置信预测转为核空间学习问题,利用核平方和方法建模。
- 支持数百至数千样本高效求解,显著优于传统半定规划方法。
- 提出基于HSIC的新超参调优策略,提升条件覆盖自适应性,适用广泛。
置信预测(Conformal Prediction, CP)是一种在有限样本下构建具有有效覆盖率的预测带的方法,且无需任何分布假设。其主要局限在于缺乏自适应性,尽管已有若干高效替代方案。本文基于将CP问题重构为统计学习问题的新思路,直接优化覆盖率与自适应性。该学习问题基于可重复再生核希尔伯特空间(RKHS)与核平方和(SoS)方法。首先,我们推导出通用表示定理并给出问题的对偶形式;关键的是,该对偶形式可通过加速梯度法在数百至数千样本下高效求解,优于依赖现成半定规划算法的旧方法。其次,我们提出一种专为提升自适应性设计的超参数调优策略,通过测试条件覆盖率的界进行优化。该策略基于希尔伯特-施密特独立性准则(HSIC),用于调节核长度尺度,在本框架中引入,但具有更广适用性——可用于任何得分函数由学习获得的CP算法。最后,大量实验验证了方法性能,并附带可复现代码。
原文摘要 · Abstract (English)
Conformal Prediction (CP) is a popular framework for constructing prediction bands with valid coverage in finite samples, while being free of any distributional assumption. A well-known limitation of conformal prediction is the lack of adaptivity, although several works introduced practically efficient alternate procedures. In this work, we build upon recent ideas that rely on recasting the CP problem as a statistical learning problem, directly targeting coverage and adaptivity. This statistical learning problem is based on reproducible kernel Hilbert spaces (RKHS) and kernel sum-of-squares (SoS) methods. First, we extend previous results with a general representer theorem and exhibit the dual formulation of the learning problem. Crucially, such dual formulation can be solved efficiently by accelerated gradient methods with several hundreds or thousands of samples, unlike previous strategies based on off-the-shelf semidefinite programming algorithms. Second, we introduce a new hyperparameter tuning strategy tailored specifically to target adaptivity through bounds on test-conditional coverage. This strategy, based on the Hilbert-Schmidt Independence Criterion (HSIC), is introduced here to tune kernel lengthscales in our framework, but has broader applicability since it could be used in any CP algorithm where the score function is learned. Finally, extensive experiments are conducted to show how our method compares to related work. All figures can be reproduced with the accompanying code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。