无需预留数据,用牛顿影响法逼近神经网络回归的置信区间。
Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influence
- 用高斯-牛顿影响局部扰动参数,近似重训练效果。
- 在标准回归和定位任务上,区间更紧且自适应。
- 适合对不确定性量化有要求的部署场景。
不确定性量化是深度学习模型在安全关键领域应用的重要前提。然而,这依赖于预测区间的校准性和紧凑性。对于仅输出点估计的预训练模型,常用后处理方法包括拉普拉斯法和分拆交叉验证(split-CP)。但拉普拉斯法在模型误设时可能校准不准,而 split-CP 需要样本分割,牺牲统计效率。本文提出一种无需预留数据的神经网络回归后处理方法,通过近似全交叉验证(full-CP)实现。尽管 full-CP 理论上需为每个测试点重新训练模型,我们仅训练一次,利用高斯-牛顿影响局部扰动参数以近似重训练效果,并结合网络线性化,将绝对残差非共形得分表示为候选标签的分段线性函数,从而避免对输出空间的穷举搜索。在标准回归基准和边界框定位任务上,所得预测区间具有局部自适应性,通常比 split-CP 更紧。
原文摘要 · Abstract (English)
Uncertainty quantification is an important prerequisite for the deployment of deep learning models in safety-critical areas. Yet, this hinges on the uncertainty estimates being useful to the extent the prediction intervals are well-calibrated and sharp. In the absence of inherent uncertainty estimates (e.g. pretrained models predicting only point estimates), popular approaches that operate post-hoc include Laplace's method and split conformal prediction (split-CP). However, Laplace's method can be miscalibrated when the model is misspecified and split-CP requires sample splitting, and thus comes at the expense of statistical efficiency. In this work, we construct prediction intervals for neural network regressors post-hoc without held-out data. This is achieved by approximating the full conformal prediction method (full-CP). Whilst full-CP nominally requires retraining the model for every test point and candidate label, we propose to train just once and locally perturb model parameters using Gauss-Newton influence to approximate the effect of retraining. Coupled with linearization of the network, we express the absolute residual nonconformity score as a piecewise linear function of the candidate label allowing for an efficient procedure that avoids the exhaustive search over the output space. On standard regression benchmarks and bounding box localization, we show the resulting prediction intervals are locally-adaptive and often tighter than those of split-CP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。