arXiv:2602.22432stat.MLcs.LG2026-02被引 1

为梯度提升树设计快速局部置信预测方法,提升不确定性量化精度。

LoBoost: Fast Model-Native Local Conformal Prediction for Gradient-Boosted Trees

  • 利用模型自身叶节点结构构建多尺度分区,实现局部残差估计
  • 在小样本局部校准下仍保持稳定覆盖,区间质量优于传统方法
  • 无需额外训练或数据划分,适合实际部署的高效率场景

梯度提升决策树是表格式回归中表现最强的现成预测器之一,但仅提供点预测无法量化不确定性。共形预测可提供无分布假设的边际覆盖率,但标准分割共形预测使用单一全局残差分位数,难以适应异方差性。本文提出LoBoost,一种模型原生的局部共形预测方法,复用已拟合集成模型的叶节点结构,定义特征空间的多尺度划分。每个输入由其在提升路径中访问的叶节点序列表示,匹配的叶节点前缀构成嵌套组,在其中局部估计残差分位数。通过复用模型已学习的预测结构,LoBoost无需辅助划分、无需干扰模型、无需重训练,也无需超出标准共形校准的额外数据划分。理论分析表明,拟合集成的稳定性与诱导单元的几何特性决定了局部残差得分同质性,提供了有限样本覆盖误差控制,并在局部校准规模增大且单元足够同质时实现渐近点态有效性。实验显示其具有竞争性的区间质量、极低的后处理校准成本,且在不同局部校准规模下表现稳定。

原文摘要 · Abstract (English)

Gradient-boosted decision trees are among the strongest off-the-shelf predictors for tabular regression, but point predictions alone do not quantify uncertainty. Conformal prediction provides distribution-free marginal coverage, yet standard split conformal uses a single global residual quantile and can adapt poorly to heteroscedasticity. We propose LoBoost, a model-native local conformal method that reuses the fitted ensemble's leaf structure to define a multiscale partition of the feature space. Each input is represented by the sequence of leaves it visits along the boosting path, and matching leaf prefixes define nested groups in which residual quantiles are estimated locally. By reusing the predictive structure already learned by the model, LoBoost requires no auxiliary partition or nuisance model, no retraining, and no additional data split beyond standard conformal calibration. Our theory connects the stability of the fitted ensemble and the geometry of its induced cells to local residual-score homogeneity, providing finite-sample coverage-error control and asymptotic pointwise validity as the local calibration size grows and the cells become sufficiently homogeneous. Experiments show competitive interval quality, low post-hoc calibration costs, and stable behavior across local calibration-size settings.

共形预测梯度提升不确定性量化局部校准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。