arXiv:2502.05157cs.LGcs.DS2025-02

提出高效算法训练可校准的非参数概率回归树,提升预测可信度。

Efficient distributional regression trees learning algorithms for calibrated non-parametric probabilistic forecasts

  • 基于最小-最大堆等数据结构优化损失函数计算
  • 在WIS和CRPS指标上实现良好覆盖率与校准性
  • 适合需要可解释性及群体条件覆盖保障的场景

为科学与工程中关键应用构建可信AI,需具备自我不确定性估计能力。在回归任务中,可通过输出预测区间或学习条件概率分布 $p(y|x)$ 实现。参数模型假设过强,非参数方法更具灵活性。针对标量输出,直接建模条件累积分布函数可提升概率估计精度,使用加权区间评分(WIS)和连续排名概率评分(CRPS)等合理评分规则能改善覆盖率与校准性。本文提出新型算法,用于在WIS或CRPS损失下学习概率回归树,借助最小-最大堆、加权平衡二叉树与芬威克树等数据结构实现计算高效。数值实验表明,方法性能优于或媲美现有方案。此外,树结构天然具备可解释性;作为副产物,本文展示如何将该方法用于分层置信预测,并解释其为何特别适合实现群体条件覆盖保证。

原文摘要 · Abstract (English)

The perspective of developing trustworthy AI for critical applications in science and engineering requires machine learning techniques that are capable of estimating their own uncertainty. In the context of regression, instead of estimating a conditional mean, this can be achieved by producing a predictive interval for the output, or to even learn a model of the conditional probability $p(y|x)$ of an output $y$ given input features $x$. While this can be done under parametric assumptions with, e.g. generalized linear model, these are typically too strong, and non-parametric models offer flexible alternatives. In particular, for scalar outputs, learning directly a model of the conditional cumulative distribution function of $y$ given $x$ can lead to more precise probabilistic estimates, and the use of proper scoring rules such as the weighted interval score (WIS) and the continuous ranked probability score (CRPS) lead to better coverage and calibration properties. This paper introduces novel algorithms for learning probabilistic regression trees for the WIS or CRPS loss functions. These algorithms are made computationally efficient thanks to an appropriate use of known data structures - namely min-max heaps, weight-balanced binary trees and Fenwick trees. Through numerical experiments, we demonstrate that the performance of our methods is competitive with alternative approaches. Additionally, our methods benefit from the inherent interpretability and explainability of trees. As a by-product, we show how our trees can be used in the context of conformal prediction and explain why they are particularly well-suited for achieving group-conditional coverage guarantees.

概率预测回归树校准可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。