用卷积平滑损失改进XGBoost,精准预测极端环境数据的分布与风险。
Convolution Smoothed Quantile Regression for XGBoost

- 在XGBoost中引入卷积平滑分位数损失,保留二阶信息提升建树效率。
- 多输出树结构使分位数无交叉,对加州野火期间细颗粒物预测准确率高。
- 适合关注极端事件概率、风险评估的环境与金融领域研究者使用。
随着各科学领域大数据日益丰富,机器学习广泛用于预测。然而多数算法仅提供点估计,缺乏对预测不确定性的刻画或响应变量条件分布的信息,难以描述罕见或极端结果。本文提出基于分位数的梯度提升框架QXGB,引入卷积平滑损失以估计条件分位数,构建密集累积分布函数(CDF)、超限概率及尾部行为,有助于极端事件分析。该方法保持极端梯度提升的计算效率,同时恢复XGBoost依赖的海森矩阵信息,实现可解释的极端值与超限概率预测。我们推导了不同核函数下集成卷积平滑分位数损失所需的梯度与海森矩阵,并通过模拟数据对比了该方法与其它平滑分位数损失、XGBoost原生分位数目标以及独立与多输出树估计的表现。实际应用展示于北加州细颗粒物(PM₂.₅)预测,涵盖野火烟雾导致浓度升高的时段。结果表明,卷积平滑QXGB(尤其配合多输出树)能实现近乎零分位数交叉、校准良好的CDF与超限概率估计,并有效刻画极端值尾部特征。区间估计也用于衡量数据离散程度。
原文摘要 · Abstract (English)
The increasing availability of large and complex datasets across many scientific disciplines has led to widespread adoption of machine learning (ML) for prediction. However, most ML algorithms focus on point estimation and provide limited information about predictive uncertainty or the conditional distribution of the response, restricting their ability to characterize rare or extreme outcomes. We develop QXGB, a quantile-based gradient boosting framework, and introduce a convolution smoothed loss within it that estimates conditional quantiles for constructing dense cumulative distribution functions (CDFs), exceedance probabilities, and tail behaviour relevant to extreme outcomes. This approach preserves the computational efficiency of extreme gradient boosting while restoring the Hessian information XGBoost relies on for tree splitting, in turn providing interpretable measures of extreme value and exceedance probability predictions. We derive the gradients and Hessians needed to integrate convolution smoothed quantile loss with different kernel specifications into XGBoost, and with simulated data, benchmark this approach against alternative smoothed quantile regression losses, the native quantile objective in the XGBoost Python package, and independent versus multi-output tree estimation. The practical relevance is illustrated in an application predicting fine particulate matter (PM$_{2.5}$) in northern California, including periods where levels were elevated due to wildfire smoke. Our results show that convolution smoothed QXGB, particularly when paired with multi-output trees, delivers accurate predictions with near-zero quantile crossing, well-calibrated CDF and exceedance probability estimates, and useful tail characterization for extreme values. Interval estimation is also evaluated as a measure of data spread.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。