为梯度提升回归提供统一的统计推断框架，实现置信区间与变量重要性检验。
Statistical Inference for Gradient Boosting Regression
- 结合丢弃法与正则化，使提升树满足中心极限定理。
- 提高丢弃率和并行树数能显著增强信号恢复与整体性能。
- 可内置置信区间、预测区间和变量重要性假设检验，适合需要可解释性的场景。
梯度提升因其灵活性和高预测精度而广受欢迎，但其统计推断与不确定性量化仍具挑战且研究不足。本文提出一个统一的梯度提升回归统计推断框架，通过整合丢弃法（dropout）与一种近期提出的正则化方法，实现了提升树的中心极限定理（CLT）。实验表明，增加丢弃率及每轮迭代中并行生成的树数量，可显著提升信号恢复能力与整体性能。所提算法具有相似的CLT性质，可用于构建内置的置信区间、预测区间及严格的变量重要性假设检验。数值实验显示，该方法表现良好，介于正则化提升树与随机森林之间，并验证了其推断过程的有效性。
原文摘要 · Abstract (English)
Gradient boosting is widely popular due to its flexibility and predictive accuracy. However, statistical inference and uncertainty quantification for gradient boosting remain challenging and under-explored. We propose a unified framework for statistical inference in gradient boosting regression. Our framework integrates dropout or parallel training with a recently proposed regularization procedure that allows for a central limit theorem (CLT) for boosting. With these enhancements, we surprisingly find that increasing the dropout rate and the number of trees grown in parallel at each iteration substantially enhances signal recovery and overall performance. Our resulting algorithms enjoy similar CLTs, which we use to construct built-in confidence intervals, prediction intervals, and rigorous hypothesis tests for assessing variable importance. Numerical experiments demonstrate that our algorithms perform well, interpolate between regularized boosting and random forests, and confirm the validity of their built-in statistical inference procedures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。