改进XGBoost抗异常值能力，提升回归模型鲁棒性
Robust XGBoosting for Regression
- 用M/S/τ-估计器替代传统损失函数
- 两步法MM-XGBoost在异常值下表现最优
- 适合存在离群点的工业预测场景
XGBoost是一种广泛使用的强大预测方法，通过迭代拟合简单决策树来逼近前一步的残差。其标准损失函数为二次损失，也可使用Huber损失。本文研究了XGBoost的鲁棒性，发现其性能会受垂直异常值和高杠杆点影响。为此，我们探索了基于稳健回归中M-、S-和τ-估计器的替代损失函数。结果表明，称为MM-XGBoost的两步法在鲁棒性与预测精度之间提供了最佳平衡。
原文摘要 · Abstract (English)
XGBoost is a very popular and powerful method for prediction. It iteratively fits simple decision trees to the residuals of the previous step. An efficient and scalable implementation is available. The standard loss function for XGBoost is the quadratic loss, but a Huber loss can also be used. In this paper, we study the robustness of XGBoost and show that its performance can be affected by vertical outliers and leverage points. To address this, we explore alternative loss functions, based on M-, S-, and τ -estimators from robust regression. Our results indicate that a two-step procedure, referred to as MM-XGBoost, provides the best trade-off between robustness and prediction accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。