提出新方法提升梯度提升树在含标签噪声数据上的鲁棒性
Training Gradient Boosted Decision Trees on Tabular Data Containing Label Noise for Classification Tasks
- 引入基于梯度的噪声检测法，适配梯度提升树
- 在成人数据集上噪声检测准确率超99%
- 适合处理带噪声的表格数据分类任务
标签噪声指数据集中实例被错误标注，会显著降低分类器性能、增加模型复杂度并影响特征选择。尽管现有研究多聚焦于图像和文本领域的深度神经网络，本文首次系统研究标签噪声对梯度提升决策树（GBDT）的影响。通过将两种深度学习中的噪声检测方法迁移至GBDT，并提出一种名为Gradients的新检测方法，同时扩展原有方法以支持标签重标注。在Covertype、Breast Cancer和Adult等数据集上引入不同水平的标签噪声，评估早停与噪声检测方法的有效性。结果表明，所提方法在所有噪声水平下均达到99%以上的检测准确率，显著优于现有方案。该研究深化了对GBDT中标签噪声影响的理解，为后续噪声检测与修正方法提供了基础。
原文摘要 · Abstract (English)
Label noise, which refers to the mislabeling of instances in a dataset, can significantly impair classifier performance, increase model complexity, and affect feature selection. While most research has concentrated on deep neural networks for image and text data, this study explores the impact of label noise on gradient-boosted decision trees (GBDTs), the leading algorithm for tabular data. This research fills a gap by examining the robustness of GBDTs to label noise, focusing on adapting two noise detection methods from deep learning for use with GBDTs and introducing a new detection method called Gradients. Additionally, we extend a method initially designed for GBDTs to incorporate relabeling. By using diverse datasets such as Covertype and Breast Cancer, we systematically introduce varying levels of label noise and evaluate the effectiveness of early stopping and noise detection methods in maintaining model performance. Our noise detection methods achieve state-of-the-art results, with a noise detection accuracy above 99% on the Adult dataset across all noise levels. This work enhances the understanding of label noise in GBDTs and provides a foundation for future research in noise detection and correction methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。