用梯度优化决策树,能适配任意可微损失函数。
A novel gradient-based method for decision trees optimizing arbitrary differential loss functions
- 基于损失函数的梯度和二阶导数动态调整分裂点。
- 在分类、回归和生存分析任务中表现优于传统方法。
- 适合需要高精度且保持可解释性的建模场景。
现有决策树训练方法多依赖启发式分裂规则。本文提出一种新型基于梯度的方法,用于构建可优化任意可微损失函数的决策树,突破了传统方法的局限性。该方法利用损失函数的一阶和二阶导数精炼预测,适用于分类、回归及含删失数据的生存分析等复杂任务。在真实与合成数据集上的实验表明,该方法在性能上优于CART、 Extremely Randomized Trees和SurvTree等经典算法。代码已公开,为研究者和实践者提供实用工具。该工作推动了基于决策树建模的发展,通过梯度优化弥合传统决策树与现代机器学习技术之间的差距,为可解释且高性能模型的创新铺平道路。
原文摘要 · Abstract (English)
There are many approaches for training decision trees. This work introduces a novel gradient-based method for constructing decision trees that optimize arbitrary differentiable loss functions, overcoming the limitations of heuristic splitting rules. Unlike traditional approaches that rely on heuristic splitting rules, the proposed method refines predictions using the first and second derivatives of the loss function, enabling the optimization of complex tasks such as classification, regression, and survival analysis. We demonstrate the method's applicability to classification, regression, and survival analysis tasks, including those with censored data. Numerical experiments on both real and synthetic datasets compare the proposed method with traditional decision tree algorithms, such as CART, Extremely Randomized Trees, and SurvTree. The implementation of the method is publicly available, providing a practical tool for researchers and practitioners. This work advances the field of decision tree-based modeling, offering a more flexible and accurate approach for handling structured data and complex tasks. By leveraging gradient-based optimization, the proposed method bridges the gap between traditional decision trees and modern machine learning techniques, paving the way for further innovations in interpretable and high-performing models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。