RaFFLE提升随机森林线性拟合能力，兼顾精度与效率。
A Powerful Random Forest Featuring Linear Extensions (RaFFLE)
- 用分段线性树替代传统决策树作为基学习器
- 在136个数据集上优于CART、随机森林、XGBoost等方法
- 适合需精准拟合线性关系的回归任务
随机森林广泛应用于回归任务，但其基学习器决策树难以有效拟合线性关系。为此，本文提出RaFFLE（Random Forest Featuring Linear Extensions），将近期提出的PILOT树（分段线性有机树）作为基学习器引入随机森林集成框架。PILOT树兼具传统决策树的计算效率与线性模型树的灵活性。为保证个体树的多样性，引入可调正则化参数并采用节点级特征采样。该设计提升了森林整体准确性。理论分析表明，RaFFLE在弱条件下具有一致性，并在数据由线性模型生成时收敛更快。在136个回归数据集上的实验显示，RaFFLE在线性和非线性数据上均显著优于经典CART、随机森林、正则化线性方法Lasso与Ridge，以及前沿的XGBoost算法。通过平衡预测精度与计算效率，RaFFLE成为应对多种回归问题的通用工具。
原文摘要 · Abstract (English)
Random forests are widely used in regression. However, the decision trees used as base learners are poor approximators of linear relationships. To address this limitation we propose RaFFLE (Random Forest Featuring Linear Extensions), a novel framework that integrates the recently developed PILOT trees (Piecewise Linear Organic Trees) as base learners within a random forest ensemble. PILOT trees combine the computational efficiency of traditional decision trees with the flexibility of linear model trees. To ensure sufficient diversity of the individual trees, we introduce an adjustable regularization parameter and use node-level feature sampling. These modifications improve the accuracy of the forest. We establish theoretical guarantees for the consistency of RaFFLE under weak conditions, and its faster convergence when the data are generated by a linear model. Empirical evaluations on 136 regression datasets demonstrate that RaFFLE outperforms the classical CART and random forest methods, the regularized linear methods Lasso and Ridge, and the state-of-the-art XGBoost algorithm, across both linear and nonlinear datasets. By balancing predictive accuracy and computational efficiency, RaFFLE proves to be a versatile tool for tackling a wide variety of regression problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。