用协方差驱动分裂，让回归树更抗过拟合
Covariance-Driven Regression Trees: Reducing Overfitting in CART
- 引入协方差驱动的分裂准则，替代传统最小化经验风险
- 在模拟和真实任务中均比CART预测更准，尤其小样本下优势明显
- 适合对模型稳定性要求高的场景，如医学、经济建模
决策树是广泛应用于经济、医学等领域的强大机器学习算法，因其简洁与可解释性备受青睐。然而，如CART这类决策树在深度生长或样本量较小时极易过拟合。传统减缓过拟合的方法包括预剪枝和后剪枝，通过限制无信息分支的生长来实现。本文提出一种互补方法——协方差驱动回归树（CovRT），其分裂准则基于协方差，相较于CART所用的经验风险最小化准则更具鲁棒性，能生成更平衡稳定的分割，并更有效地识别出具有真实信号的变量。我们建立了CovRT的泛化误差上界（oracle inequality），证明其在高维设置下的预测性能可媲美CART。实验表明，无论在模拟数据还是真实任务中，CovRT均显著优于CART。
原文摘要 · Abstract (English)
Decision trees are powerful machine learning algorithms, widely used in fields such as economics and medicine for their simplicity and interpretability. However, decision trees such as CART are prone to overfitting, especially when grown deep or the sample size is small. Conventional methods to reduce overfitting include pre-pruning and post-pruning, which constrain the growth of uninformative branches. In this paper, we propose a complementary approach by introducing a covariance-driven splitting criterion for regression trees (CovRT). This method is more robust to overfitting than the empirical risk minimization criterion used in CART, as it produces more balanced and stable splits and more effectively identifies covariates with true signals. We establish an oracle inequality of CovRT and prove that its predictive accuracy is comparable to that of CART in high-dimensional settings. We find that CovRT achieves superior prediction accuracy compared to CART in both simulations and real-world tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。