给随机森林加平滑,小数据下预测更准
Improving Random Forests by Smoothing
- 用核函数对树模型输出做局部平滑,融合平滑假设
- 在小样本场景下显著提升预测性能,尤其在低数据量时
- 适合需要稳定预测的小数据任务,如医疗、金融建模
随机森林回归是一种强大的非参数方法,通过数据驱动的分块划分自适应地捕捉局部数据特征,在多种应用中表现良好。然而,其预测结果是分块常数形式,各分区独立预测,忽略了潜在函数的平滑性。尤其在小数据情况下,输入空间间的信息共享不足会导致性能下降。本文提出一种基于核函数的平滑机制,增强随机森林的预测能力,在保持其自适应分块特性的同时引入局部正则性。该方法对随机森林的分块常数输出进行核平滑,有效结合了树模型的自适应性与核方法的平滑假设。我们证明该平滑过程可解释为在训练数据重采样下树切点变异性的捕捉。实验表明,所提平滑随机森林模型在多种测试场景中持续提升预测性能,尤其在数据稀缺条件下表现更优。代码、数据集和实验结果已公开于 https://github.com/Neal-Liu-Ziyi/SmoothedRandomForest.git。
原文摘要 · Abstract (English)
Random forest regression is a powerful non-parametric method that adapts to local data characteristics through data-driven partitioning, making it effective across diverse application domains. However, the piecewise constant nature of random forest predictions means each partition is predicted independently, ignoring potential smoothness in the underlying function. Particularly in the small data regime, this lack of information sharing across the input space can lead to suboptimal performance. In this work, we propose a kernel-based smoothing mechanism that enhances random forests by introducing local regularity to their predictions while preserving their adaptive partitioning capabilities. Our approach applies kernel smoothing to the piecewise constant outputs of random forests, effectively combining the adaptability of tree-based methods with the smoothness assumptions of kernel methods. We show that this smoothing procedure can be interpreted as capturing the variability/uncertainty in the tree cut points under resampling of the training inputs. Empirical results demonstrate that the proposed smoothed random forest model consistently improves predictive performance across diverse test cases, particularly in data-scarce settings. Code, datasets, and experiment results are publicly available at https://github.com/Neal-Liu-Ziyi/SmoothedRandomForest.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。