调参新发现:随机森林回归中,采样率影响模型性能,需根据数据特点调整。
The Impact of Bootstrap Sampling Rate on Random Forest Performance in Regression Tasks
- 通过系统测试不同采样率,发现最优值常不在默认1.0
- 39个数据集上,24个用低于1.0的采样率表现更好
- 数据全局关系强则宜用高采样率,局部波动大则宜低采样率
随机森林(RF)通常以与训练集相同规模的自助采样(即采样率BR=1.0)训练每棵树。我们系统考察了在39个异构回归数据集和16种随机森林配置下,将采样率从0.2调至5.0对性能的影响,采用重复两折交叉验证和均方误差评估。结果表明,调优采样率可显著优于默认设置:24个数据集最佳采样率≤1.0,15个>1.0,仅4个在1.0时最优。我们发现数据特征与优选采样率相关:具有强全局特征-目标关系的数据偏好更高采样率,而局部目标方差高的数据更适合低采样率。在可控噪声的合成数据上进一步验证,低噪声下高采样率有效降低偏差,高噪声下低采样率更利于减少方差。总体而言,采样率是影响随机森林回归的重要超参数,应被调优。
原文摘要 · Abstract (English)
Random Forests (RFs) typically train each tree on a bootstrap sample of the same size as the training set, i.e., bootstrap rate (BR) equals 1.0. We systematically examine how varying BR from 0.2 to 5.0 affects RF performance across 39 heterogeneous regression datasets and 16 RF configurations, evaluating with repeated two-fold cross-validation and mean squared error. Our results demonstrate that tuning the BR can yield significant improvements over the default: the best setup relied on BR \leq 1.0 for 24 datasets, BR > 1.0 for 15, and BR = 1.0 was optimal in 4 cases only. We establish a link between dataset characteristics and the preferred BR: datasets with strong global feature-target relationships favor higher BRs, while those with higher local target variance benefit from lower BRs. To further investigate this relationship, we conducted experiments on synthetic datasets with controlled noise levels. These experiments reproduce the observed bias-variance trade-off: in low-noise scenarios, higher BRs effectively reduce model bias, whereas in high-noise settings, lower BRs help reduce model variance. Overall, BR is an influential hyperparameter that should be tuned to optimize RF regression models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。