arXiv:2606.03549cs.LGmath.PR2026-06被引 1

提出新方法自动确定随机森林最优树数,避免盲目增加。

How Many Trees in a Random Forest? A Revisited Approach with Plateau Search and Optuna Integration

论文配图:How Many Trees in a Random Forest? A Revisited Approach with Plateau Search and Optuna Integration
图 1 · 摘自论文原文
  • 用三元组检测袋外误差变化,动态追踪最小足够集成规模。
  • 在多个数据集上发现最优树数比传统经验法则少或更多。
  • 方法无需预设范围,对噪声不敏感,适合实际应用者使用。

随机森林的超参数优化中,树的数量难以调优:预测得分随集成规模单调提升,导致TPE、Hyperband等标准方法需预设搜索范围,常将结果推向边界。早停策略虽免去范围设定,却易受评分噪声影响并过早停止。为此,我们提出一种基于三元组的平台搜索算法,将树数移出直接搜索空间,仍利用历史试验信息。该方法通过监测三个森林规模下的袋外(OOB)得分相对变化,自适应调整三元组位置,从而自动识别近似最小足够集成规模。整个过程依赖于容忍度参数,具有用户可解释性。理论分析表明,所提相对OOB得分准则与当前得分和极限得分之间的差距相关,并推导了基于OOB的绝对相对差异的渐近方差估计。实验显示,所选树数与常见启发式显著不同:在多数经典基准数据集上更少,而在高维生物信息学数据集(如Arcene和Dorothea)上则更多。源代码与可复现实验已公开于https://github.com/lange-am/rf_plateau_hpo。

原文摘要 · Abstract (English)

Hyperparameter optimization (HPO) for Random Forest faces a specific difficulty in tuning the number of trees: the predictive score typically improves monotonically with ensemble size, so standard methods such as Tree-structured Parzen Estimator (TPE) and Hyperband require a predefined search range and often drive the estimate toward its right boundary. Early-stopping strategies avoid fixing such a range, but can be sensitive to score noise and prone to premature stopping. To address this, we propose an integrated triplet-based plateau-search algorithm that removes the number of trees from the direct TPE search space and still exploits information accumulated across HPO trials. The method adaptively tracks a near-minimal sufficient ensemble size by monitoring relative changes in the out-of-bag (OOB) score across a triplet of forest sizes and shifting this triplet accordingly. This yields an automated and user-interpretable procedure based on a tolerance parameter. We also provide a theoretical analysis: we relate the proposed relative OOB-score criterion to the gap between the current and limiting scores, and derive an asymptotic variance estimate for the corresponding OOB-based absolute relative difference. Experiments show that the selected number of trees can differ substantially from the common heuristic: for most classical benchmark datasets it is smaller, whereas for some high-dimensional bioinformatics datasets, such as Arcene and Dorothea, it is larger. The source code and reproducible experiments are available at https://github.com/lange-am/rf_plateau_hpo.

随机森林超参优化自动化机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。