揭示随机森林树数选择中平台搜索的随机规律
A Stationary-Distribution Theory for Triplet-Based Plateau Search in Random Forest Ensemble-Size Selection

- 将树数调整建模为马尔可夫链,分析其稳定分布
- 发现最优树数与误差呈ε⁻²量级关系,方差达ε⁻⁴
- 适合关注模型调参理论的机器学习研究者
随机森林的树数量是核心计算参数:增加树数可降低有限集合的变异性,但会提高训练和预测成本。基于平台的调参通过几何三元组树数的袋外评分局部比较来调整该参数。当其他超参数稳定后,中心三元组点不再收敛到确定值,而是在一个稳态区间波动。本文建立了该过程的稳态分布理论:将中心集成规模 $B_t$ 建模为几何网格上的生灭马尔可夫链,通过局部平衡推导其稳态分布。在主导中心折叠正态近似下,得到原始更新规则与对称改进版本的平衡方程,表明稳态中心 $B_*=O(ε^{-2})$(当 $ε\downarrow 0$)。稳态扩散也得以刻画。通过局部高斯近似与福克-普朗克解释,获得网格层级方差常数。转换至集成规模尺度后,$σ_{B,*}=O(ε^{-2})$,方差为 $O(ε^{-4})$。主导相对扩散与 $ε$ 无关,由尺度因子和更新规则控制。这些结果将平台式随机森林调参理解为随机过程,而非确定性停止规则。
原文摘要 · Abstract (English)
The number of trees is a central computational parameter in Random Forests: increasing it reduces finite-ensemble variability but increases training and prediction cost. Plateau-based tuning adapts this parameter through local comparisons of out-of-bag scores at a geometric triplet of tree counts. After the remaining hyperparameters have stabilized, however, the central triplet point need not converge to a deterministic value; instead, it fluctuates around a stationary regime. This paper develops a stationary-distribution theory for this process. The central ensemble size $B_t$ is modeled as a birth-death Markov chain on a geometric grid, and its stationary distribution is derived through local balance. Under a leading centered folded-normal approximation, equilibrium equations are obtained for the original update rule and a symmetric modified variant, implying that the stationary center $B_*=O(\varepsilon^{-2})$ as $\varepsilon\downarrow 0$. The stationary spread is also characterized. A local Gaussian approximation and a Fokker-Planck interpretation give grid-level variance constants. After conversion to the ensemble-size scale, $σ_{B,*}=O(\varepsilon^{-2})$, while the variance is $O(\varepsilon^{-4})$. The leading relative spread is independent of $\varepsilon$ and controlled by the scale factor and update rule. These results interpret plateau-based Random Forest tuning as a stochastic process rather than a deterministic stopping rule.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。