为随机森林提供可解释的方差分析,揭示预测不确定性的来源。
Random Forests as Statistical Procedures: Design, Variance, and Dependence
- 基于设计的有限样本理论,将森林方差拆解为蒙特卡洛误差与协方差下界。
- 即使样本分割消除重用,树间分区对齐仍导致正的协方差下界。
- 提出PASR方法,首次实现部署森林的置信区间与不确定性分解。
我们构建了一种有限样本、基于设计的随机森林理论,其中每棵树是作用于固定协变量的随机化条件预测器,森林为其蒙特卡洛平均。一个精确的方差恒等式将蒙特卡洛误差与无限聚合下仍存在的协方差下界分离。该下界由两个机制引发:观测值重用(相同训练结果在多棵树中被加权)和分区对齐(独立生成的树发现相似的条件预测规则)。我们在最弱条件下证明该下界严格为正,并显示即使样本分割完全消除观测重叠,对齐仍持续存在。我们引入程序对齐合成抽样(PASR)来估计协方差下界,将部署森林的总预测不确定性分解为可解释成分。对于连续结果,所得预测区间达到名义覆盖度,且具有理论保证的保守偏差方向;对于分类森林,PASR估计量渐近无偏,首次提供部署森林的条件概率点态置信区间。在多种设计配置下,包括高维情形,名义覆盖度均得以保持。该理论适用于任何具有交换树生成机制的树基集成方法。
原文摘要 · Abstract (English)
We develop a finite-sample, design-based theory for random forests in which each tree is a randomized conditional predictor acting on fixed covariates and the forest is their Monte Carlo average. An exact variance identity separates Monte Carlo error from a covariance floor that persists under infinite aggregation. The floor arises through two mechanisms: observation reuse, where the same training outcomes receive weight across multiple trees, and partition alignment, where independently generated trees discover similar conditional prediction rules. We prove the floor is strictly positive under minimal conditions and show that alignment persists even when sample splitting eliminates observation overlap entirely. We introduce procedure-aligned synthetic resampling (PASR) to estimate the covariance floor, decomposing the total prediction uncertainty of a deployed forest into interpretable components. For continuous outcomes, resulting prediction intervals achieve nominal coverage with a theoretically guaranteed conservative bias direction. For classification forests, the PASR estimator is asymptotically unbiased, providing the first pointwise confidence intervals for predicted conditional probabilities from a deployed forest. Nominal coverage is maintained across a range of design configurations for both outcome types, including high-dimensional settings. The underlying theory extends to any tree-based ensemble with an exchangeable tree-generating mechanism.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。