arXiv:2607.23721stat.MLcs.LG2026-07

用分布对比替代均值分裂,提升随机森林在多维响应中的建模能力

Distributional Split Criteria for Random Forests: Extensions, Shrinkage, and the Robustness of Mean Splitting

论文配图:Distributional Split Criteria for Random Forests: Extensions, Shrinkage, and the Robustness of Mean Splitting
图 1 · 摘自论文原文
  • 以全条件响应分布对比取代传统均值分裂,引入多种分布距离度量
  • 多维响应场景下分布分裂显著优于均值分裂,单维数据则均值分裂更稳健
  • 适用于需捕捉非位置结构的多变量回归任务,如复杂依赖关系建模

分布式随机森林将基于均值的CART分裂替换为比较候选子节点完整条件响应分布的准则。我们在一个统一的诚实森林实现中系统研究了多种准则:各向同性随机傅里叶特征最大均值差异(MMD)、各向异性对角带宽变体、自适应每分裂频率选择变体以及非核切片沃瑟斯坦准则,并引入后处理核均值收缩。通过配对种子比较,在合成分位数机制、真实一元基准、加州房价子集曲线以及多维合成与真实响应上,分析各扩展的效果。三个发现反复出现:第一,各向同性MMD已接近最优,其他扩展和收缩未系统提升性能;第二,一维表格回归中,均值分裂仍是鲁棒默认选择;第三,多维响应是分布分裂真正发挥作用的场景,尤其在纯依赖拷贝函数中,能量评分可区分不同准则,而边际CRPS无法区分。证据支持一个简单分配逻辑:仅当非位置结构既存在又可估计时,分布分裂才有效;否则会削弱分裂选择能力。所有准则、诚实森林及配对比较框架均已开源至drforest库,其基于Rust的分裂搜索支持大规模准则扫描。

原文摘要 · Abstract (English)

Distributional random forests replace mean-based CART splitting with criteria that compare the full conditional response distribution in candidate children. We implement and systematically study a family of such criteria inside a single honest-forest implementation: isotropic random-Fourier-feature maximum mean discrepancy (MMD), an anisotropic diagonal-bandwidth variant, an adaptive per-split frequency-selection variant, and a non-kernel sliced-Wasserstein criterion, together with post-hoc kernel-mean shrinkage of the forest weights. Using paired-seed comparisons across synthetic quantile mechanisms, real univariate benchmarks, a California-housing subsample curve, and multivariate synthetic and real responses, we characterize where each extension pays. Three findings recur. First, among distributional criteria ordinary isotropic MMD is already close to best in class: the anisotropic, adaptive-frequency, and sliced-Wasserstein extensions, and post-hoc shrinkage, do not systematically improve on it. Second, on scalar tabular regression mean-based CART splitting remains the robust default and wins many cells. Third, multivariate responses are the regime where distributional splitting clearly earns its keep, most sharply on a pure-dependence copula where the energy score separates the criteria even though marginal CRPS does not. The evidence supports a simple allocation story: distributional splitting helps only when non-location structure is both present and estimable; otherwise it dilutes split-selection power away from the mean. All criteria, the honest forest, and the paired-comparison harness are implemented in the open-source \texttt{drforest} library, whose Rust-backed split search makes broad criterion sweeps inexpensive.

随机森林分布分裂多变量回归机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。