用浅层随机森林实现快速精准预测,显著降低延迟与内存占用。
DiNo and RanBu: Lightweight Predictions from Shallow Random Forests
- 通过距离加权机制将少量浅层树转化为高效预测器
- 在25个公开数据集上提速95%,高噪声下精度仍优于完整随机森林
- 适合对速度敏感的部署场景,支持量化回归且无需重新训练
随机森林是表格预测任务的强基准,但其依赖数百棵深层树常导致推理延迟高、内存需求大,限制了在低延迟或资源受限环境中的应用。本文提出DiNo(基于节点距离)和RanBu(随机灌木),两种浅层森林方法,将少量深度受限的树转换为高效的距离加权预测器。DiNo通过观测对的最近共同祖先计算共聚类距离,RanBu则对Breiman的经典邻近度采用核平滑。两者均在森林训练后运行:不增加新树,仅需轻量级矩阵-向量运算调节单一带宽参数 $h$。在三个合成基准和25个公开数据集上,RanBu在高噪声环境下达到或超越全深度随机森林的精度,同时将训练加推理时间减少高达95%。DiNo在低噪声场景中实现最佳偏差-方差平衡,计算开销适中。两方法均可直接扩展至分位数回归,保持精度并获得显著加速。代码已开源,支持R/C++,聚焦独立同分布的结构化表格数据,其他模态的扩展留待未来工作。
原文摘要 · Abstract (English)
Random Forest ensembles are a strong baseline for tabular prediction tasks, but their reliance on hundreds of deep trees often results in high inference latency and memory demands, limiting deployment in latency-sensitive or resource-constrained environments. We introduce DiNo (Distance with Nodes) and RanBu (Random Bushes), two shallow-forest methods that convert a small set of depth-limited trees into efficient, distance-weighted predictors. DiNo measures cophenetic distances via the most recent common ancestor of observation pairs, while RanBu applies kernel smoothing to Breiman's classical proximity measure. Both approaches operate entirely after forest training: no additional trees are grown, and tuning of the single bandwidth parameter $h$ requires only lightweight matrix-vector operations. Across three synthetic benchmarks and 25 public datasets, RanBu matches or exceeds the accuracy of full-depth random forests-particularly in high-noise settings-while reducing training plus inference time by up to 95\%. DiNo achieves the best bias-variance trade-off in low-noise regimes at a modest computational cost. Both methods extend directly to quantile regression, maintaining accuracy with substantial speed gains. The implementation is available as an open-source R/C++ package at https://github.com/tiagomendonca/dirf. We focus on structured tabular random samples (i.i.d.), leaving extensions to other modalities for future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。