提出自适应Lasso加权随机森林，理论证明优于传统方法
Lassoed Forests: Random Forests with Adaptive Lasso Post-selection
- 用自适应Lasso对树模型预测结果加权，实现动态选择重要树
- 理论证明在信号噪声比条件下性能严格优于标准与Lasso加权方法
- 适合需要高精度变量选择的复杂数据建模场景
随机森林通过自助采样平均高方差、低偏差的树来提升预测性能。已有研究尝试用Lasso回归对树的预测结果加权以降低模型偏差，但有时反而导致性能下降（如均方误差上升）。本文从理论上揭示：标准随机森林与Lasso加权随机森林的相对表现取决于信号-噪声比。我们提出统一框架，结合随机森林与自适应Lasso选择，通过自适应加权实现严格优于前两者的性能。通过模拟实验对比了三种方法，包括偏差-方差分解、误差估计评估和变量重要性分析，并在多种真实数据集上验证了方法的通用性。
原文摘要 · Abstract (English)
Random forests are a statistical learning technique that use bootstrap aggregation to average high-variance and low-bias trees. Improvements to random forests, such as applying Lasso regression to the tree predictions, have been proposed in order to reduce model bias. However, these changes can sometimes degrade performance (e.g., an increase in mean squared error). In this paper, we show in theory that the relative performance of these two methods, standard and Lasso-weighted random forests, depends on the signal-to-noise ratio. We further propose a unified framework to combine random forests and Lasso selection by applying adaptive weighting and show mathematically that it can strictly outperform the other two methods. We compare the three methods through simulation, including bias-variance decomposition, error estimates evaluation, and variable importance analysis. We also show the versatility of our method by applications to a variety of real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。