用随机森林提升分布外数据的预测稳定性。
Maximum Risk Minimization with Random Forests
- 基于最大风险最小化原则改进随机森林结构。
- 在多种风险度量下均实现稳定性能，尤其对未知分布有保障。
- 适合需要鲁棒性预测的现实场景，如医疗或金融建模。
我们研究观测数据来自不同环境（即不同数据分布）的回归问题。分布外（OOD）泛化旨在设计能更好适应训练时未见分布的测试环境的方法。其中一种思路是极小化各环境中的最大风险，称为 MaxRM（最大风险最小化）。本文提出基于 MaxRM 原则的随机森林变体，给出高效的计算算法，并证明主要方法的统计一致性。所提方法可适用于三种风险：均方误差、负奖励与后悔（衡量相对于最优预测器的超额风险）。针对以后悔为风险的 MaxRM，我们建立了对未见测试分布的新样本保证。最后，我们在模拟数据与真实世界数据上评估了所提方法。
原文摘要 · Abstract (English)
We consider a regression setting where observations are collected in different environments modeled by different data distributions. The field of out-of-distribution (OOD) generalization aims to design methods that generalize better to test environments whose distributions differ from those observed during training. One line of such works has proposed to minimize the maximum risk across environments, a principle that we refer to as MaxRM (Maximum Risk Minimization). In this work, we introduce variants of random forests based on the principle of MaxRM. We provide computationally efficient algorithms and prove statistical consistency for our primary method. Our proposed method can be used with each of the following three risks: the mean squared error, the negative reward, and the regret (which quantifies the excess risk relative to the best predictor). For MaxRM with regret as the risk, we prove a novel out-of-sample guarantee over unseen test distributions. Finally, we evaluate the proposed methods on both simulated and real-world data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。