arXiv:2503.06985cs.LG2025-03被引 2

用强化学习生成可解释决策树,性能超越主流方法。

Learning Decision Trees as Amortized Structure Inference

  • 将决策树构建转化为序列规划问题,用GFlowNet生成树结构。
  • 在多个真实数据集上优于现有树模型与深度学习方法。
  • 生成的树可组合成随机森林,规模越大越稳定且可解释。

构建表格式数据的预测模型面临根本性挑战,尤其在于资源增加是否能持续提升性能,以及能否系统性泛化到训练分布之外。决策树模型的设计尤为困难,因其搜索空间呈指数级增长,现有方法多依赖贪心启发式,而深度学习的归纳偏置则期望时间或空间结构,这在表格式数据中并不自然存在。本文提出一种混合型摊销结构推断方法,基于数据学习预测性决策树集成,将决策树构建视为序列规划问题。通过训练深度强化学习(GFlowNet)策略求解该问题,得到一个从贝叶斯后验中采样决策树的生成模型。实验表明,所提出的DT-GFN方法在标准分类基准测试中优于最先进的决策树与深度学习方法,在分布外鲁棒性与异常检测任务中表现优异,同时生成可解释模型且描述长度更短。训练后的DT-GFN模型样本可组成随机森林,进一步证明其性能随集成规模一致提升,持续实现系统性泛化。

原文摘要 · Abstract (English)

Building predictive models for tabular data presents fundamental challenges, notably in scaling consistently, i.e., more resources translating to better performance, and generalizing systematically beyond the training data distribution. Designing decision tree models remains especially challenging given the intractably large search space, and most existing methods rely on greedy heuristics, while deep learning inductive biases expect a temporal or spatial structure not naturally present in tabular data. We propose a hybrid amortized structure inference approach to learn predictive decision tree ensembles given data, formulating decision tree construction as a sequential planning problem. We train a deep reinforcement learning (GFlowNet) policy to solve this problem, yielding a generative model that samples decision trees from the Bayesian posterior. We show that our approach, DT-GFN, outperforms state-of-the-art decision tree and deep learning methods on standard classification benchmarks derived from real-world data, robustness to distribution shifts, and anomaly detection, all while yielding interpretable models with shorter description lengths. Samples from the trained DT-GFN model can be ensembled to construct a random forest, and we further show that the performance of scales consistently in ensemble size, yielding ensembles of predictors that continue to generalize systematically.

决策树强化学习可解释性生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。