arXiv:2410.21595cs.LG2024-10

提出可处理结构化与非结构化数据的可解释深度决策树,性能超越传统方法。

Deep Trees for (Un)structured Data: Tractability, Performance, and Interpretability

  • 用软树扩展为通用软树,支持直接处理图像等非结构化数据。
  • 在多个数据集上表现优于CART、XGBoost等,尤其在CIFAR-10和Fashion MNIST上显著领先。
  • 模型节点少、可解释性强,适合需要透明决策的医疗与高风险场景。

决策树因其可解释性在表格数据中广受欢迎,但难以处理高度非线性或非结构化数据。我们提出广义软树(GSTs),拓展了软决策树能力,可直接处理图像数据。通过提出的DeepTree算法,实现了可计算的树生长机制,引入新正则项,使模型节点更少、性能更高、可解释性更强。在MIMIC-IV、MNIST、Fashion MNIST、CIFAR-10和Celeb-A等基准数据集上测试表明,我们的方法在几乎所有数据集中均优于CART、随机森林、XGBoost等主流树方法,其中卷积树在最难的CIFAR-10和Fashion MNIST上优势明显。进一步分析显示,即使最复杂的GSTs也远比深度神经网络更具可解释性。总体而言,该方法在(非)结构化数据上兼具高性能与可解释性,且计算可行。

原文摘要 · Abstract (English)

Decision Trees have remained a popular machine learning method for tabular datasets, mainly due to their interpretability. However, they lack the expressiveness needed to handle highly nonlinear or unstructured datasets. Motivated by recent advances in tree-based machine learning (ML) techniques and first-order optimization methods, we introduce Generalized Soft Trees (GSTs), which extend soft decision trees (STs) and are capable of processing images directly. We demonstrate their advantages with respect to tractability, performance, and interpretability. We develop a tractable approach to growing GSTs, given by the DeepTree algorithm, which, in addition to new regularization terms, produces high-quality models with far fewer nodes and greater interpretability than traditional soft trees. We test the performance of our GSTs on benchmark tabular and image datasets, including MIMIC-IV, MNIST, Fashion MNIST, CIFAR-10 and Celeb-A. We show that our approach outperforms other popular tree methods (CART, Random Forests, XGBoost) in almost all of the datasets, with Convolutional Trees having a significant edge in the hardest CIFAR-10 and Fashion MNIST datasets. Finally, we explore the interpretability of our GSTs and find that even the most complex GSTs are considerably more interpretable than deep neural networks. Overall, our approach of Generalized Soft Trees provides a tractable method that is high-performing on (un)structured datasets and preserves interpretability more than traditional deep learning methods.

决策树可解释性图像分类深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。