用随机斜向切分提升树模型表达能力,更准更小还易解释。
RO-FIGS: Efficient and Expressive Tree-Based Ensembles for Tabular Data
- 引入随机特征组合的斜向切分,捕捉特征间复杂交互
- 22个数据集上性能超越主流树与神经网络模型,模型更小
- 可解释性增强,能揭示特征交互,适合需要透明决策的场景
树模型对无关特征鲁棒,能有效捕捉非光滑、复杂的决策边界,常在表格数据上以更低计算成本优于神经网络。但传统树集成方法因仅用单一特征切分,表达效率受限。为此,本文提出随机斜向快速可解释贪婪树求和(RO-FIGS),基于快速可解释贪婪树求和,通过学习由随机特征子集线性组合构成的斜向切分,提升模型对特征交互的捕捉能力。该方法适用于含数值与类别特征的表格数据。在22个真实世界表格数据集上评估,结果表明其性能优于其他树与神经网络方法,且模型显著更小。进一步分析切分机制,揭示了有价值的特征交互信息,丰富了SHAP摘要图的信息,验证了其更强的可解释性。该方法适合需兼顾准确率与可解释性的实际应用。
原文摘要 · Abstract (English)
Tree-based models are often robust to uninformative features and can accurately capture non-smooth, complex decision boundaries. Consequently, they often outperform neural network-based models on tabular datasets at a significantly lower computational cost. Nevertheless, the capability of traditional tree-based ensembles to express complex relationships efficiently is limited by using a single feature to make splits. To improve the efficiency and expressiveness of tree-based methods, we propose Random Oblique Fast Interpretable Greedy-Tree Sums (RO-FIGS). RO-FIGS builds on Fast Interpretable Greedy-Tree Sums, and extends it by learning trees with oblique or multivariate splits, where each split consists of a linear combination learnt from random subsets of features. This helps uncover interactions between features and improves performance. The proposed method is suitable for tabular datasets with both numerical and categorical features. We evaluate RO-FIGS on 22 real-world tabular datasets, demonstrating superior performance and much smaller models over other tree- and neural network-based methods. Additionally, we analyse their splits to reveal valuable insights into feature interactions, enriching the information learnt from SHAP summary plots, and thereby demonstrating the enhanced interpretability of RO-FIGS models. The proposed method is well-suited for applications, where balance between accuracy and interpretability is essential.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。