arXiv:2410.19154stat.MEcs.LG2024-10

一种统一框架,让多种经典模型在神经网络中更可解释且不易过拟合。

Cross Spline Net and a Unified World

  • 用样条变换与交叉网络结合,构建可解释的新型神经网络架构
  • 性能媲美XGBoost和全连接网络,但结构更简单、抗过拟合更强
  • 适合需要可解释性与鲁棒性的数据科学家和工程实践者

在表格数据的机器学习领域,XGBoost和全连接神经网络(FCNN)因性能优异且使用方便而广受欢迎。然而它们结构复杂、难以解释,易过拟合。本文提出一种新框架——交叉样条网络(Cross Spline Net, CSN),基于样条变换与交叉网络(Wang et al. 2017, 2021)。CSN兼具高性能、易用性,同时更简洁、更可解释、更稳健。该框架灵活,通过调整样条层可复现或近似多种非神经网络模型:线性模型、样条统计模型、决策树、规则拟合、梯度提升树、随机森林、斜向树/森林、多变量自适应回归样条(MARS)、多项式核支持向量机等。因此,CSN将上述模型统一于神经网络框架下。借助神经网络库中的高效梯度下降算法,避免了传统方法中诸如启发式、贪心或不可扩展的优化问题。我们以一种特殊形式的CSN——TreeNet为例进行说明,并与XGBoost和FCNN对比,验证其优势。我们认为,CSN为从业者提供了一个灵活、便捷、高性能、强鲁棒且可解释的建模工具。

原文摘要 · Abstract (English)

In today's machine learning world for tabular data, XGBoost and fully connected neural network (FCNN) are two most popular methods due to their good model performance and convenience to use. However, they are highly complicated, hard to interpret, and can be overfitted. In this paper, we propose a new modeling framework called cross spline net (CSN) that is based on a combination of spline transformation and cross-network (Wang et al. 2017, 2021). We will show CSN is as performant and convenient to use, and is less complicated, more interpretable and robust. Moreover, the CSN framework is flexible, as the spline layer can be configured differently to yield different models. With different choices of the spline layer, we can reproduce or approximate a set of non-neural network models, including linear and spline-based statistical models, tree, rule-fit, tree-ensembles (gradient boosting trees, random forest), oblique tree/forests, multi-variate adaptive regression spline (MARS), SVM with polynomial kernel, etc. Therefore, CSN provides a unified modeling framework that puts the above set of non-neural network models under the same neural network framework. By using scalable and powerful gradient descent algorithms available in neural network libraries, CSN avoids some pitfalls (such as being ad-hoc, greedy or non-scalable) in the case-specific optimization methods used in the above non-neural network models. We will use a special type of CSN, TreeNet, to illustrate our point. We will compare TreeNet with XGBoost and FCNN to show the benefits of TreeNet. We believe CSN will provide a flexible and convenient framework for practitioners to build performant, robust and more interpretable models.

表格数据可解释性统一框架神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。