通过特征选择提升神经加法模型的可扩展性与效率。
Neural Additive and Basis Models with Feature Selection and Interactions
- 在神经加法/基模型中加入可训练特征选择层，降低计算开销。
- 支持高维数据下使用双输入神经网络捕捉特征交互，计算成本显著下降。
- 适合需要可解释性且处理高维数据的机器学习场景。
深度神经网络在多个领域表现优异，但可解释性差。神经加法模型（NAM）和神经基模型（NBM）利用神经网络作为广义加性模型（GAM）中的非线性形状函数，兼具可解释性与良好性能。两者可通过GAM结构可视化各特征对预测的贡献。然而，在引入双输入神经网络建模特征交互或处理高维数据时，训练变得不可行，因计算资源需求急剧上升。本文提出在NAM和NBM中引入特征选择机制，通过在模型中加入可训练的特征选择层，并在训练过程中更新选择权重，实现高效计算。该方法简单有效，相比原始NAM/NBM显著降低计算成本与模型规模，同时支持在高维数据中使用双输入神经网络以捕获特征交互。实验表明，所提模型在计算效率上优于原版NAM/NBM，性能达到或超过当前主流GAM方法。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) exhibit attractive performance in various fields but often suffer from low interpretability. The neural additive model (NAM) and its variant called the neural basis model (NBM) use neural networks (NNs) as nonlinear shape functions in generalized additive models (GAMs). Both models are highly interpretable and exhibit good performance and flexibility for NN training. NAM and NBM can provide and visualize the contribution of each feature to the prediction owing to GAM-based architectures. However, when using two-input NNs to consider feature interactions or when applying them to high-dimensional datasets, training NAM and NBM becomes intractable due to the increase in the computational resources required. This paper proposes incorporating the feature selection mechanism into NAM and NBM to resolve computational bottlenecks. We introduce the feature selection layer in both models and update the selection weights during training. Our method is simple and can reduce computational costs and model sizes compared to vanilla NAM and NBM. In addition, it enables us to use two-input NNs even in high-dimensional datasets and capture feature interactions. We demonstrate that the proposed models are computationally efficient compared to vanilla NAM and NBM, and they exhibit better or comparable performance with state-of-the-art GAMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。