提出可解释神经网络SparXnet,自动筛选关键特征并保证预测性能。
Explainable Neural Networks with Guarantees: A Sparse Estimation Approach
- 通过稀疏特征组合构建模型,每项特征由单维输入经可训练函数生成。
- 在真实与合成数据上保持高预测性能,同时实现特征重要性直接可读。
- 理论证明泛化误差随选中特征数线性增长,适合需要可解释性的研究者。
平衡预测能力与可解释性是神经网络等复杂模型长期面临的挑战,因其非线性结构难以直接解读。本文提出SparXnet——一种新型可解释神经网络,通过线性组合一组稀疏且联合学习的特征实现。每个特征由单一1维输入经可训练函数生成,模型能自动选择关键特征,最终预测为这些特征的缩放线性组合。在合成与真实数据集上的实验表明,该方法在保持接近原始性能的同时,具备直接可解释性。理论分析显示,模型泛化界在所选特征数上线性增长,仅对输入特征数呈对数依赖;在极弱条件下,样本复杂度不再依赖参数量或架构细节。本研究为可解释稀疏神经网络的可保证设计开辟新路径。
原文摘要 · Abstract (English)
Balancing predictive power and interpretability has long been a challenging research area, particularly in powerful yet complex models like neural networks, where nonlinearity obstructs direct interpretation. This paper introduces a novel approach to constructing an explainable neural network that harmonizes predictiveness and explainability. Our model, termed SparXnet, is designed as a linear combination of a sparse set of jointly learned features, each derived from a different trainable function applied to a single 1-dimensional input feature. Leveraging the ability to learn arbitrarily complex relationships, our neural network architecture enables automatic selection of a sparse set of important features, with the final prediction being a linear combination of rescaled versions of these features. We demonstrate the ability to select significant features while maintaining comparable predictive performance and direct interpretability through extensive experiments on synthetic and real-world datasets. We also provide theoretical analysis on the generalization bounds of our framework, which is favorably linear in the number of selected features and only logarithmic in the number of input features. We further lift any dependence of sample complexity on the number of parameters or the architectural details under very mild conditions. Our research paves the way for further research on sparse and explainable neural networks with guarantee.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。