用神经网络动态稀疏输入层，通过首次“印象”判断特征重要性。
EntryPrune: Neural Network Feature Selection using First Impressions
- 基于神经元进入网络时的变化量进行特征筛选
- 在13个数据集上平均性能优于现有方法，低维数据提升明显
- 比传统剪枝更快，适合追求效率与可解释性的研究者
为提升预测模型的可解释性、降低计算开销并减少过拟合，研究者持续探索特征选择算法。神经网络因其结构优势成为构建特征选择方法的理想平台，近年涌现的神经元剪枝与再生技术为稀疏网络提供了新思路。本文提出EntryPrune，一种基于密集神经网络与动态稀疏输入层的新型监督特征选择算法。其核心是入口剪枝（entry-based pruning），通过比较神经元在首次进入网络时所引发的相对变化来评估特征重要性。在13个不同数据集上的大量实验表明，该方法普遍优于当前最优方法，尤其在低维数据集上显著提升平均准确率。此外，实验证明在相同框架下，入口剪枝优于传统权重大小剪枝，且运行时间低于对比方法。代码已开源：https://github.com/flxzimmer/entryprune。
原文摘要 · Abstract (English)
There is an ongoing effort to develop feature selection algorithms to improve interpretability, reduce computational resources, and minimize overfitting in predictive models. Neural networks stand out as architectures on which to build feature selection methods, and recently, neuron pruning and regrowth have emerged from the sparse neural network literature as promising new tools. We introduce EntryPrune, a novel supervised feature selection algorithm using a dense neural network with a dynamic sparse input layer. It employs entry-based pruning, a novel approach that compares neurons based on their relative change induced when they have entered the network. Extensive experiments on 13 different datasets show that our approach generally outperforms the current state-of-the-art methods, and in particular improves the average accuracy on low-dimensional datasets. Furthermore, we show that EntryPruning surpasses traditional techniques such as magnitude pruning within the EntryPrune framework and that EntryPrune achieves lower runtime than competing approaches. Our code is available at https://github.com/flxzimmer/entryprune.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。