自动选出最少必要特征,兼顾精度与效率。
AutoNFS: Automatic Neural Feature Selection
- 用可微分采样自动确定最优特征数
- 在多个数据集上选更少特征仍保持高精度
- 适合高维表格数据的自动化特征筛选
特征选择(FS)是机器学习中的基础挑战,尤其在高维表格数据中,可解释性与计算效率至关重要。现有方法通常无法自动确定任务所需特征数量,需人工干预或多次重训不同特征预算。且滤波法忽略特征关系,包裹法需耗时优化。为此,我们提出AutoNFS,结合基于Gumbel-Sigmoid采样的特征选择模块与可预测特征相关性的评估模型,通过可微分损失端到端训练,自动确定解决下游任务所需的最小特征集。不同于多数包裹式方法,AutoNFS具有低且可预测的训练开销,避免跨特征预算重复训练。实际中,掩码模块的额外成本基本与输入特征数无关(仅含不可避免的输入处理成本),使其可扩展至高维表格数据。我们在经典分类与回归基准及真实世界宏基因组数据集上评估,结果表明AutoNFS在多数情况下优于或媲美强基线,同时平均选取更少特征。
原文摘要 · Abstract (English)
Feature selection (FS) is a fundamental challenge in machine learning, particularly for high-dimensional tabular data, where interpretability and computational efficiency are critical. Existing FS methods often cannot automatically detect the number of attributes required to solve a given task and involve user intervention or model retraining with different feature budgets. Additionally, they either neglect feature relationships (filter methods) or require time-consuming optimization (wrapper methods). To address these limitations, we propose AutoNFS, which combines the FS module based on Gumbel-Sigmoid sampling with a predictive model evaluating the relevance of the selected attributes. The model is trained end-to-end using a differentiable loss and automatically determines the minimal set of features essential to solve a given downstream task. Unlike many wrapper-style approaches, AutoNFS introduces a low and predictable training overhead and avoids repeated model retraining across feature budgets. In practice, the additional cost of the masking module is largely independent of the number of input features (beyond the unavoidable cost of processing the input itself), making the method scalable to high-dimensional tabular data. We evaluate AutoNFS on well-established classification and regression benchmarks as well as real-world metagenomic datasets. The results show that AutoNFS is competitive with, and often improves upon, strong classical and neural FS baselines while selecting fewer features on average across the evaluated benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。