arXiv:2607.24145cs.LG2026-07

对比全局选特征与逐轮剔除法,发现后者更准但更慢。

An Empirical Study of Feature Selection Granularity

论文配图:An Empirical Study of Feature Selection Granularity
图 1 · 摘自论文原文
  • 用逐轮剔除方式重做五种特征选择算法,每步重新评估重要性。
  • 新方法在多数指标上表现更好,提升特征选择质量。
  • 适合追求精度、能接受更高计算成本的研究者参考。

特征选择旨在识别对数据结构或下游任务最有信息量的特征。现有研究多聚焦于新算法、评估指标或基准测试。本文从算法设计视角出发,考察常规方法——对全集特征一次性全局评分并选前k个——是否因噪声特征干扰而掩盖真实重要特征。为此,我们采用五种不同算法,分别在传统全局选择和贪婪递归剔除两种设计下实现,并在多种标准评价指标上分析算法选择的影响。实验结果表明,递归剔除法几乎始终提升整体特征选择质量,尽管计算开销更高,验证了高维数据中‘维度诅咒’会掩盖缓解策略的真相。

原文摘要 · Abstract (English)

Feature selection aims to identify the most informative and relevant features for a given dataset, either in terms of capturing the underlying data structure and distribution better, or with respect to the performance on a downstream task. Existing research in this area has largely focused on developing novel algorithms (in both supervised and unsupervised settings), proposing new evaluation metrics and frameworks, or benchmarking the performance of existing methods. In this work, we examine feature selection through an algorithmic design perspective. Conventional feature selection algorithms typically compute feature importance scores globally across the entire feature set and then select the top-ranked features in a single step. However, this approach raises a critical question: Can the presence of less informative (or noisy) features mask or obscure the true importance of other, more relevant features? In other words, would a recursive strategy, where features are removed one by one while re-evaluating importance at each step, yield different and potentially better results than the standard global ranking approach? To answer this question, we conduct an extensive empirical study using five diverse feature selection algorithms. We implement each algorithm under both the conventional global selection design and the greedy recursive elimination design. We then analyze the impact of this algorithmic choice, both individually for each method and collectively across all methods, on a range of standard feature selection evaluation metrics. The empirical evaluation results show that the greedy approach improves the overall feature selection quality almost consistently, albeit on the expense of higher computational cost, supporting our initial expectation that the curse of dimensionality also obscures the ways of mitigating it.

特征选择算法设计实验研究

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。