用进化算法选特征,兼顾精度与效率。
Permutation-based multi-objective evolutionary feature selection for high-dimensional data
- 基于特征置换重要性,优化特征子集而非单个特征。
- 在24个数据集上优于9种主流方法,平衡准确率与计算开销。
- 适合高维数据中需解释性和高效性的研究者使用。
高维数据分析中,特征数量常远超样本数。有效特征选择不仅能提升模型性能和可解释性,还能降低计算成本并缓解过拟合风险。本文提出一种新方法,基于经典的特征置换重要性,但将其扩展为评估特征子集而非单个特征,更有效捕捉特征间交互对模型性能的影响。该方法采用多目标进化算法搜索候选特征子集,目标是最大化特征被随机打乱时模型性能的下降程度,同时最小化特征子集的大小。在24个公开的高维分类与回归数据集上验证了该方法的有效性,并与9种主流高维特征选择方法(包括传统置换重要性方法)进行对比。结果表明,该方法在准确性与计算效率之间取得了良好平衡,为复杂高维数据提供了强有力的特征选择工具。
原文摘要 · Abstract (English)
Feature selection is a critical step in the analysis of high-dimensional data, where the number of features often vastly exceeds the number of samples. Effective feature selection not only improves model performance and interpretability but also reduces computational costs and mitigates the risk of overfitting. In this context, we propose a novel feature selection method for high-dimensional data, based on the well-known permutation feature importance approach, but extending it to evaluate subsets of attributes rather than individual features. This extension more effectively captures how interactions among features influence model performance. The proposed method employs a multi-objective evolutionary algorithm to search for candidate feature subsets, with the objectives of maximizing the degradation in model performance when the selected features are shuffled, and minimizing the cardinality of the feature subset. The effectiveness of our method has been validated on a set of 24 publicly available high-dimensional datasets for classification and regression tasks, and compared against 9 well-established feature selection methods designed for high-dimensional problems, including the conventional permutation feature importance method. The results demonstrate the ability of our approach in balancing accuracy and computational efficiency, providing a powerful tool for feature selection in complex, high-dimensional datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。