用SHAP值做回归，快速选出重要特征
Shap-Select: Lightweight Feature Selection Using SHAP Values and Regression
- 基于特征SHAP值在验证集上做回归，根据系数符号和显著性筛选特征
- 在信用卡欺诈数据集上，性能优于RFE、Boruta等主流方法
- 兼顾可解释性与效率，适合表格型数据的特征筛选
特征选择是机器学习中的关键步骤，尤其在高维数据场景下。它能降低模型复杂度、提升性能、缓解过拟合并减少计算时间。本文提出一种新型特征选择框架shap-select：在验证集上对目标变量进行线性或逻辑回归，以特征的Shapley值为输入，利用回归系数的符号和显著性水平，构建高效启发式特征筛选策略，适用于表格型回归与分类任务。我们在Kaggle信用卡欺诈数据集上评估了该方法，结果表明其在性能上优于递归特征消除（RFE）、基于互信息的HISEL、Boruta以及更简单的基于SHAP值的方法。实验显示，shap-select在可解释性、计算效率和预测性能之间实现了良好平衡，提供了一种稳健的特征选择解决方案。
原文摘要 · Abstract (English)
Feature selection is an essential process in machine learning, especially when dealing with high-dimensional datasets. It helps reduce the complexity of machine learning models, improve performance, mitigate overfitting, and decrease computation time. This paper presents a novel feature selection framework, shap-select. The framework conducts a linear or logistic regression of the target on the Shapley values of the features, on the validation set, and uses the signs and significance levels of the regression coefficients to implement an efficient heuristic for feature selection in tabular regression and classification tasks. We evaluate shap-select on the Kaggle credit card fraud dataset, demonstrating its effectiveness compared to established methods such as Recursive Feature Elimination (RFE), HISEL (a mutual information-based feature selection method), Boruta and a simpler Shapley value-based method. Our findings show that shap-select combines interpretability, computational efficiency, and performance, offering a robust solution for feature selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。