arXiv:2509.23366cs.LG2025-09被引 3

用样条网络分析特征重要性,提升表格数据降维的可解释性。

Splines-Based Feature Importance in Kolmogorov-Arnold Networks: A Framework for Supervised Tabular Data Dimensionality Reduction

  • 基于样条的KAN模型自动计算每特征重要性得分。
  • 在分类与回归任务中表现优于或媲美传统方法,尤其擅长捕捉非线性关系。
  • 适合需要可解释降维的表格数据场景,如金融、医疗建模。

特征选择是许多表格预测问题的关键步骤,其中多个候选变量可能存在冗余、噪声或弱信息性。本文研究基于柯尔莫哥洛夫-阿诺德网络(KAN)的特征选择方法,该网络以样条参数化特征变换,并自然地暴露每个特征的重要性分数。基于此思路,我们提出了四种KAN-based选择标准(系数范数、梯度显著性、击退分数),并在一系列真实与合成的分类与回归数据集上,与LASSO、随机森林特征重要性、互信息及SVM-RFE等经典方法进行对比。通过在三个特征保留率(20%、40%、60%)下的平均F1和$R^2$得分评估,发现KAN-based选择器普遍具有竞争力,甚至在某些情况下更优。在分类任务中,其能有效剔除冗余特征并捕捉非线性交互,在多类任务中常达到或超越现有方法;在回归任务中,对噪声和异质数据表现稳健,性能接近强集成学习模型;同时观察到$\ ext{ℓ}_1$准则存在过度剪枝的典型失效模式。稳定性与冗余性分析表明,KAN-based选择器在不同交叉验证折中具有可复现的特征子集,避免不必要相关性膨胀,确保可靠且非冗余的变量选择。总体而言,本研究证明了基于KAN的特征选择是一种强大且可解释的替代方案,能够揭示超越稀疏性或纯度度量的非线性与多变量特征相关性。

原文摘要 · Abstract (English)

Feature selection is a key step in many tabular prediction problems, where multiple candidate variables may be redundant, noisy, or weakly informative. We investigate feature selection based on Kolmogorov-Arnold Networks (KANs), which parameterize feature transformations with splines and expose per-feature importance scores in a natural way. From this idea we derive four KAN-based selection criteria (coefficient norms, gradient-based saliency, and knockout scores) and compare them with standard methods such as LASSO, Random Forest feature importance, Mutual Information, and SVM-RFE on a suite of real and synthetic classification and regression datasets. Using average F1 and $R^2$ scores across three feature-retention levels (20%, 40%, 60%), we find that KAN-based selectors are generally competitive with, and sometimes superior to, classical baselines. In classification, KAN criteria often match or exceed existing methods on multi-class tasks by removing redundant features and capturing nonlinear interactions. In regression, KAN-based scores provide robust performance on noisy and heterogeneous datasets, closely tracking strong ensemble predictors; we also observe characteristic failure modes, such as overly aggressive pruning with an $\ell_1$ criterion. Stability and redundancy analyses further show that KAN-based selectors yield reproducible feature subsets across folds while avoiding unnecessary correlation inflation, ensuring reliable and non-redundant variable selection. Overall, our findings demonstrate that KAN-based feature selection provides a powerful and interpretable alternative to traditional methods, capable of uncovering nonlinear and multivariate feature relevance beyond sparsity or impurity-based measures.

特征选择表格数据可解释性样条网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。