arXiv:2509.10853stat.MLcs.LG2025-09被引 2

用相对重要性排序选变量,比传统方法更准更稳。

Variable Selection Using Relative Importance Rankings

  • 用相对重要性衡量变量,兼顾直接与联合效应。
  • 在高相关变量簇中表现优于lasso等主流方法。
  • 适合基因表达等高维数据,代码开源可用。

尽管概念相关,变量选择与相对重要性(RI)分析在文献中却常被分开处理。本文探索将RI用于模型构建前的特征排序与过滤式选择。我们提出新度量CRI.Z,计算效率高于传统方法。通过大量模拟实验发现,当存在抑制或弱预测变量时,RI比边际相关性更准确地排序变量;基于该排序构建的预测模型性能优异,常超越lasso和松弛lasso等先进线性模型。尤其在高度相关变量簇场景下,本方法表现突出,而这类情况正是许多基准方法失效的根源。在两个高维基因表达数据集上进一步验证了方法的实际效用与高效性。虽然lasso方法近年主导变量选择研究,但本研究表明,基于相对重要性的方法是一种强大且具竞争力的替代方案。代码已公开:https://github.com/tien-endotchang/RI-variable-selection。

原文摘要 · Abstract (English)

Although conceptually related, variable selection and relative importance (RI) analysis have been treated quite differently in the literature. While RI is typically used for post-hoc model explanation, this paper explores its potential for variable or feature ranking and filter-based selection before model creation. Specifically, we anticipate strong performance from the RI measures because they incorporate both direct and combined effects of predictors, addressing a key limitation of marginal correlation, which ignores dependencies among predictors. We implement and evaluate the RI-based variable ranking and selection methods, including a newly proposed RI measure, CRI.Z, with improved computational efficiency relative to conventional RI measures. Through extensive simulations, we first demonstrate how the RI measures more accurately rank the variables than the marginal correlation, especially when there are suppressed or weak predictors. We then show that predictive models built on these rankings are highly competitive, often outperforming state-of-the-art linear-model methods such as the lasso and relaxed lasso. The proposed RI-based methods are particularly effective in challenging cases involving clusters of highly correlated predictors, a setting known to cause failures in many benchmark methods. The practical utility and efficiency of RI-based methods are further demonstrated through two high-dimensional gene expression datasets. Although lasso methods have dominated the recent literature on variable selection, our study reveals that the RI-based method is a powerful and competitive alternative. We believe these underutilized tools deserve greater attention in statistics and machine learning communities. The code is available at: https://github.com/tien-endotchang/RI-variable-selection.

变量选择相对重要性高维数据基因表达

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。