arXiv:2508.04646stat.MEcs.LG2025-08

解决特征共线性下Lasso选择不稳定问题,区分真实与虚假相关特征。

Accept-Reject Lasso

  • 基于子集聚类分析特征选择模式,划分真实与虚假相关特征
  • 对真实相关特征只选一个代表,避免多重共线性;接受可能被误删的虚假相关特征
  • 适合高相关性特征场景下的稳健变量选择,尤其适用于生物医学等复杂数据

Lasso方法在高度相关特征存在时表现出不稳定性,常导致预测变量选择任意。主要表现为两类错误:遗漏无真正替代关系的特征(虚假冗余特征)和包含具有真实替代关系的特征(真实冗余特征)。现有方法多仅解决其中一类问题,本文提出新的接受-拒绝Lasso(ARL),通过细粒度分析数据子集上的特征选择行为,构建接受-拒绝框架。该框架将集成方法输出划分为有益与有害成分。核心挑战在于变量间相关性掩盖了信息真实来源。ARL首先利用聚类识别数据子集中的结构差异,再分析Lasso在不同子集中的表现,以区分真实与虚假相关性。对于因真实相关性导致多重共线性的特征,ARL倾向于选择单一代表特征并拒绝其余;而对于随子集变化而消失的虚假相关特征,即使被常规Lasso遗漏,也予以接受。真实与虚假相关性的不同模式形成可分离的判别边界,通过设定合适阈值,可有效区分二者,在最大化纳入信息变量的同时最小化引入有害变量。通过大量模拟与真实数据实验验证了方法的有效性。

原文摘要 · Abstract (English)

The Lasso method is known to exhibit instability in the presence of highly correlated features, often leading to an arbitrary selection of predictors. This issue manifests itself in two primary error types: the erroneous omission of features that lack a true substitutable relationship (falsely redundant features) and the inclusion of features with a true substitutable relationship (truly redundant features). Although most existing methods address only one of these challenges, we introduce the Accept-Reject Lasso (ARL), a novel approach that resolves this dilemma. ARL operationalizes an Accept-Reject framework through a fine-grained analysis of feature selection across data subsets. This framework is designed to partition the output of an ensemble method into beneficial and detrimental components through fine-grained analysis. The fundamental challenge for Lasso is that inter-variable correlation obscures the true sources of information. ARL tackles this by first using clustering to identify distinct subset structures within the data. It then analyzes Lasso's behavior across these subsets to differentiate between true and spurious correlations. For truly correlated features, which induce multicollinearity, ARL tends to select a single representative feature and reject the rest to ensure model stability. Conversely, for features linked by spurious correlations, which may vanish in certain subsets, ARL accepts those that Lasso might have incorrectly omitted. The distinct patterns arising from true versus spurious correlations create a divisible separation. By setting an appropriate threshold, our framework can effectively distinguish between these two phenomena, thereby maximizing the inclusion of informative variables while minimizing the introduction of detrimental ones. We illustrate the efficacy of the proposed method through extensive simulation and real-data experiments.

特征选择Lasso共线性变量筛选

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。