arXiv:2602.08913cs.LGstat.ML2026-02

提出GEMSS方法,从高维数据中同时发现多个稀疏且有效的特征组合。

GEMSS: A Variational Method for Discovering Multiple Sparse Solutions in Classification and Regression Problems

  • 用变分混合模型捕捉多重稀疏解的后验分布,自动避免解之间的重复。
  • 在合成数据上比现有方法更准确识别真实解集,尤其在解重叠时优势更大。
  • 适用于需要可解释性分析的科研或医疗场景,适合关注特征选择多样性的研究者。

在欠定回归与分类问题中,多个不同的特征子集常能产生等效预测性能。在实际应用中,特别是在样本数远小于特征数($n \\< p$)、高维或存在共线性的情况下,向领域专家提供一组统计上合理的解释,而非单一任意解,具有重要价值。为此,本文提出高斯集成多重稀疏解方法(GEMSS),通过单个变分混合模型近似对应的多模态后验分布。其证据下界内置了成分间的排斥机制,使模型能够同时生成多个显著不同的稀疏解。我们在一个新型可复用基准上评估GEMSS,该基准的真值解集及其结构由构造决定,采用解集级恢复指标进行评估。结果表明,GEMSS始终优于专用多重解方法(Enumeration LASSO、ALFESE)、两个强大采样基线(Randomized-LASSO ensemble、BB-SSL)以及朴素迭代掩码法。随着解之间重叠程度增加,差距进一步扩大,额外集成重启无法弥补。仅ALFESE表现相当。此外,GEMSS在真实数据集上也成功生成多个差异明显且高度预测的解,达成了现有方法难以实现的实用目标。开源Python包'gemss'已发布(github.com/kat-er-ina/gemss),并通过Hugging Face免费在线应用(huggingface.co/spaces/kat-er-ina/gemss)实现普及。

原文摘要 · Abstract (English)

In underdetermined regression and classification problems, multiple feature subsets often yield equivalent predictive performance. In applied settings, especially with $n \ll p$, high dimension or collinearities, it is valuable to provide a domain expert with a menu of statistically plausible explanations, rather than one arbitrary solution. This creates the need for appropriate methods. We present Gaussian Ensemble for Multiple Sparse Solutions (GEMSS), a method that uses a single variational mixture to approximate the corresponding multimodal posterior. Its evidence lower bound contains a built-in repulsion between the mixture's components, enabling the model to simultaneously produce several distinct sparse solutions. We evaluate GEMSS on a novel, reusable benchmark. The ground-truth solution set and its structure are known by construction and set-level recovery metrics are evaluated. GEMSS consistently outperforms dedicated multiplicity methods (Enumeration LASSO, ALFESE), two strong sampling baselines that approximate the same posterior (Randomized-LASSO ensemble, BB-SSL), and naive iterative masking. As solutions' overlap increases, the gap widens and additional ensemble restarts cannot close it. Only ALFESE proves competitive. Further, GEMSS is validated on real-world datasets, producing multiple distinct and highly predictive solutions: the practical goal that existing methods struggle to meet. The open-source Python package 'gemss' is available (github.com/kat-er-ina/gemss) and democratized through a free online application at huggingface.co/spaces/kat-er-ina/gemss.

特征选择稀疏解变分推断多解发现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。