用多头注意力融合统计方法,提升高维生物数据特征选择的稳定性和可解释性。
MAFS: Multi-head Attention Feature Selection for High-Dimensional Data via Deep Fusion of Filter Methods
- 结合滤波法先验与多头注意力,从多角度捕捉特征复杂关系。
- 在癌症和阿尔茨海默病数据上,覆盖率达90%以上且排名稳定。
- 适合需要可解释性与高效性的高维生物医学分析场景。
特征选择对高维生物医学数据至关重要,可提升预测性能、降低计算成本并增强可解释性。现有方法存在局限:滤波法虽可扩展但难以捕捉复杂关系与冗余;深度学习方法虽能建模非线性模式,却常缺乏稳定性、可解释性与规模化效率。单头注意力虽提高可解释性,但受限于多层级依赖捕获能力,且对初始化敏感,影响复现性。多数方法未能有效结合统计可解释性与深度学习表征能力,尤其在超高维场景。本文提出MAFS(基于多头注意力的特征选择),融合统计先验与深度学习优势。MAFS以滤波法实现稳定初始化并指导学习,通过多头注意力并行分析特征的多维度关系,捕捉复杂非线性交互。再经重排模块整合各注意力头输出,解决冲突、减少信息损失,生成鲁棒一致的特征排序。该设计兼顾统计引导与深度建模能力,提供可解释的重要性评分,同时最大化保留信息信号。在模拟及真实数据集(包括癌症基因表达与阿尔茨海默病数据)上,MAFS持续优于现有滤波与深度学习方法,展现更高覆盖率与更强稳定性,为高维生物医学数据提供可扩展、可解释且稳健的特征选择方案。
原文摘要 · Abstract (English)
Feature selection is essential for high-dimensional biomedical data, enabling stronger predictive performance, reduced computational cost, and improved interpretability in precision medicine applications. Existing approaches face notable challenges. Filter methods are highly scalable but cannot capture complex relationships or eliminate redundancy. Deep learning-based approaches can model nonlinear patterns but often lack stability, interpretability, and efficiency at scale. Single-head attention improves interpretability but is limited in capturing multi-level dependencies and remains sensitive to initialization, reducing reproducibility. Most existing methods rarely combine statistical interpretability with the representational power of deep learning, particularly in ultra-high-dimensional settings. Here, we introduce MAFS (Multi-head Attention-based Feature Selection), a hybrid framework that integrates statistical priors with deep learning capabilities. MAFS begins with filter-based priors for stable initialization and guide learning. It then uses multi-head attention to examine features from multiple perspectives in parallel, capturing complex nonlinear relationships and interactions. Finally, a reordering module consolidates outputs across attention heads, resolving conflicts and minimizing information loss to generate robust and consistent feature rankings. This design combines statistical guidance with deep modeling capacity, yielding interpretable importance scores while maximizing retention of informative signals. Across simulated and real-world datasets, including cancer gene expression and Alzheimer's disease data, MAFS consistently achieves superior coverage and stability compared with existing filter-based and deep learning-based alternatives, offering a scalable, interpretable, and robust solution for feature selection in high-dimensional biomedical data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。