arXiv:2410.10082stat.MLcs.LG2024-10

fastHDMI加速高维数据互信息估计,提升脑影像变量筛选效率。

fastHDMI: Fast Mutual Information Estimation for High-Dimensional Data

  • 基于FFT-KDE与分箱法的互信息估计算法,适配不同数据类型。
  • 连续非线性结果中FFT-KDE表现最优,二值结果分箱法更优。
  • 适用于脑影像等高维数据筛选,实测高效且可提升模型预测力。

本文提出fastHDMI,一个用于高维数据(特别是脑影像数据)高效变量筛选的Python工具包。首次将三种互信息估计算法应用于脑影像变量选择,显著增强对复杂数据结构的分析能力。基于预处理后的ABIDE数据集,通过大量模拟实验评估方法性能,覆盖线性与非线性关联、连续与二值结果。结果显示:连续非线性结果中,基于FFT-KDE的互信息估计表现最佳;二值结果中非线性概率映射时,分箱法更优;线性条件下,皮尔逊相关与FFT-KDE在连续结果上表现相当,皮尔逊在二值线性概率映射中占优。基于ABIDE数据的案例研究进一步验证了fastHDMI的实际应用价值,所选变量能构建具有强预测能力的模型。该研究证实fastHDMI兼具计算效率与方法优势,丰富了脑影像分析工具集。

原文摘要 · Abstract (English)

In this paper, we introduce fastHDMI, a Python package designed for efficient variable screening in high-dimensional datasets, particularly neuroimaging data. This work pioneers the application of three mutual information estimation methods for neuroimaging variable selection, a novel approach implemented via fastHDMI. These advancements enhance our ability to analyze the complex structures of neuroimaging datasets, providing improved tools for variable selection in high-dimensional spaces. Using the preprocessed ABIDE dataset, we evaluate the performance of these methods through extensive simulations. The tests cover a range of conditions, including linear and nonlinear associations, as well as continuous and binary outcomes. Our results highlight the superiority of the FFTKDE-based mutual information estimation for feature screening in continuous nonlinear outcomes, while binning-based methods outperform others for binary outcomes with nonlinear probability preimages. For linear simulations, both Pearson correlation and FFTKDE-based methods show comparable performance for continuous outcomes, while Pearson excels in binary outcomes with linear probability preimages. A comprehensive case study using the ABIDE dataset further demonstrates fastHDMI's practical utility, showcasing the predictive power of models built from variables selected using our screening techniques. This research affirms the computational efficiency and methodological strength of fastHDMI, significantly enriching the toolkit available for neuroimaging analysis.

互信息变量筛选脑影像高维数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。