aweSOM加速大规模数据聚类，支持多维高通量分析
aweSOM: a CPU/GPU-accelerated Self-organizing Map and Statistically Combined Ensemble Framework for Machine-learning Clustering Analysis
- 基于自组织映射的可扩展框架，融合CPU/GPU并行计算
- 处理超百万级数据点，速度比旧方案快10-100倍，内存效率更高
- 适合需要高效聚类的大规模生物、天文等科学数据研究者
我们提出aweSOM，一个开源的Python机器学习聚类与分类工具包，采用自组织映射（SOM）算法，并结合CPU/GPU加速，适用于大规模（N > 10^6，N为数据点数量）、多维数据集。aweSOM包含两个核心模块：一个负责SOM的初始化与训练，另一个通过集成多个SOM结果实现更稳健的聚类。现有基于Python的SOM实现（如POPSOM、MiniSom、sklearn-som）主要作为概念验证，针对小规模数据优化，缺乏对大规模多维数据的扩展能力。aweSOM填补了这一空白，性能可扩展至约10^8个数据点，支持每点多个特征。与基准实现对比，代码性能提升10-100倍，且内存效率显著改善，得益于多项内置优化。
原文摘要 · Abstract (English)
We introduce aweSOM, an open-source Python package for machine learning (ML) clustering and classification, using a Self-organizing Maps (SOM) algorithm that incorporates CPU/GPU acceleration to accommodate large ($N > 10^6$, where $N$ is the number of data points), multidimensional datasets. aweSOM consists of two main modules, one that handles the initialization and training of the SOM, and another that stacks the results of multiple SOM realizations to obtain more statistically robust clusters. Existing Python-based SOM implementations (e.g., POPSOM, Yuan (2018); MiniSom, Vettigli (2018); sklearn-som) primarily serve as proof-of-concept demonstrations, optimized for smaller datasets, but lacking scalability for large, multidimensional data. aweSOM provides a solution for this gap in capability, with good performance scaling up to $\sim 10^8$ individual points, and capable of utilizing multiple features per point. We compare the code performance against the legacy implementations it is based on, and find a 10-100x speed up, as well as significantly improved memory efficiency, due to several built-in optimizations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。