通过优化代码本存储布局，让单张消费级显卡跑出百万级神经元自组织映射。
A Feature-Major Codebook for Memory-Efficient Sparse-Binary Self-Organizing Maps: Scaling a MEDLINE Atlas to 1.05 Million Neurons on a Single Consumer GPU
- 将代码本按特征优先排列，实现稀疏-密集乘法加速，大幅提升最近匹配单元搜索效率。
- 在24GB显存上训练512x512地图（26万神经元），其他方法均因内存超限失败。
- 首次在单卡上实现超百万神经元自组织映射，适合大规模医学文献可视化与探索。
自组织映射可将大规模语料转化为可浏览的二维知识图谱，但构建梅德林（MEDLINE）规模的地图曾不切实际：训练中占主导的最近匹配单元（BMU）搜索受限于每轮迭代必须读取整个代码本的带宽。本文发现该瓶颈主要源于代码本布局方式。采用特征优先存储（每特征权重连续存放），将搜索重构为分块稀疏-稠密矩阵乘法，使每次加载的权重列可在多个样本块中复用。仅改变布局，保持实现、精度和更新规则不变，便使BMU搜索速度提升4.5–8.5倍。由于精确的argmin BMU对存储方式不变，此加速无代价：保留误差在所有地图尺寸下与cuSPARSE基线相差小于0.5%。对比基线，加速优势为交叉而非恒定：小地图时cuSPARSE.SOM更快，128×128时快1.5倍，256×256时快2.6倍；在512×512时，唯独SparseBin.SOM能运行于24GB显存。结合半径无关的盒式模糊更新与基于收敛的停止策略，该方法在单张24GB GPU上64×64地图仅用72秒完成2990万篇梅德林文献的训练，并支持262,144神经元（512×512网格），远超所有测试替代算法。在141GB H200上更达1,048,576神经元（1024×1024网格），据我所知为迄今最大自组织映射。保留误差随地图尺寸呈平滑幂律变化，无拐点，表明分辨率极限由计算能力决定，而非数据本身。同等工作量下，设计比MedSOM（前代CUDA实现）快约82倍，128×128时比最佳多核CPU库快621倍。
原文摘要 · Abstract (English)
A self-organising map turns a large corpus into a browsable two-dimensional atlas, but building one at MEDLINE scale has been impractical: the best-matching-unit (BMU) search that dominates training is bound by the bandwidth needed to read the codebook every epoch. I show that this bottleneck is largely an artefact of codebook layout. Storing it feature-major with each feature's weights contiguous, W[v.M+i], recasts the search as a tiled sparse-dense product in which every loaded weight column is reused across a tile of samples. Varying only the layout, with implementation, precision and update rule held fixed, accelerates the BMU search by 4.5-8.5x. Because an exact-argmin BMU is invariant to how the codebook is stored, this gain costs nothing: held-out quantisation error agrees with a cuSPARSE baseline to within 0.5% at every map size. Against that baseline the advantage is a crossover rather than a constant: cuSPARSE.SOM is faster at small maps, SparseBin.SOM is 1.5x faster at 128x128 and 2.6x at 256x256, and at 512x512 it is the only one that runs at all on 24 GB. Paired with a radius-independent box-blur update and a convergence-based stopping rule, it trains a converged map over 29.9 million MEDLINE articles in about 72 s at 64x64 on one 24 GB GPU, and accommodates 262,144 neurons (512x512 edges) where every alternative algorithm I tested exceeds memory constraints. On a 141 GB H200 it reaches 1,048,576 neurons (1024x1024 edges) - to my knowledge the largest self-organising map yet reported. Held-out error follows a smooth power law with no elbow across three decades of map size, so the limit on resolution is compute rather than any breakpoint in the data. At matched work the design is ~82x faster than MedSOM, the CUDA implementation behind our earlier MEDLINE atlases and, at 128x128, 621x faster than the best available multicore-CPU library.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。