arXiv:2512.01678cs.LGcs.DC2025-12

Morphling通过代码生成实现高效图神经网络训练,显著提升性能与内存效率。

Morphling: Fast, Fused, and Flexible GNN Training at Scale

  • 自动生成适配CPU/GPU/分布式环境的优化代码,融合稀疏与密集计算
  • 平均提速20倍(CPU)、19倍(GPU)、6倍(分布式),峰值达66倍
  • 动态选择执行路径,内存占用降低最多15倍,适合大规模图模型训练

图神经网络(GNN)在硬件层面面临不规则、内存密集型图遍历与规则、计算密集型矩阵运算的双重挑战。尽管PyTorch Geometric(PyG)和Deep Graph Library(DGL)等框架注重易用性,但其依赖通用内核,导致缓存命中率低、内存移动频繁且中间分配过多。为此,我们提出Morphling,一个领域专用代码合成器,可将高层GNN描述编译为面向OpenMP、CUDA和MPI的便携式、后端定制化实现。它通过实例化一系列针对不同执行环境优化的架构感知原语来实现。Morphling还集成运行时稀疏感知执行引擎,基于输入特征统计动态选择稠密或稀疏执行路径,避免对零值条目进行无效计算。我们在十一组涵盖多样化图结构、特征维度与稀疏度的真实数据集上评估了Morphling,结果表明其在单轮训练吞吐量上相比PyG和DGL平均提升20倍(CPU)、19倍(GPU)、6倍(分布式),峰值速度提升达66倍。同时,其内存高效的布局设计使峰值内存消耗降低最多15倍,从而可在消费级硬件上实现大规模GNN训练。这些发现表明,特定于架构的代码合成是跨异构并行与分布式平台实现高性能GNN执行的有效且可扩展路径。

原文摘要 · Abstract (English)

Graph Neural Networks (GNNs) present a fundamental hardware challenge by fusing irregular, memory-bound graph traversals with regular, compute-intensive dense matrix operations. While frameworks such as PyTorch Geometric (PyG) and Deep Graph Library (DGL) prioritize high-level usability, they fail to address these divergent execution characteristics. As a result, they rely on generic kernels that suffer from poor cache locality, excessive memory movement, and substantial intermediate allocations. To address these limitations, we present Morphling, a domain-specific code synthesizer designed to bridge this gap. Morphling compiles high-level GNN specifications into portable, backend-specialized implementations targeting OpenMP, CUDA, and MPI. It achieves this by instantiating a library of optimized, architecture-aware primitives tailored to each execution environment. Morphling also incorporates a runtime sparsity-aware execution engine that dynamically selects dense or sparse execution paths using input feature statistics, reducing unnecessary computation on zero-valued entries. We evaluate Morphling on eleven real-world datasets spanning diverse graph structures, feature dimensionalities, and sparsity regimes. Morphling improves per-epoch training throughput by an average of 20X on CPUs, 19X on GPUs, and 6X in distributed settings over PyG and DGL, with peak speedups reaching 66X. Morphling's memory-efficient layouts further reduce peak memory consumption by up to 15X, enabling large-scale GNN training on commodity hardware. These findings demonstrate that specialized, architecture-aware code synthesis provides an effective and scalable path toward high-performance GNN execution across diverse parallel and distributed platforms.

图神经网络代码生成高性能计算内存优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。