图神经网络训练中,剪枝稀疏化能大幅提速且不降精度。
Not All Neighbors Matter: Understanding the Impact of Graph Sparsification on GNN Pipelines
- 对图进行边缘剪枝,降低邻居规模以减少计算开销。
- 在PubMed上随机剪枝使GAT准确率提升6.8%,在Products上推理加速11.7倍。
- 剪枝代价很快被性能提升抵消,适合超大规模图应用。
随着图规模扩展至数十亿节点和边,图机器学习任务受限于多跳邻居遍历带来的高昂成本。尽管已有多种系统级与算法优化方案,数据管理与传输仍是大规模场景下的主要瓶颈。本文探讨图稀疏化——一种通过减少边数来压缩邻居规模的经典技术——是否可作为轻量预处理步骤,在保持节点分类准确率的同时缓解这些瓶颈。我们构建了一个可扩展的实验框架,系统评估不同稀疏化方法对GNN模型性能与精度的影响。这是首次对稀疏化图上GNN训练与推理的全面研究。结果表明:稀疏化通常能保持甚至提升预测性能;例如在PubMed图上,随机稀疏化使GAT模型准确率提升6.8%;随着规模增大,收益更显著,如在Products图上,K-邻域稀疏化使模型服务性能提升11.7倍,仅损失0.7%准确率。更重要的是,稀疏化计算开销可快速被性能增益覆盖,适用于超大规模图场景。
原文摘要 · Abstract (English)
As graphs scale to billions of nodes and edges, graph Machine Learning workloads are constrained by the cost of multi-hop traversals over exponentially growing neighborhoods. While various system-level and algorithmic optimizations have been proposed to accelerate Graph Neural Network (GNN) pipelines, data management and movement remain the primary bottlenecks at scale. In this paper, we explore whether graph sparsification, a well-established technique that reduces edges to create sparser neighborhoods, can serve as a lightweight pre-processing step to address these bottlenecks while preserving accuracy on node classification tasks. We develop an extensible experimental framework that enables systematic evaluation of how different sparsification methods affect the performance and accuracy of GNN models. We conduct the first comprehensive study of GNN training and inference on sparsified graphs, revealing several key findings. First, sparsification often preserves or even improves predictive performance. As an example, random sparsification raises the accuracy of the GAT model by 6.8% on the PubMed graph. Second, benefits increase with scale, substantially accelerating both training and inference. Our results show that the K-Neighbor sparsifier improves model serving performance on the Products graph by 11.7x with only a 0.7% accuracy drop. Importantly, we find that the computational overhead of sparsification is quickly amortized, making it practical for very large graphs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。