arXiv:2412.07605cs.LG2024-12AAAI被引 1

提出快速寻优的单次剪枝框架,让GNN模型在更稀疏下更快运行且性能不降。

Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks

  • 用单次剪枝+去噪框架替代迭代剪枝,实现快速定位高性能子网络。
  • 在多个数据集上实现1.32%~45.62%权重稀疏度提升,图稀疏度提升7.49%~22.71%。
  • 相比传统方法速度提升1.7~44倍,计算量节省95.3%~98.6%,适合大规模GNN部署。

图神经网络(GNN)在各类图学习任务中表现优异,但其在大规模图上的实际应用受限于计算开销。为解决此问题,图彩票假说(GLT)主张识别不损失性能的子图与子网络(即胜出票)。现有方法依赖迭代幅度剪枝(IMP),虽效果好但计算成本高。本文重新评估单次剪枝与IMP的关系:尽管单次剪枝结果次优,但能快速通向更高性能的胜出票。为此提出一种单次剪枝与去噪框架,验证了该‘快速通道’的有效性。相比主流IMP-based GLT方法,本框架在保持性能的同时实现更高稀疏度与更快速度。在4种骨干网络和6个数据集上实验显示,权重稀疏度提升1.32%~45.62%,图稀疏度提升7.49%~22.71%,推理速度提升1.7~44倍,计算量(MAC)节省95.3%~98.6%。

原文摘要 · Abstract (English)

Graph Neural Networks (GNNs) demonstrate superior performance in various graph learning tasks, yet their wider real-world application is hindered by the computational overhead when applied to large-scale graphs. To address the issue, the Graph Lottery Hypothesis (GLT) has been proposed, advocating the identification of subgraphs and subnetworks, \textit{i.e.}, winning tickets, without compromising performance. The effectiveness of current GLT methods largely stems from the use of iterative magnitude pruning (IMP), which offers higher stability and better performance than one-shot pruning. However, identifying GLTs is highly computationally expensive, due to the iterative pruning and retraining required by IMP. In this paper, we reevaluate the correlation between one-shot pruning and IMP: while one-shot tickets are suboptimal compared to IMP, they offer a \textit{fast track} to tickets with a stronger performance. We introduce a one-shot pruning and denoising framework to validate the efficacy of the \textit{fast track}. Compared to current IMP-based GLT methods, our framework achieves a double-win situation of graph lottery tickets with \textbf{higher sparsity} and \textbf{faster speeds}. Through extensive experiments across 4 backbones and 6 datasets, our method demonstrates $1.32\% - 45.62\%$ improvement in weight sparsity and a $7.49\% - 22.71\%$ increase in graph sparsity, along with a $1.7-44 \times$ speedup over IMP-based methods and $95.3\%-98.6\%$ MAC savings.

图神经网络模型剪枝稀疏化加速推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。