通过稀疏分解加速GNN推理,降低动态图预测的计算开销。
Sparse Decomposition of Graph Neural Networks
- 选取邻域子集,用线性变换特征加权组合近似节点表示。
- 实现与平均度数和层数线性相关的复杂度,推理速度显著提升。
- 适用于需要在线预测的动态图任务,兼顾精度与效率。
图神经网络(GNN)在图表示学习中表现优异,但其推理成本较高,主要源于聚合操作需访问大量节点,成为部署支持在线预测、反映动态节点特征的GNN模型的主要障碍。为此,本文提出一种稀疏分解方法,通过减少聚合过程中的节点数量来降低开销。该方法学习用精心选择的扩展邻域内节点子集的线性变换特征加权组合,来近似节点表示。该方法实现了与平均节点度和网络层数呈线性关系的复杂度。我们设计了算法以计算最优稀疏分解参数,确保对原始GNN模型的准确逼近,并提出了有效策略以缩短训练时间并提升学习效果。大量实验表明,本方法在节点分类与时空预测任务中均优于其他针对推理加速设计的基线模型,在相近推理时间内取得显著精度提升。
原文摘要 · Abstract (English)
Graph Neural Networks (GNN) exhibit superior performance in graph representation learning, but their inference cost can be high, due to an aggregation operation that can require a memory fetch for a very large number of nodes. This inference cost is the major obstacle to deploying GNN models with \emph{online prediction} to reflect the potentially dynamic node features. To address this, we propose an approach to reduce the number of nodes that are included during aggregation. We achieve this through a sparse decomposition, learning to approximate node representations using a weighted sum of linearly transformed features of a carefully selected subset of nodes within the extended neighbourhood. The approach achieves linear complexity with respect to the average node degree and the number of layers in the graph neural network. We introduce an algorithm to compute the optimal parameters for the sparse decomposition, ensuring an accurate approximation of the original GNN model, and present effective strategies to reduce the training time and improve the learning process. We demonstrate via extensive experiments that our method outperforms other baselines designed for inference speedup, achieving significant accuracy gains with comparable inference times for both node classification and spatio-temporal forecasting tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。