用四元数压缩图神经网络,参数减到1/4仍保持精度。
Graph Neural Networks at a Fraction
- 用四元数空间计算节点表示,实现模型压缩
- 参数量仅为原模型的1/4,性能接近原始模型
- 提出新视角的图彩票理论,进一步减少可训练参数
图神经网络(GNNs)已成为学习图结构数据表示的强大工具。除了实数域的GNN,四元数GNN在图结构任务中也表现良好。为降低能耗,本文在保持准确率的前提下缩小模型规模。提出四元数消息传递神经网络(QMPNN),利用四元数空间计算节点表示,可在参数量仅为原模型1/4的情况下,实现与原始模型相当的性能。此外,本文从新角度重新定义了图彩票理论(Graph Lottery Tickets),旨在从原GNN子网络中找到特定初始化,使其在训练后达到与原模型相当的性能,从而进一步减少可训练参数。通过在真实世界数据集上对节点分类、链接预测和图分类三个基础任务的实验,验证了所提QMPNN框架及图彩票理论在GNN与QMPNN中的有效性。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have emerged as powerful tools for learning representations of graph-structured data. In addition to real-valued GNNs, quaternion GNNs also perform well on tasks on graph-structured data. With the aim of reducing the energy footprint, we reduce the model size while maintaining accuracy comparable to that of the original-sized GNNs. This paper introduces Quaternion Message Passing Neural Networks (QMPNNs), a framework that leverages quaternion space to compute node representations. Our approach offers a generalizable method for incorporating quaternion representations into GNN architectures at one-fourth of the original parameter count. Furthermore, we present a novel perspective on Graph Lottery Tickets, redefining their applicability within the context of GNNs and QMPNNs. We specifically aim to find the initialization lottery from the subnetwork of the GNNs that can achieve comparable performance to the original GNN upon training. Thereby reducing the trainable model parameters even further. To validate the effectiveness of our proposed QMPNN framework and LTH for both GNNs and QMPNNs, we evaluate their performance on real-world datasets across three fundamental graph-based tasks: node classification, link prediction, and graph classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。