用可学习提示弥补子图训练的信息缺口,让轻量GCN更准
PromptGCN: Bridging Subgraph Gaps in Lightweight GCNs
- 用可学习的提示嵌入获取全局图信息
- 在子图中附加提示,实现跨子图信息传递
- 兼容任意子图采样方法,提升准确率最多5.48%
图卷积网络(GCNs)广泛应用于社交网络和推荐系统等场景。然而,大规模图或深层聚合层会导致全批处理GCN占用大量GPU内存,主流显卡(如29GB)在ogbn-products图上运行5层模型时易出现内存溢出。子图采样方法通过将图分割为多个子图并逐个训练以降低内存消耗,但导致子图间存在信息断层,使模型精度下降。本文提出PromptGCN,一种基于提示的轻量级GCN模型,用于弥合子图间的差距。首先设计可学习的提示嵌入以获取全局信息;随后将提示嵌入附加到每个子图中,实现全局信息在子图间的传递。在七个大规模图上的实验表明,PromptGCN性能优于基线方法,尤其在Flickr数据集上相较子图采样方法最高提升5.48%的准确率。总体而言,PromptGCN可无缝集成于任意子图采样方法,构建更高精度的轻量级GCN模型。
原文摘要 · Abstract (English)
Graph Convolutional Networks (GCNs) are widely used in graph-based applications, such as social networks and recommendation systems. Nevertheless, large-scale graphs or deep aggregation layers in full-batch GCNs consume significant GPU memory, causing out of memory (OOM) errors on mainstream GPUs (e.g., 29GB memory consumption on the Ogbnproducts graph with 5 layers). The subgraph sampling methods reduce memory consumption to achieve lightweight GCNs by partitioning the graph into multiple subgraphs and sequentially training GCNs on each subgraph. However, these methods yield gaps among subgraphs, i.e., GCNs can only be trained based on subgraphs instead of global graph information, which reduces the accuracy of GCNs. In this paper, we propose PromptGCN, a novel prompt-based lightweight GCN model to bridge the gaps among subgraphs. First, the learnable prompt embeddings are designed to obtain global information. Then, the prompts are attached into each subgraph to transfer the global information among subgraphs. Extensive experimental results on seven largescale graphs demonstrate that PromptGCN exhibits superior performance compared to baselines. Notably, PromptGCN improves the accuracy of subgraph sampling methods by up to 5.48% on the Flickr dataset. Overall, PromptGCN can be easily combined with any subgraph sampling method to obtain a lightweight GCN model with higher accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。