通过自适应剪枝优化图提示,提升模型效率与性能。
Heterogeneous Graph Prompt Learning via Adaptive Weight Pruning
- 基于重要性评估动态筛选正负图提示,分层剪枝去除无效提示。
- 在三个基准数据集上实现参数显著减少,节点分类性能保持竞争力。
- 适合关注图神经网络效率优化的研究者与工程应用开发者。
图神经网络(GNN)在节点分类、链接预测等任务中表现优异,但仍面临训练与推理耗时长、复杂关系捕捉难、特征提取不足等问题。为此,图预训练和图提示方法因其能利用大规模数据进行初始学习并实现任务适配而受到关注。然而,现有研究忽视了图提示对模型的优化潜力及其正负提示对模型稳定性与效率的影响。为此,本文提出一种结合图提示与权重剪枝的新框架GPAWP,旨在通过减少提示数量提升图提示的性能与效率。通过重要性评估函数,在不同粒度上识别正负提示权重,并采用分层剪枝策略剔除负提示标签,从而实现更参数高效的提示设计。在三个基准数据集上的大量实验表明,GPAWP显著降低了节点分类任务中的参数量,同时保持了优异的性能。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have achieved remarkable success in various graph-based tasks (e.g., node classification or link prediction). Despite their triumphs, GNNs still face challenges such as long training and inference times, difficulty in capturing complex relationships, and insufficient feature extraction. To tackle these issues, graph pre-training and graph prompt methods have garnered increasing attention for their ability to leverage large-scale datasets for initial learning and task-specific adaptation, offering potential improvements in GNN performance. However, previous research has overlooked the potential of graph prompts in optimizing models, as well as the impact of both positive and negative graph prompts on model stability and efficiency. To bridge this gap, we propose a novel framework combining graph prompts with weight pruning, called GPAWP, which aims to enhance the performance and efficiency of graph prompts by using fewer of them. We evaluate the importance of graph prompts using an importance assessment function to determine positive and negative weights at different granularities. Through hierarchically structured pruning, we eliminate negative prompt labels, resulting in more parameter-efficient and competitively performing prompts. Extensive experiments on three benchmark datasets demonstrate the superiority of GPAWP, leading to a significant reduction in parameters in node classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。