用低秩适配让图神经网络跨图迁移更高效,只调20%参数就有效。
GraphLoRA: Structure-Aware Contrastive Low-Rank Adaptation for Cross-Graph Transfer Learning
- 通过结构感知MMD对齐源与目标图的特征分布差异。
- 注入小规模可训练图网络,仅调20%参数即实现跨域性能提升。
- 适合标签稀疏场景,尤其适用于跨领域图数据迁移任务。
图神经网络(GNN)在电商、社交网络等领域的图分析任务中表现出色,但其迁移能力受限,尤其在面对不同分布的图数据时效果不佳。现有研究忽略了图数据间的分布差异,导致跨域迁移困难。受低秩适配(LoRA)在大语言模型中的成功启发,本文提出GraphLoRA,一种高效且参数高效的跨图迁移方法。首先引入结构感知最大均值差异(SMMD),对齐源图与目标图之间的节点特征分布;其次通过注入一个小型可训练图网络,弥补结构分布差距,同时缓解灾难性遗忘;此外,设计结构感知正则化目标,提升预训练模型在标签稀缺情况下的适应能力。在八个真实世界数据集上的实验表明,GraphLoRA仅需调整20%参数,即可超越14种基线方法,即使在分布差异较大的图域间仍表现优异。代码已开源。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have demonstrated remarkable proficiency in handling a range of graph analytical tasks across various domains, such as e-commerce and social networks. Despite their versatility, GNNs face significant challenges in transferability, limiting their utility in real-world applications. Existing research in GNN transfer learning overlooks discrepancies in distribution among various graph datasets, facing challenges when transferring across different distributions. How to effectively adopt a well-trained GNN to new graphs with varying feature and structural distributions remains an under-explored problem. Taking inspiration from the success of Low-Rank Adaptation (LoRA) in adapting large language models to various domains, we propose GraphLoRA, an effective and parameter-efficient method for transferring well-trained GNNs to diverse graph domains. Specifically, we first propose a Structure-aware Maximum Mean Discrepancy (SMMD) to align divergent node feature distributions across source and target graphs. Moreover, we introduce low-rank adaptation by injecting a small trainable GNN alongside the pre-trained one, effectively bridging structural distribution gaps while mitigating the catastrophic forgetting. Additionally, a structure-aware regularization objective is proposed to enhance the adaptability of the pre-trained GNN to target graph with scarce supervision labels. Extensive experiments on eight real-world datasets demonstrate the effectiveness of GraphLoRA against fourteen baselines by tuning only 20% of parameters, even across disparate graph domains. The code is available at https://github.com/AllminerLab/GraphLoRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。