arXiv:2505.18697cs.LGcs.AI2025-05被引 5

用大模型缓解图学习中的遗忘问题,效果显著且方法简单。

Can LLMs Alleviate Catastrophic Forgetting in Graph Continual Learning? A Systematic Study

  • 将大语言模型用于图持续学习,避免从头训练
  • 新方法在无回放条件下性能比之前最优基线高约20%
  • 提出可复现的基准工具LLM4GCL,解决评估漏洞

现实世界数据,包括图结构数据,通常以流式方式持续到来,要求学习系统在不遗忘旧知识的前提下不断获取新知识。尽管已有大量工作尝试解决图机器学习中的灾难性遗忘问题,但这些方法均基于从头训练流式数据。随着预训练模型的兴起,越来越多研究利用其强大的泛化能力开展持续学习。因此,本文探讨大语言模型(LLMs)能否缓解图持续学习(GCL)中的灾难性遗忘。我们首先指出当前GCL实验设置存在严重缺陷,评估阶段可能导致任务ID泄露。随后,在更贴近真实场景的设定下评估了LLMs的表现,发现微小修改即可带来优异结果。基于大量实验,我们提出一种简单而有效的方案——简化的图持续学习(SimGCL),在无回放约束下性能相比之前的最先进图神经网络基线提升约20%。为促进可复现性,我们开发了易于使用的基准工具LLM4GCL,用于训练和评估现有GCL方法。代码已公开于:https://github.com/ZhixunLEE/LLM4GCL。

原文摘要 · Abstract (English)

Nowadays, real-world data, including graph-structure data, often arrives in a streaming manner, which means that learning systems need to continuously acquire new knowledge without forgetting previously learned information. Although substantial existing works attempt to address catastrophic forgetting in graph machine learning, they are all based on training from scratch with streaming data. With the rise of pretrained models, an increasing number of studies have leveraged their strong generalization ability for continual learning. Therefore, in this work, we attempt to answer whether large language models (LLMs) can mitigate catastrophic forgetting in Graph Continual Learning (GCL). We first point out that current experimental setups for GCL have significant flaws, as the evaluation stage may lead to task ID leakage. Then, we evaluate the performance of LLMs in more realistic scenarios and find that even minor modifications can lead to outstanding results. Finally, based on extensive experiments, we propose a simple-yet-effective method, Simple Graph Continual Learning (SimGCL), that surpasses the previous state-of-the-art GNN-based baseline by around 20% under the rehearsal-free constraint. To facilitate reproducibility, we have developed an easy-to-use benchmark LLM4GCL for training and evaluating existing GCL methods. The code is available at: https://github.com/ZhixunLEE/LLM4GCL.

图神经网络持续学习大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。