提出无需重放与遗忘的图分类增量学习方法,通过图任务原型和提示机制实现精准任务识别与性能提升。
Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach
- 基于拉普拉斯平滑构建图任务原型,实现100%任务ID预测准确率
- 提出轻量级图提示学习,使模型在无数据重放下仍保持高性能
- 在4个基准上显著超越现有方法,且完全避免灾难性遗忘
类增量学习(CIL)旨在持续学习一系列任务,每个任务包含一组唯一类别。图类增量学习(GCIL)遵循相同设定,但需处理图任务(如图上的节点分类)。其核心挑战在于推理时缺乏任务标识符,导致不同任务间类别难以区分。本文理论上证明,通过基于拉普拉斯平滑的图任务原型方法,可实现图数据上准确的任务ID预测:同一任务的原型在大平滑步长下趋于一致,而不同任务因图结构与节点属性差异而明显分离。为进一步避免先前图任务知识的灾难性遗忘,提出一种新型图提示方法,为每个任务学习一个小型判别式图提示,本质上为每任务构建独立分类模型。该提示学习仅需在首任务上训练一次图神经网络(GNN),后续无需数据重放,实现完全免重放、免遗忘的GCIL模型。在四个GCIL基准上的实验表明:i) 任务原型方法在所有四数据集上达到100%任务ID预测准确率;ii) GCIL模型平均精度较现有最优方法至少提升18%;iii) 模型在四个数据集上完全无遗忘。
原文摘要 · Abstract (English)
Class-incremental learning (CIL) aims to continually learn a sequence of tasks, with each task consisting of a set of unique classes. Graph CIL (GCIL) follows the same setting but needs to deal with graph tasks (e.g., node classification in a graph). The key characteristic of CIL lies in the absence of task identifiers (IDs) during inference, which causes a significant challenge in separating classes from different tasks (i.e., inter-task class separation). Being able to accurately predict the task IDs can help address this issue, but it is a challenging problem. In this paper, we show theoretically that accurate task ID prediction on graph data can be achieved by a Laplacian smoothing-based graph task profiling approach, in which each graph task is modeled by a task prototype based on Laplacian smoothing over the graph. It guarantees that the task prototypes of the same graph task are nearly the same with a large smoothing step, while those of different tasks are distinct due to differences in graph structure and node attributes. Further, to avoid the catastrophic forgetting of the knowledge learned in previous graph tasks, we propose a novel graph prompting approach for GCIL which learns a small discriminative graph prompt for each task, essentially resulting in a separate classification model for each task. The prompt learning requires the training of a single graph neural network (GNN) only once on the first task, and no data replay is required thereafter, thereby obtaining a GCIL model being both replay-free and forget-free. Extensive experiments on four GCIL benchmarks show that i) our task prototype-based method can achieve 100% task ID prediction accuracy on all four datasets, ii) our GCIL model significantly outperforms state-of-the-art competing methods by at least 18% in average CIL accuracy, and iii) our model is fully free of forgetting on the four datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。