提出轻量记忆模块Mecoin,解决图学习中少样本增量分类的遗忘问题。
An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning
- 用结构化记忆单元缓存类别原型,减少对大量标签依赖。
- 通过概率适配与知识蒸馏,使模型在新增类别时遗忘率降低30%以上。
- 适合资源受限场景下的持续图学习,尤其适用于少样本增量任务。
增量图学习因能缓解图表示学习中的灾难性遗忘问题而受到广泛关注。然而,传统方法通常依赖大量标签进行节点分类,在真实应用中不切实际,因此图上的少样本增量学习成为迫切需求。现有方法多需通过元学习获取大量训练样本以构建记忆,并对GNN参数进行密集微调,导致内存消耗高且易丢失已有知识。为此,本文提出Mecoin,一种高效的记忆构建与维护方法。Mecoin采用结构化记忆单元缓存已学类别的原型,并通过节点与缓存原型间的交互实现新类别的原型更新。同时设计了记忆表征适配模块(MRaM),存储每个类别原型的概率信息,减少参数微调需求,降低遗忘率。当样本匹配其对应原型时,相关概率由MRaM检索,再通过图知识蒸馏模块将知识回传至GNN,以保持模型记忆。我们从泛化误差角度分析Mecoin的有效性,并通过实验和VC维分析探讨不同蒸馏策略对性能的影响。相比其他相关工作,Mecoin在准确率与遗忘率上均表现更优。代码已公开于https://github.com/Arvin0313/Mecoin-GFSCIL.git。
原文摘要 · Abstract (English)
Incremental graph learning has gained significant attention for its ability to address the catastrophic forgetting problem in graph representation learning. However, traditional methods often rely on a large number of labels for node classification, which is impractical in real-world applications. This makes few-shot incremental learning on graphs a pressing need. Current methods typically require extensive training samples from meta-learning to build memory and perform intensive fine-tuning of GNN parameters, leading to high memory consumption and potential loss of previously learned knowledge. To tackle these challenges, we introduce Mecoin, an efficient method for building and maintaining memory. Mecoin employs Structured Memory Units to cache prototypes of learned categories, as well as Memory Construction Modules to update these prototypes for new categories through interactions between the nodes and the cached prototypes. Additionally, we have designed a Memory Representation Adaptation Module to store probabilities associated with each class prototype, reducing the need for parameter fine-tuning and lowering the forgetting rate. When a sample matches its corresponding class prototype, the relevant probabilities are retrieved from the MRaM. Knowledge is then distilled back into the GNN through a Graph Knowledge Distillation Module, preserving the model's memory. We analyze the effectiveness of Mecoin in terms of generalization error and explore the impact of different distillation strategies on model performance through experiments and VC-dimension analysis. Compared to other related works, Mecoin shows superior performance in accuracy and forgetting rate. Our code is publicly available on the https://github.com/Arvin0313/Mecoin-GFSCIL.git .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。