提出MEGA方法,用二阶梯度对齐缓解图学习中的灾难性遗忘。
MEGA: Second-Order Gradient Alignment for Catastrophic Forgetting Mitigation in GFSCIL
- 通过元训练计算增量二阶梯度,学习高质量先验
- 在四个图数据集上达到当前最优性能
- 适用于各类图持续学习方法,提升泛化能力
图少样本类增量学习(GFSCIL)使模型在初始大基数据集训练后,能从少量新任务样本中持续学习。现有基于原型网络(PNs)的方法在增量阶段仅微调查询集,简化了学习过程,且因架构限制难以融合图持续学习(GCL)技术。为此,我们提出一个更严谨的GFSCIL设置:在增量训练阶段排除查询集。在此基础上,提出模型无关的元图持续学习框架MEGA,旨在有效缓解灾难性遗忘。具体而言,在元训练阶段计算增量二阶梯度,使模型学习到高质量先验,从而对齐元训练与增量学习阶段的行为。在四个主流图数据集上的大量实验表明,MEGA取得当前最优结果,并显著增强多种GCL方法在GFSCIL中的效果。我们认为,MEGA为GFSCIL提供了一种模型无关的新范式,推动未来研究发展。
原文摘要 · Abstract (English)
Graph Few-Shot Class-Incremental Learning (GFSCIL) enables models to continually learn from limited samples of novel tasks after initial training on a large base dataset. Existing GFSCIL approaches typically utilize Prototypical Networks (PNs) for metric-based class representations and fine-tune the model during the incremental learning stage. However, these PN-based methods oversimplify learning via novel query set fine-tuning and fail to integrate Graph Continual Learning (GCL) techniques due to architectural constraints. To address these challenges, we propose a more rigorous and practical setting for GFSCIL that excludes query sets during the incremental training phase. Building on this foundation, we introduce Model-Agnostic Meta Graph Continual Learning (MEGA), aimed at effectively alleviating catastrophic forgetting for GFSCIL. Specifically, by calculating the incremental second-order gradient during the meta-training stage, we endow the model to learn high-quality priors that enhance incremental learning by aligning its behaviors across both the meta-training and incremental learning stages. Extensive experiments on four mainstream graph datasets demonstrate that MEGA achieves state-of-the-art results and enhances the effectiveness of various GCL methods in GFSCIL. We believe that our proposed MEGA serves as a model-agnostic GFSCIL paradigm, paving the way for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。