arXiv:2509.12727cs.LGcs.AI2025-09

提出在线无偏曲率近似,提升图持续学习稳定性与适应性。

Unbiased Online Curvature Approximation for Regularized Graph Continual Learning

  • 基于费雪信息矩阵构建通用正则化框架,改进传统EWC方法。
  • 在三个图数据集上显著优于现有方法,平衡旧知识保留与新知识学习。
  • 无需存储费雪信息矩阵,实时估计正则化项,适合资源受限场景。

图持续学习(GCL)旨在从连续的图任务序列中学习。正则化方法对防止灾难性遗忘至关重要,尤其在无重放、类增量设置下,每个任务包含独特类别。本文首先基于费雪信息矩阵(FIM)诱导的弯曲参数空间,建立通用的正则化框架,证明主流弹性权重巩固(EWC)及其变体是该框架的特例,依赖于前序任务参数的对角经验FIM近似。为克服其局限,提出一种新的无偏在线全FIM曲率近似方法,直接基于当前学习状态在线估计正则化项,无需显式计算和存储FIM。该方法能更准确捕捉学习过程中的损失曲面,同时保留旧知识。在三个图数据集上的大量实验表明,本方法显著优于现有基于正则化的基线,实现了稳定性和可塑性的更优权衡。

原文摘要 · Abstract (English)

Graph continual learning (GCL) aims to learn from a continuous sequence of graph-based tasks. Regularization methods are vital for preventing catastrophic forgetting in GCL, particularly in the challenging replay-free, class-incremental setting, where each task consists of a set of unique classes. In this work, we first establish a general regularization framework for GCL based on the curved parameter space induced by the Fisher information matrix (FIM). We show that the dominant Elastic Weight Consolidation (EWC) and its variants are a special case within this framework, using a diagonal approximation of the empirical FIM based on parameters from previous tasks. To overcome their limitations, we propose a new unbiased online curvature approximation of the full FIM based on the model's current learning state. Our method directly estimates the regularization term in an online manner without explicitly evaluating and storing the FIM itself. This enables the model to better capture the loss landscape during learning new tasks while retaining the knowledge learned from previous tasks. Extensive experiments on three graph datasets demonstrate that our method significantly outperforms existing regularization-based methods, achieving a superior trade-off between stability (retaining old knowledge) and plasticity (acquiring new knowledge).

图神经网络持续学习正则化曲率估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。