解决跨图领域增量学习中的遗忘问题,提升模型持续学习能力。
GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation
- 通过解耦域内与域间知识,实现参数高效微调。
- 在多个图领域上提升性能,较次优方法最高增益16.6%。
- 适用于多种图基础模型,适合需要持续学习的场景。
图增量学习(GIL)近年来受到广泛关注,但现有方法主要聚焦于单一领域的任务或类别增量学习。随着图基础模型(GFMs)的发展,跨图领域增量学习(Domain-IL)变得日益重要,却尚未被充分研究。本文提出GraphKeeper,从嵌入偏移和决策边界偏离的角度缓解Domain-IL中的灾难性遗忘。首先,采用域特定的参数高效微调,并引入域内与域间解耦目标,防止嵌入混淆。其次,设计无偏差知识保持机制,稳定决策边界以持续适应新领域。针对不可见域的图数据,进一步引入域感知分布判别以获得精确嵌入。大量实验表明,GraphKeeper在多个基准上达到领先性能,相较次优方法提升6.5%~16.6%,且遗忘极小。此外,该方法可无缝集成至多种代表性GFMs,展现出广泛的应用潜力。
原文摘要 · Abstract (English)
Graph incremental learning (GIL), which continuously updates graph models by sequential knowledge acquisition, has garnered significant interest recently. However, existing GIL approaches focus on task-incremental and class-incremental scenarios within a single domain. Graph domain-incremental learning (Domain-IL), aiming at updating models across multiple graph domains, has become critical with the development of graph foundation models (GFMs), but remains unexplored in the literature. In this paper, we propose Graph Domain-Incremental Learning via Knowledge Dientanglement and Preservation (GraphKeeper), to address catastrophic forgetting in Domain-IL scenario from the perspectives of embedding shifts and decision boundary deviations. Specifically, to prevent embedding shifts and confusion across incremental graph domains, we first propose the domain-specific parameter-efficient fine-tuning together with intra- and inter-domain disentanglement objectives. Consequently, to maintain a stable decision boundary, we introduce deviation-free knowledge preservation to continuously fit incremental domains. Additionally, for graphs with unobservable domains, we perform domain-aware distribution discrimination to obtain precise embeddings. Extensive experiments demonstrate the proposed GraphKeeper achieves state-of-the-art results with 6.5%~16.6% improvement over the runner-up with negligible forgetting. Moreover, we show GraphKeeper can be seamlessly integrated with various representative GFMs, highlighting its broad applicative potential.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。