arXiv:2512.16871cs.LGcs.AI2025-12

通过优化任务顺序缓解模型遗忘,提升持续学习性能。

Sequencing to Mitigate Catastrophic Forgetting in Continual Learning

  • 提出基于零样本评分的任务排序方法,动态确定最优学习顺序。
  • 实验表明合理排序可显著降低灾难性遗忘,提升模型稳定性。
  • 适用于持续学习与课程学习,对实际应用有广泛价值。

为应对现实世界中的动态变化,智能系统需具备在生命周期内持续获取、更新和利用知识的能力,这被称为持续学习。灾难性遗忘是持续学习面临的主要挑战,即学习新任务时会大幅降低对旧任务的性能。现有方法主要分为五类:基于重放、正则化、优化、表示和架构的方法。本文从新角度出发,关注任务呈现顺序对遗忘的影响,研究任务排序在缓解灾难性遗忘中的作用,并提出一种确定最优任务顺序的方法。该方法借鉴神经网络架构搜索(NAS)中的零样本评分算法。实验结果表明,智能任务排序能显著减少灾难性遗忘;当与传统持续学习策略结合时,进一步提升了性能和抗遗忘能力。此外,该方法还可应用于课程学习等领域。

原文摘要 · Abstract (English)

To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, and exploit knowledge throughout its lifetime. This ability, known as Continual learning, provides a foundation for AI systems to develop themselves adaptively. Catastrophic forgetting is a major challenge to the progress of Continual Learning approaches, where learning a new task usually results in a dramatic performance drop on previously learned ones. Many approaches have emerged to counteract the impact of CF. Most of the proposed approaches can be categorized into five classes: replay-based, regularization-based, optimization-based, representation-based, and architecture-based. In this work, we approach the problem from a different angle, specifically by considering the optimal sequencing of tasks as they are presented to the model. We investigate the role of task sequencing in mitigating CF and propose a method for determining the optimal task order. The proposed method leverages zero-shot scoring algorithms inspired by neural architecture search (NAS). Results demonstrate that intelligent task sequencing can substantially reduce CF. Moreover, when combined with traditional continual learning strategies, sequencing offers enhanced performance and robustness against forgetting. Additionally, the presented approaches can find applications in other fields, such as curriculum learning.

持续学习任务排序遗忘缓解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。