提出新方法S2IL，让模型在学新知识时更稳定地记住旧知识。
S2IL: Structurally Stable Incremental Learning
- 通过保持特征空间整体结构，实现新旧知识的灵活与稳定平衡。
- 在CIFAR-100、ImageNet-100和ImageNet-1K上均超越现有方法。
- 特别在大量增量任务场景下表现突出，适合持续学习应用。
特征蒸馏（FD）策略被证明能有效缓解类别增量学习（CIL）中的灾难性遗忘（CF）。然而，现有FD方法强制要求跨增量步骤的特征幅度和方向严格对齐，限制了模型对新知识的适应能力。本文提出结构稳定的增量学习（S2IL），一种针对CIL的FD方法，通过保留特征的整体空间模式，促进兼具灵活性（可塑性）与稳定性（保留旧知识）的表示。实验表明，S2IL在CIFAR-100、ImageNet-100和ImageNet-1K等主流基准数据集上均取得优异的增量准确率，显著优于其他FD方法。尤其在包含大量增量任务的场景中，优势更为明显。
原文摘要 · Abstract (English)
Feature Distillation (FD) strategies are proven to be effective in mitigating Catastrophic Forgetting (CF) seen in Class Incremental Learning (CIL). However, current FD approaches enforce strict alignment of feature magnitudes and directions across incremental steps, limiting the model's ability to adapt to new knowledge. In this paper we propose Structurally Stable Incremental Learning(S22IL), a FD method for CIL that mitigates CF by focusing on preserving the overall spatial patterns of features which promote flexible (plasticity) yet stable representations that preserve old knowledge (stability). We also demonstrate that our proposed method S2IL achieves strong incremental accuracy and outperforms other FD methods on SOTA benchmark datasets CIFAR-100, ImageNet-100 and ImageNet-1K. Notably, S2IL outperforms other methods by a significant margin in scenarios that have a large number of incremental tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。