提出新方法缓解持续学习中的遗忘问题
EXACFS -- A CIL Method to mitigate Catastrophic Forgetting
- 基于特征重要性动态评估与渐进老化机制
- 在CIFAR-100和ImageNet-100上实现稳定与可塑性平衡
- 适合关注持续学习与模型记忆保持的研究者
深度神经网络在静态数据集上表现优异，但在持续学习场景下面临灾难性遗忘问题。本文提出指数平均类级特征重要性（EXACFS）方法，通过损失梯度估计每个已学类别的模型特征重要性，对重要性进行渐进衰减，并利用知识蒸馏保留关键特征。该方法在类增量学习（CIL）设置下有效平衡了旧知识记忆（稳定性）与新知识学习（可塑性）。在CIFAR-100和ImageNet-100上的大量实验表明，EXACFS在保持高稳定性的同时显著提升了学习能力。
原文摘要 · Abstract (English)
Deep neural networks (DNNS) excel at learning from static datasets but struggle with continual learning, where data arrives sequentially. Catastrophic forgetting, the phenomenon of forgetting previously learned knowledge, is a primary challenge. This paper introduces EXponentially Averaged Class-wise Feature Significance (EXACFS) to mitigate this issue in the class incremental learning (CIL) setting. By estimating the significance of model features for each learned class using loss gradients, gradually aging the significance through the incremental tasks and preserving the significant features through a distillation loss, EXACFS effectively balances remembering old knowledge (stability) and learning new knowledge (plasticity). Extensive experiments on CIFAR-100 and ImageNet-100 demonstrate EXACFS's superior performance in preserving stability while acquiring plasticity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。