GradMix通过梯度选择性混合样本,缓解增量学习中的灾难性遗忘。
GradMix: Gradient-based Selective Mixup for Robust Data Augmentation in Class-Incremental Learning
- 基于类别梯度筛选有益样本对进行混合,避免有害混合
- 在多个数据集上显著降低旧知识遗忘率,提升准确率
- 适合需要长期稳定学习的增量学习场景
在持续学习中,如何在获取新知识的同时保持旧知识是一个重大挑战。现有方法常采用经验回放技术,存储少量旧任务数据用于训练。在经验回放框架下,数据增强成为提升模型性能的有效策略,通过将有限的旧任务数据与充足的当前任务数据混合来实现。然而,我们从理论和实证角度分析发现,随机样本对的混合可能损害旧知识,加剧灾难性遗忘。为此,我们提出GradMix,一种专为缓解类别增量学习中灾难性遗忘设计的鲁棒数据增强方法。GradMix基于梯度的类别准则,仅混合有助于学习的样本对,避免不利组合,从而减少遗忘。在多个真实数据集上的实验表明,GradMix在保持旧知识的前提下,显著优于基线数据增强方法的准确率。
原文摘要 · Abstract (English)
In the context of continual learning, acquiring new knowledge while maintaining previous knowledge presents a significant challenge. Existing methods often use experience replay techniques that store a small portion of previous task data for training. In experience replay approaches, data augmentation has emerged as a promising strategy to further improve the model performance by mixing limited previous task data with sufficient current task data. However, we theoretically and empirically analyze that training with mixed samples from random sample pairs may harm the knowledge of previous tasks and cause greater catastrophic forgetting. We then propose GradMix, a robust data augmentation method specifically designed for mitigating catastrophic forgetting in class-incremental learning. GradMix performs gradient-based selective mixup using a class-based criterion that mixes only samples from helpful class pairs and not from detrimental class pairs for reducing catastrophic forgetting. Our experiments on various real datasets show that GradMix outperforms data augmentation baselines in accuracy by minimizing the forgetting of previous knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。