通过稀疏正则化实现固定容量CNN的持续学习,防止遗忘且减少计算量。
Group and Exclusive Sparse Regularization-based Continual Learning of CNNs
- 引入分组与独占稀疏正则化,平衡网络稳定与可塑性。
- 在多个视觉基准上准确率超越现有方法,有效缓解灾难性遗忘。
- 适合资源受限场景下需长期学习的轻量化模型部署。
我们提出一种基于正则化的持续学习方法(GESCL),用于固定容量卷积神经网络(CNN),在顺序学习多个任务时避免灾难性遗忘。该方法通过稳定性正则项保持过去任务重要滤波器的参数不变,从而确保网络稳定性;同时利用可塑性正则项,借助CNN的过参数化特性高效稀疏化网络,调整不重要滤波器以适应未来任务。相比动态扩展网络或记忆旧数据的方法,GESCL显著减少参数量和计算开销。在主流持续学习视觉基准上的实验表明,GESCL在分类准确率和防止遗忘方面均优于现有最先进方法。
原文摘要 · Abstract (English)
We present a regularization-based approach for continual learning (CL) of fixed capacity convolutional neural networks (CNN) that does not suffer from the problem of catastrophic forgetting when learning multiple tasks sequentially. This method referred to as Group and Exclusive Sparsity based Continual Learning (GESCL) avoids forgetting of previous tasks by ensuring the stability of the CNN via a stability regularization term, which prevents filters detected as important for past tasks to deviate too much when learning a new task. On top of that, GESCL makes the network plastic via a plasticity regularization term that leverage the over-parameterization of CNNs to efficiently sparsify the network and tunes unimportant filters making them relevant for future tasks. Doing so, GESCL deals with significantly less parameters and computation compared to CL approaches that either dynamically expand the network or memorize past tasks' data. Experiments on popular CL vision benchmarks show that GESCL leads to significant improvements over state-of-the-art method in terms of overall CL performance, as measured by classification accuracy as well as in terms of avoiding catastrophic forgetting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。