AnaCP让模型在不遗忘旧知识的前提下持续优化特征表示,逼近最优性能。
AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projection
- 用解析对比投影实现无梯度的特征增量适应
- 在CIFAR-100上达到与联合训练相当的准确率
- 适合追求高精度且需避免遗忘的持续学习场景
本文研究类增量学习(CIL)问题,即模型需按顺序学习不同类别的任务。传统方法因需同时增量学习特征和分类器,易产生灾难性遗忘。近期引入预训练模型(PTM)的方法虽通过固定特征提取与解析分类器实现高效训练,但无法持续优化特征表示,导致性能受限。为此,我们提出AnaCP(解析对比投影),在保持解析分类器效率的同时,实现无需梯度更新的特征增量适应,从而消除梯度更新引发的遗忘。实验表明,AnaCP不仅优于现有基线,更达到了联合训练水平的准确率,被视为CIL的上界性能。
原文摘要 · Abstract (English)
This paper studies the problem of class-incremental learning (CIL), a core setting within continual learning where a model learns a sequence of tasks, each containing a distinct set of classes. Traditional CIL methods, which do not leverage pre-trained models (PTMs), suffer from catastrophic forgetting (CF) due to the need to incrementally learn both feature representations and the classifier. The integration of PTMs into CIL has recently led to efficient approaches that treat the PTM as a fixed feature extractor combined with analytic classifiers, achieving state-of-the-art performance. However, they still face a major limitation: the inability to continually adapt feature representations to best suit the CIL tasks, leading to suboptimal performance. To address this, we propose AnaCP (Analytic Contrastive Projection), a novel method that preserves the efficiency of analytic classifiers while enabling incremental feature adaptation without gradient-based training, thereby eliminating the CF caused by gradient updates. Our experiments show that AnaCP not only outperforms existing baselines but also achieves the accuracy level of joint training, which is regarded as the upper bound of CIL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。