arXiv:2507.12305cs.LGcs.AI2025-07ICCV被引 2

无需记忆回放的在线持续学习方法,用轻量提示机制防止遗忘。

PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning

  • 用轻量提示生成器+可训练缩放位移模块,分离通用与特定知识。
  • 在CIFAR100等4个数据集上性能超越当前最优方法。
  • 参数少、推理快,适合隐私敏感的实时数据场景。

在线持续学习(OCL)中,数据只能访问一次,加剧了灾难性遗忘问题。现有最优方法常依赖存储旧类样本或特征进行回放,但受限于数据隐私政策;而基于提示的方法虽表现优异,却因可训练参数不断增长导致吞吐量下降。本文提出PROL方法,包含四个核心组件:(1) 单一轻量提示生成器作为通用知识,(2) 可训练缩放与偏移模块作为特定知识,(3) 保持预训练模型泛化能力,(4) 硬-软更新机制。实验表明,PROL在CIFAR100、ImageNet-R、ImageNet-A和CUB数据集上显著优于当前SOTA。复杂度分析显示,该方法参数量小,训练与推理时间适中,吞吐率良好。代码已开源:https://github.com/anwarmaxsum/PROL。

原文摘要 · Abstract (English)

The data privacy constraint in online continual learning (OCL), where the data can be seen only once, complicates the catastrophic forgetting problem in streaming data. A common approach applied by the current SOTAs in OCL is with the use of memory saving exemplars or features from previous classes to be replayed in the current task. On the other hand, the prompt-based approach performs excellently in continual learning but with the cost of a growing number of trainable parameters. The first approach may not be applicable in practice due to data openness policy, while the second approach has the issue of throughput associated with the streaming data. In this study, we propose a novel prompt-based method for online continual learning that includes 4 main components: (1) single light-weight prompt generator as a general knowledge, (2) trainable scaler-and-shifter as specific knowledge, (3) pre-trained model (PTM) generalization preserving, and (4) hard-soft updates mechanism. Our proposed method achieves significantly higher performance than the current SOTAs in CIFAR100, ImageNet-R, ImageNet-A, and CUB dataset. Our complexity analysis shows that our method requires a relatively smaller number of parameters and achieves moderate training time, inference time, and throughput. For further study, the source code of our method is available at https://github.com/anwarmaxsum/PROL.

持续学习提示学习在线学习轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。