先微调预训练模型再持续学习，缓解遗忘又提升适应性。
Adapt before Continual Learning
- 在每个新任务前，用原型对齐方式微调预训练模型骨干
- 在多个基准上显著提升持续学习性能，减少灾难性遗忘
- 适合需要稳定与灵活兼备的工业级持续学习场景
持续学习（CL）旨在让神经网络在不断获取新知识（可塑性）的同时保留已有知识（稳定性）。尽管预训练模型（PTM）为CL提供了坚实基础，但现有方法在平衡这两项目标时面临根本挑战。当前方法通常冻结PTM主干以保证稳定性，严重限制了模型的可塑性，尤其当新数据分布与预训练数据差异较大时。相反，逐次微调整个PTM虽能适应新知识，却常导致灾难性遗忘，凸显了基于PTM的持续学习中稳定性与可塑性的核心权衡问题。为此，我们提出在核心持续学习流程前引入预适应阶段（ACL），一种新型框架。该阶段通过将嵌入向量对齐其原始类别原型、远离无关类别，精细化调整PTM主干。该机制在理论上和实证上均表现出理想的稳定性与可塑性平衡，在多个基准和集成方法中显著提升持续学习表现。代码已公开于https://github.com/byyx666/ACL_code。
原文摘要 · Abstract (English)
Continual Learning (CL) seeks to enable neural networks to incrementally acquire new knowledge (plasticity) while retaining existing knowledge (stability). Although pre-trained models (PTMs) have provided a strong foundation for CL, existing approaches face a fundamental challenge in balancing these two competing objectives. Current methods typically address stability by freezing the PTM backbone, which severely limits the model's plasticity, particularly when incoming data distribution diverges largely from the pre-training data. Alternatively, sequentially fine-tuning the entire PTM can adapt to new knowledge but often leads to catastrophic forgetting, highlighting the critical stability-plasticity trade-off in PTM-based CL. To address this limitation, we propose Adapting PTMs before the core CL} process (ACL), a novel framework that introduces a plug-and-play adaptation phase prior to learning each new task. During this phase, ACL refines the PTM backbone by aligning embeddings with their original class prototypes while distancing them from irrelevant classes. This mechanism theoretically and empirically demonstrates desirable balance between stability and plasticity, significantly improving CL performance across benchmarks and integrated methods. Code is available at https://github.com/byyx666/ACL_code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。