提出轻量级持续学习方法SPARC，无需记忆回放或完整模型副本。
Continual Learning Beyond Experience Rehearsal and Full Model Surrogates
- 用任务特定与通用语义记忆结合实现知识融合
- 参数仅需全模型代理的6%，性能仍超同类方法
- 适合资源受限场景，兼顾效率与稳定性
持续学习（CL）对深度神经网络仍是重大挑战，因学习新任务会部分或完全遗忘旧知识。现有方法多依赖经验回放或完整模型代理来缓解灾难性遗忘，但带来显著内存与计算开销，限制了实际应用。为此，我们提出SPARC，一种无需经验回放和全模型代理的可扩展持续学习方法。通过有效结合任务特定工作记忆与任务无关语义记忆，实现跨任务知识整合，仅需全模型代理6%的参数即可达到优异性能。在Seq-TinyImageNet上表现突出，并在多个CL基准上匹配回放类方法。此外，分类层权重重归一化缓解了任务偏差问题，使SPARC成为严苛效率约束下的实用、可扩展解决方案。
原文摘要 · Abstract (English)
Continual learning (CL) has remained a significant challenge for deep neural networks as learning new tasks erases previously acquired knowledge, either partially or completely. Existing solutions often rely on experience rehearsal or full model surrogates to mitigate CF. While effective, these approaches introduce substantial memory and computational overhead, limiting their scalability and applicability in real-world scenarios. To address this, we propose SPARC, a scalable CL approach that eliminates the need for experience rehearsal and full-model surrogates. By effectively combining task-specific working memories and task-agnostic semantic memory for cross-task knowledge consolidation, SPARC results in a remarkable parameter efficiency, using only 6% of the parameters required by full-model surrogates. Despite its lightweight design, SPARC achieves superior performance on Seq-TinyImageNet and matches rehearsal-based methods on various CL benchmarks. Additionally, weight re-normalization in the classification layer mitigates task-specific biases, establishing SPARC as a practical and scalable solution for CL under stringent efficiency constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。