arXiv:2511.09871cs.LGcs.AI2025-11

提出双记忆机制,让模型持续学习不遗忘且不混淆。

Expandable and Differentiable Dual Memories with Orthogonal Regularization for Exemplar-free Continual Learning

  • 用两个可扩展的互补记忆分别学通用特征和样本特有特征。
  • 在三个数据集上达到55.13%、37.24%、30.11%的最终准确率。
  • 无需样本存储,适合长期增量学习任务。

持续学习方法通常迫使神经网络孤立处理序列任务,难以利用跨任务有用关系,导致重复学习相似特征或过度区分。为此,我们提出一种全可微、无需示例的可扩展双记忆方法:一个记忆学习所有任务共用的通用特征,另一个将共享特征组合以学习每个样本的独特判别特征。两个记忆均支持可微更新,使网络能自主学习样本潜在表示。每个任务通过记忆调整模块自适应裁剪关键槽位并最小化扩容,以容纳新概念;正交正则化强制保留与新学记忆组件之间的几何分离,防止干扰。在CIFAR-10、CIFAR-100和Tiny-ImageNet上的实验表明,该方法超越14种先进方法,实现55.13%、37.24%、30.11%的最终准确率。额外分析证实,通过有效整合与利用知识,该方法显著提升各序列任务平均性能,并使特征提取结果最接近理论上限,确立了持续学习的新里程碑。

原文摘要 · Abstract (English)

Continual learning methods used to force neural networks to process sequential tasks in isolation, preventing them from leveraging useful inter-task relationships and causing them to repeatedly relearn similar features or overly differentiate them. To address this problem, we propose a fully differentiable, exemplar-free expandable method composed of two complementary memories: One learns common features that can be used across all tasks, and the other combines the shared features to learn discriminative characteristics unique to each sample. Both memories are differentiable so that the network can autonomously learn latent representations for each sample. For each task, the memory adjustment module adaptively prunes critical slots and minimally expands capacity to accommodate new concepts, and orthogonal regularization enforces geometric separation between preserved and newly learned memory components to prevent interference. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that the proposed method outperforms 14 state-of-the-art methods for class-incremental learning, achieving final accuracies of 55.13\%, 37.24\%, and 30.11\%, respectively. Additional analysis confirms that, through effective integration and utilization of knowledge, the proposed method can increase average performance across sequential tasks, and it produces feature extraction results closest to the upper bound, thus establishing a new milestone in continual learning.

持续学习双记忆可扩展正交正则

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。