通过全局固定、局部调整,解决持续学习中的遗忘与混淆问题。
Global Pre-fixing, Local Adjusting: A Simple yet Effective Contrastive Strategy for Continual Learning
- 将特征空间划分为不重叠区域,用等角紧框架固定任务间结构。
- 在每个任务内动态调整特征分布,提升类内紧凑性和类间分离性。
- 可无缝嵌入现有对比学习框架,适合追求稳定性能的模型开发者。
持续学习(CL)旨在从不断变化的任务中持续获取并积累知识,同时缓解灾难性遗忘。近期,利用对比损失构建更可迁移、更少遗忘的表征成为有前景的方向。然而,由于任务间和任务内特征的混淆,其性能仍受限。为此,我们提出一种简单而有效的对比策略——全局预固定、局部调整(GPLASC)。具体而言,为避免任务级混淆,我们将整个表征超球面划分为非重叠区域,区域中心构成任务间预固定的等角紧框架(ETF)。同时,针对单个任务,方法可调节特征结构,在各自分配区域内形成任务内可调的ETF。结果表明,该方法能同时保证任务间与任务内的判别性特征结构,并可无缝集成到任意现有对比持续学习框架中。大量实验验证了其有效性。
原文摘要 · Abstract (English)
Continual learning (CL) involves acquiring and accumulating knowledge from evolving tasks while alleviating catastrophic forgetting. Recently, leveraging contrastive loss to construct more transferable and less forgetful representations has been a promising direction in CL. Despite advancements, their performance is still limited due to confusion arising from both inter-task and intra-task features. To address the problem, we propose a simple yet effective contrastive strategy named \textbf{G}lobal \textbf{P}re-fixing, \textbf{L}ocal \textbf{A}djusting for \textbf{S}upervised \textbf{C}ontrastive learning (GPLASC). Specifically, to avoid task-level confusion, we divide the entire unit hypersphere of representations into non-overlapping regions, with the centers of the regions forming an inter-task pre-fixed \textbf{E}quiangular \textbf{T}ight \textbf{F}rame (ETF). Meanwhile, for individual tasks, our method helps regulate the feature structure and form intra-task adjustable ETFs within their respective allocated regions. As a result, our method \textit{simultaneously} ensures discriminative feature structures both between tasks and within tasks and can be seamlessly integrated into any existing contrastive continual learning framework. Extensive experiments validate its effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。