通过动态稀疏化实现类别增量学习，避免遗忘且提升模型适应性。
Semantic-Guided Dynamic Sparsification for Pre-Trained Model-based Class-Incremental Learning
- 基于语义引导的动态稀疏化，调控激活空间子空间方向与秩。
- 相似类别共享紧凑子空间，不相关类别互不重叠，减少干扰。
- 无需约束参数，适用于预训练模型的持续学习，适合工业场景。
类别增量学习（CIL）要求模型在不遗忘旧类的前提下持续学习新类。现有方法通常冻结预训练模型并使用轻量适配器，强制适配器参数正交以防止任务间干扰。但本文认为这种参数约束损害了模型可塑性。为此提出语义引导的动态稀疏化（SGDS），通过有目的地稀疏化主动调控激活空间的子空间方向与秩。具体而言，SGDS通过促进相似类别共享紧凑激活子空间来增强知识迁移，同时为不相似类别分配互不重叠的激活子空间以抑制干扰。通过在激活空间中构建类别特异性稀疏子空间，SGDS有效缓解了干扰，且无需对参数空间施加刚性约束。在多个基准数据集上的大量实验表明，该方法达到当前最优性能。
原文摘要 · Abstract (English)
Class-Incremental Learning (CIL) requires a model to continually learn new classes without forgetting old ones. A common and efficient solution freezes a pre-trained model and employs lightweight adapters, whose parameters are often forced to be orthogonal to prevent inter-task interference. However, we argue that this parameter-constraining method is detrimental to plasticity. To this end, we propose Semantic-Guided Dynamic Sparsification (SGDS), a novel method that proactively guides the activation space by governing the orientation and rank of its subspaces through targeted sparsification. Specifically, SGDS promotes knowledge transfer by encouraging similar classes to share a compact activation subspace, while simultaneously preventing interference by assigning non-overlapping activation subspaces to dissimilar classes. By sculpting class-specific sparse subspaces in the activation space, SGDS effectively mitigates interference without imposing rigid constraints on the parameter space. Extensive experiments on various benchmark datasets demonstrate the state-of-the-art performance of SGDS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。