动态扩展多骨干模型,解决持续学习中知识遗忘问题。
Self-Controlled Dynamic Expansion Model for Continual Learning
- 用多个预训练ViT骨干动态生成新专家,适应新任务
- 在CIFAR-100和ImageNet-1K上达到最好性能,优于基线10%以上
- 适合需要长期学习、避免遗忘的视觉系统研发人员
持续学习(CL)要求模型在不访问旧数据的情况下学习新任务。现有方法多采用单一静态骨干网络,难以适应不同数据域,且存在大量参数闲置。本文提出自控式动态扩展模型(SCDEM),通过多个可训练的预训练ViT骨干协同工作,动态生成仅需少量参数的新专家以应对新任务。引入协同优化机制(COM),利用历史专家的预测信号联合优化多个骨干,实现新任务学习而不遗忘旧知识。提出特征分布一致性(FDC)方法,基于最优传输距离对齐新旧表示的语义相似性,有效缓解负迁移。同时设计动态逐层特征注意力机制(DLWFAM),自动调节各层正则化强度,缓解过强正则化问题。大量实验表明,该方法在CIFAR-100与ImageNet-1K上均达到当前最佳性能。
原文摘要 · Abstract (English)
Continual Learning (CL) epitomizes an advanced training paradigm wherein prior data samples remain inaccessible during the acquisition of new tasks. Numerous investigations have delved into leveraging a pre-trained Vision Transformer (ViT) to enhance model efficacy in continual learning. Nonetheless, these approaches typically utilize a singular, static backbone, which inadequately adapts to novel tasks, particularly when engaging with diverse data domains, due to a substantial number of inactive parameters. This paper addresses this limitation by introducing an innovative Self-Controlled Dynamic Expansion Model (SCDEM), which orchestrates multiple distinct trainable pre-trained ViT backbones to furnish diverse and semantically enriched representations. Specifically, by employing the multi-backbone architecture as a shared module, the proposed SCDEM dynamically generates a new expert with minimal parameters to accommodate a new task. A novel Collaborative Optimization Mechanism (COM) is introduced to synergistically optimize multiple backbones by harnessing prediction signals from historical experts, thereby facilitating new task learning without erasing previously acquired knowledge. Additionally, a novel Feature Distribution Consistency (FDC) approach is proposed to align semantic similarity between previously and currently learned representations through an optimal transport distance-based mechanism, effectively mitigating negative knowledge transfer effects. Furthermore, to alleviate over-regularization challenges, this paper presents a novel Dynamic Layer-Wise Feature Attention Mechanism (DLWFAM) to autonomously determine the penalization intensity on each trainable representation layer. An extensive series of experiments have been conducted to evaluate the proposed methodology's efficacy, with empirical results corroborating that the approach attains state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。