用动态融合方法构建单一全局适配器,解决持续学习遗忘问题
Dynamical Adapter Fusion: Constructing A Global Adapter for Pre-Trained Model-based Class-Incremental Learning
- 基于泛化理论设计动态融合机制,整合任务适配器、旧全局参数和初始化参数
- 在多个基准上达到当前最优性能,准确率提升显著
- 适合需要高效持续学习的工业场景,尤其关注模型轻量化与稳定性
类别增量学习(CIL)要求模型在不遗忘旧类的前提下持续学习新类别。主流方法冻结预训练模型,仅训练轻量级任务专属适配器。但维护多个任务参数阻碍知识迁移且检索成本高,而简单参数融合常引发破坏性干扰和灾难性遗忘。为此,我们提出动态适配器融合(DAF),构建一个单一稳健的全局适配器。基于PAC-Bayes定理,我们推导出融合机制,显式结合三部分:优化后的任务专属适配器参数、先前全局适配器参数和初始化参数。利用损失函数的泰勒展开推导最优融合系数,动态实现稳定与可塑性的最佳平衡。此外,提出鲁棒初始化策略以有效捕捉全局知识模式。在多个CIL基准上的实验表明,DAF实现当前最优(SOTA)性能。
原文摘要 · Abstract (English)
Class-Incremental Learning (CIL) requires models to continuously acquire new classes without forgetting previously learned ones. A dominant paradigm involves freezing a pre-trained model and training lightweight, task-specific adapters. However, maintaining task-specific parameters hinders knowledge transfer and incurs high retrieval costs, while naive parameter fusion often leads to destructive interference and catastrophic forgetting. To address these challenges, we propose Dynamical Adapter Fusion (DAF) to construct a single robust global adapter. Grounded in the PAC-Bayes theorem, we derive a fusion mechanism that explicitly integrates three components: the optimized task-specific adapter parameters, the previous global adapter parameters, and the initialization parameters. We utilize the Taylor expansion of the loss function to derive the optimal fusion coefficients, dynamically achieving the best balance between stability and plasticity. Furthermore, we propose a Robust Initialization strategy to effectively capture global knowledge patterns. Experiments on multiple CIL benchmarks demonstrate that DAF achieves state-of-the-art (SOTA) performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。