通过直接操作权重实现增量学习，减少遗忘且无需修改模型结构。
Merge and Bound: Direct Manipulations on Weights for Class Incremental Learning
- 用跨任务与同任务权重平均，融合历史与当前知识
- 新模型与旧模型距离近，有效缓解灾难性遗忘
- 可无缝集成到现有方法中，无需调整架构或目标
我们提出一种名为合并与约束（Merge-and-Bound, M&B）的新型训练方法，用于类增量学习（CIL），直接在参数空间中操纵模型权重以实现优化。该算法包含两种权重合并方式：跨任务权重合并通过平均所有先前阶段模型的权重来统一历史模型；同任务权重合并则通过整合当前阶段内的模型参数，促进当前任务的学习。为确保可靠的权重合并，我们还提出了有界更新技术，旨在以最小累积更新量优化目标模型，并保留前序任务的知识；该策略表明，可在接近旧模型的基础上高效获得新模型，从而减轻灾难性遗忘。M&B可无缝集成至现有CIL方法中，无需修改网络架构或学习目标。我们在标准CIL基准上进行了广泛评估，结果表明其性能优于现有最先进方法。
原文摘要 · Abstract (English)
We present a novel training approach, named Merge-and-Bound (M&B) for Class Incremental Learning (CIL), which directly manipulates model weights in the parameter space for optimization. Our algorithm involves two types of weight merging: inter-task weight merging and intra-task weight merging. Inter-task weight merging unifies previous models by averaging the weights of models from all previous stages. On the other hand, intra-task weight merging facilitates the learning of current task by combining the model parameters within current stage. For reliable weight merging, we also propose a bounded update technique that aims to optimize the target model with minimal cumulative updates and preserve knowledge from previous tasks; this strategy reveals that it is possible to effectively obtain new models near old ones, reducing catastrophic forgetting. M&B is seamlessly integrated into existing CIL methods without modifying architecture components or revising learning objectives. We extensively evaluate our algorithm on standard CIL benchmarks and demonstrate superior performance compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。