通过动态加权与知识蒸馏，缓解增量学习中的遗忘与干扰问题。
Joint Input and Output Coordination for Class-Incremental Learning
- 根据输出梯度动态分配数据权重，缓解类别偏差。
- 采用知识蒸馏减少新旧任务间的输出干扰。
- 通用灵活，可集成到多种带记忆的增量学习方法中。
增量学习因严重灾难性遗忘而面临挑战。尽管在增量学习过程中存储少量旧任务数据是可行方案，但现有策略仍存在三个问题：1）未能充分解决类别偏差问题；2）难以缓解新旧任务间的相互干扰；3）未考虑任务内部的类别偏差。为此，我们提出一种联合输入与输出协调机制（JIOC），该机制根据输出分数的梯度为不同类别的数据分配不同权重，并利用知识蒸馏（KD）降低新旧任务输出之间的相互干扰。所提机制具有通用性和灵活性，可嵌入使用记忆存储的不同增量学习方法中。大量实验表明，该机制能显著提升现有方法的性能。
原文摘要 · Abstract (English)
Incremental learning is nontrivial due to severe catastrophic forgetting. Although storing a small amount of data on old tasks during incremental learning is a feasible solution, current strategies still do not 1) adequately address the class bias problem, and 2) alleviate the mutual interference between new and old tasks, and 3) consider the problem of class bias within tasks. This motivates us to propose a joint input and output coordination (JIOC) mechanism to address these issues. This mechanism assigns different weights to different categories of data according to the gradient of the output score, and uses knowledge distillation (KD) to reduce the mutual interference between the outputs of old and new tasks. The proposed mechanism is general and flexible, and can be incorporated into different incremental learning approaches that use memory storage. Extensive experiments show that our mechanism can significantly improve their performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。