arXiv:2605.14785cs.LGcs.CV2026-05

揭示了重放式增量学习中类别遗忘不均的根源,提出可预测遗忘程度的新指标。

Understanding Imbalanced Forgetting in Rehearsal-Based Class-Incremental Learning

  • 从梯度层面构建三类系数,分析旧类别在增量训练中的干扰来源。
  • 其中自诱导干扰系数预测力最强,解释了为何某些类别遗忘更严重。
  • 研究结果为缓解遗忘不均提供了可操作的改进方向,适合关注模型稳定性者阅读。

神经网络在类别增量学习(CIL)中面临灾难性遗忘问题。重放策略(rehearsal)通过回放部分历史样本以缓解该问题,但近期研究发现,即使重放分配均衡,某些类别仍被显著遗忘。本文系统性地揭示了这种遗忘不均现象的存在,并深入分析其成因。基于理论分析,我们构建了三个位于最后一层的系数,分别捕捉不同梯度层面的干扰源,用于衡量每个旧类别在增量步骤中的影响。实验表明,这三类系数共同能可靠预测各旧类别在步骤结束时的遗忘排名。尽管不能直接证明因果关系,但结果支持这些系数作为梯度交互与类别遗忘结果之间的机制性解释。尤其值得注意的是,反映自诱导干扰的系数成为最强预测因子,控制实验进一步显示其受新类别干扰系数的影响。研究为缓解类别间遗忘差异提供了重要洞见,建议通过降低识别出的干扰源差异来改善模型表现。

原文摘要 · Abstract (English)

Neural networks suffer from catastrophic forgetting in class-incremental learning (CIL) settings. Rehearsal$\unicode{x2013}$replaying a subset of past samples$\unicode{x2013}$is a well-established mitigation strategy. However, recent results suggest that, despite balanced rehearsal allocation, some classes are forgotten substantially more than others. Despite its relevance, this imbalanced forgetting phenomenon remains underexplored. This work shows that imbalanced forgetting arises systematically and severely in rehearsal-based CIL and investigates it extensively. Specifically, we construct, from a principled analysis, three last-layer coefficients that capture different gradient-level sources of interference affecting each past class during an incremental step. We then demonstrate that, together, they reliably predict how past classes will rank in terms of forgetting at the end of that step. While predictive performance alone does not establish causality, these results support the interpretation of the coefficients as a plausible mechanistic account linking last-layer gradient-level interactions during training to class-level forgetting outcomes. Notably, one coefficient$\unicode{x2013}$capturing self-induced interference$\unicode{x2013}$emerges as the strongest predictor, with controlled experiments providing evidence consistent with this coefficient being influenced by the new-class interference coefficient. Overall, our findings provide valuable insights and suggest promising directions for mitigating imbalanced forgetting by reducing class-wise disparities in the identified sources of interference.

增量学习遗忘机制重放策略分类偏差

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。