提出双缓冲+分治策略,缓解在线持续学习中的遗忘问题
Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning
- 用短期记忆和分组原型长期记忆存储关键样本
- 在多个数据集上超越现有方法,显著降低遗忘率
- 适合需要高效持续学习的工业场景使用
在线持续学习(OCL)面临数据顺序到达和灾难性遗忘的挑战。本文提出一种新框架——在线动态可扩展双内存(ODEDM),包含快速更新的短期记忆和基于聚类原型分组的长期记忆,以保存多样化且类别特定的样本。采用基于K均值的原型识别方法和基于最优传输机制的关键样本保留策略,优先保留与原型高度相似的样本,从而保持语义丰富信息。同时提出分治(DAC)优化策略,将内存更新分解为子问题,降低计算开销。ODEDM可作为即插即用模块集成至现有重放类方法中。在标准与不平衡的OCL设置下,实验表明其在多个数据集上均达到领先性能,显著优于DER系列及近期方法如VR-MCL和POCL。
原文摘要 · Abstract (English)
Online Continual Learning (OCL) involves sequentially arriving data and is particularly challenged by catastrophic forgetting, which significantly impairs model performance. To address this issue, we introduce a novel framework, Online Dynamic Expandable Dual Memory (ODEDM), that integrates a short-term memory for fast memory and a long-term memory structured into sub-buffers anchored by cluster prototypes, enabling the storage of diverse and category-specific samples to mitigate forgetting. We propose a novel K-means-based strategy for prototype identification and an optimal transport-based mechanism to retain critical samples, prioritising those exhibiting high similarity to their corresponding prototypes. This design preserves semantically rich information. Additionally, we propose a Divide-and-Conquer (DAC) optimisation strategy that decomposes memory updates into subproblems, thereby reducing computational overhead. ODEDM functions as a plug-and-play module that can be seamlessly integrated with existing rehearsal-based approaches. Experimental results under both standard and imbalanced OCL settings show that ODEDM consistently achieves state-of-the-art performance across multiple datasets, delivering substantial improvements over the DER family as well as recent methods such as VR-MCL and POCL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。