提出主动遗忘机制,解决数据偏见导致的持续学习失效问题
ErrorEraser: Unlearning Data Bias for Improved Continual Learning
- 通过识别特征空间中的偏差样本,主动清除错误记忆
- 在三种持续学习方法上均提升准确率并降低遗忘率
- 适合关注真实数据偏见与模型鲁棒性的研究者
持续学习(CL)旨在防止灾难性遗忘并促进知识迁移。然而,现有方法忽视了真实数据中的偏见,导致模型学习到虚假相关性并在任务间传播放大。我们发现,数据偏见同时削弱了知识保留与迁移能力。为此,提出ErrorEraser——一种通用插件式框架,用于消除由偏见引发的错误记忆。该框架包含两个模块:误差识别模块通过无先验地学习特征空间中任务数据的概率密度分布,精准定位潜在偏差样本;误差擦除模块通过调整代表性异常样本的决策空间,确保仅删除错误知识。此外,设计增量式特征分布学习策略,降低下游任务中误差识别的资源开销。大量实验表明,ErrorEraser显著缓解数据偏见的负面影响,在三种典型持续学习方法上均实现更高准确率与更低遗忘率。代码已开源。
原文摘要 · Abstract (English)
Continual Learning (CL) primarily aims to retain knowledge to prevent catastrophic forgetting and transfer knowledge to facilitate learning new tasks. Unlike traditional methods, we propose a novel perspective: CL not only needs to prevent forgetting, but also requires intentional forgetting.This arises from existing CL methods ignoring biases in real-world data, leading the model to learn spurious correlations that transfer and amplify across tasks. From feature extraction and prediction results, we find that data biases simultaneously reduce CL's ability to retain and transfer knowledge. To address this, we propose ErrorEraser, a universal plugin that removes erroneous memories caused by biases in CL, enhancing performance in both new and old tasks. ErrorEraser consists of two modules: Error Identification and Error Erasure. The former learns the probability density distribution of task data in the feature space without prior knowledge, enabling accurate identification of potentially biased samples. The latter ensures only erroneous knowledge is erased by shifting the decision space of representative outlier samples. Additionally, an incremental feature distribution learning strategy is designed to reduce the resource overhead during error identification in downstream tasks. Extensive experimental results show that ErrorEraser significantly mitigates the negative impact of data biases, achieving higher accuracy and lower forgetting rates across three types of CL methods. The code is available at https://github.com/diadai/ErrorEraser.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。