利用无标签数据缓解持续学习中的知识遗忘问题
Technical Report for the 5th CLVision Challenge at CVPR: Addressing the Class-Incremental with Repetition using Unlabeled Data -- 4th Place Solution
- 通过知识蒸馏与伪标签技术保留旧类知识
- 在预选阶段准确率达16.68%，最终评估达21.19%
- 适合研究持续学习与无标签数据利用的学者
本文介绍了在CVPR 2024年第五届CLVision挑战赛中针对带重复的类别增量学习（CIR）场景的解决方案。与传统增量学习不同，CIR场景下已学类别可能在后续阶段重复出现，且每个训练阶段仅包含整体类别分布的一部分。训练期间提供的无标签数据可能包含未见类别或无关类别，需加以区分。本方法通过知识蒸馏与伪标签技术，有效利用无标签数据，防止灾难性遗忘，保持对旧类别的识别性能。在预选阶段平均准确率为16.68%，最终评估阶段达21.19%，显著优于基线的9.39%。代码已开源：https://github.com/panagiotamoraiti/continual-learning-challenge-2024。
原文摘要 · Abstract (English)
This paper outlines our approach to the 5th CLVision challenge at CVPR, which addresses the Class-Incremental with Repetition (CIR) scenario. In contrast to traditional class incremental learning, this novel setting introduces unique challenges and research opportunities, particularly through the integration of unlabeled data into the training process. In the CIR scenario, encountered classes may reappear in later learning experiences, and each experience may involve only a subset of the overall class distribution. Additionally, the unlabeled data provided during training may include instances of unseen classes, or irrelevant classes which should be ignored. Our approach focuses on retaining previously learned knowledge by utilizing knowledge distillation and pseudo-labeling techniques. The key characteristic of our method is the exploitation of unlabeled data during training, in order to maintain optimal performance on instances of previously encountered categories and reduce the detrimental effects of catastrophic forgetting. Our method achieves an average accuracy of 16.68\% during the pre-selection phase and 21.19% during the final evaluation phase, outperforming the baseline accuracy of 9.39%. We provide the implementation code at https://github.com/panagiotamoraiti/continual-learning-challenge-2024 .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。