构建统一框架,揭示持续学习真实场景下的性能陷阱
LibContinual: A Comprehensive Library towards Realistic Continual Learning
- 设计模块化库,整合19种主流持续学习算法
- 发现三大隐含假设导致性能虚高,真实场景下显著下降
- 提供严格在线设置与统一内存预算,适合追求真实性的研究者
持续学习中的灾难性遗忘问题长期存在。尽管方法层出不穷,但缺乏统一框架导致实现不一致、评估标准混乱,难以公平比较和复现。为此,我们提出LibContinual——一个高内聚、低耦合的综合性开源库,集成5类共19种代表性持续学习算法,提供标准化执行环境。基于该平台,我们系统分析了主流评估中三个隐含假设:(1)离线数据可访问,(2)内存资源无约束,(3)任务内语义同质。结果表明,这些假设常夸大方法在真实场景中的表现。通过严格的在线学习设置、新的统一内存预算协议及类别随机化测试,许多代表性方法性能大幅下降。研究强调需发展资源敏感且语义鲁棒的持续学习策略,并将LibContinual作为未来真实持续学习研究的基础工具。代码已公开于https://github.com/RL-VIG/LibContinual。
原文摘要 · Abstract (English)
A fundamental challenge in Continual Learning (CL) is catastrophic forgetting, where adapting to new tasks degrades the performance on previous ones. While the field has evolved with diverse methods, this rapid surge in diverse methodologies has culminated in a fragmented research landscape. The lack of a unified framework, including inconsistent implementations, conflicting dependencies, and varying evaluation protocols, makes fair comparison and reproducible research increasingly difficult. To address this challenge, we propose LibContinual, a comprehensive and reproducible library designed to serve as a foundational platform for realistic CL. Built upon a high-cohesion, low-coupling modular architecture, LibContinual integrates 19 representative algorithms across five major methodological categories, providing a standardized execution environment. Meanwhile, leveraging this unified framework, we systematically identify and investigate three implicit assumptions prevalent in mainstream evaluation: (1) offline data accessibility, (2) unregulated memory resources, and (3) intra-task semantic homogeneity. We argue that these assumptions often overestimate the real-world applicability of CL methods. Through our comprehensive analysis using strict online CL settings, a novel unified memory budget protocol, and a proposed category-randomized setting, we reveal significant performance drops in many representative CL methods when subjected to these real-world constraints. Our study underscores the necessity of resource-aware and semantically robust CL strategies, and offers LibContinual as a foundational toolkit for future research in realistic continual learning. The source code is available from \href{https://github.com/RL-VIG/LibContinual}{https://github.com/RL-VIG/LibContinual}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。