突破持续学习准确率瓶颈，实现与联合训练同等效果
Achieving Upper Bound Accuracy of Joint Training in Continual Learning
- 基于大模型理论，解决任务间类别分离问题
- 在文本与图像数据集上实现联合训练级准确率
- 为真实场景应用铺平道路，适合关注实用性的研究者
持续学习是机器学习中的活跃研究领域，旨在增量式地学习一系列任务。主要挑战是灾难性遗忘（CF），多数研究致力于缓解此问题。然而，当前先进持续学习算法的准确率仍远低于联合训练的理想上限，这一差距阻碍了其在实际应用中的采纳，因准确率至关重要。近期发现另一挑战——任务间类别分离（ICS），推动了对持续学习原则性解决方案的理论研究。进一步研究表明，借助大基础模型的理论与能力，现已可实现上限准确率，并在文本与图像分类数据集上得到实证验证。持续学习现具备实际应用条件。本文综述了达成该成果的主要研究，从直观和神经科学角度论证方法合理性，并分享关键洞见。
原文摘要 · Abstract (English)
Continual learning has been an active research area in machine learning, focusing on incrementally learning a sequence of tasks. A key challenge is catastrophic forgetting (CF), and most research efforts have been directed toward mitigating this issue. However, a significant gap remains between the accuracy achieved by state-of-the-art continual learning algorithms and the ideal or upper-bound accuracy achieved by training all tasks together jointly. This gap has hindered or even prevented the adoption of continual learning in applications, as accuracy is often of paramount importance. Recently, another challenge, termed inter-task class separation (ICS), was also identified, which spurred a theoretical study into principled approaches for solving continual learning. Further research has shown that by leveraging the theory and the power of large foundation models, it is now possible to achieve upper-bound accuracy, which has been empirically validated using both text and image classification datasets. Continual learning is now ready for real-life applications. This paper surveys the main research leading to this achievement, justifies the approach both intuitively and from neuroscience research, and discusses insights gained.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。