提出一种无需存储旧数据的连续学习模型,兼顾隐私与高效。
Continual learning via probabilistic exchangeable sequence modelling
- 基于神经过程构建概率化框架,实现可扩展的贝叶斯更新
- 在图像与生物医学数据集上优于现有方法,避免灾难性遗忘
- 适合对数据隐私或存储受限的应用场景
连续学习(CL)指在持续学习新知识的同时保留过往经验的能力。尽管近年提出了大量CL方法,但其计算开销大且缺乏不确定性量化,难以直接用于真实决策任务。为此,我们提出CL-BRUNO,一种基于神经过程的概率化连续学习模型,支持可扩展且可处理的贝叶斯更新与预测。该方法利用深度生成模型构建统一的概率框架,能同时处理任务增量与类别增量学习,使用户可通过单一模型整合不同场景的信息。通过分布与函数正则化,该方法无需保留历史样本即可防止灾难性遗忘,适用于数据隐私或存储受限的场景。实验表明,CL-BRUNO在自然图像与生物医学数据集上均优于现有方法,验证了其在真实应用中的有效性。
原文摘要 · Abstract (English)
Continual learning (CL) refers to the ability to continuously learn and accumulate new knowledge while retaining useful information from past experiences. Although numerous CL methods have been proposed in recent years, it is not straightforward to deploy them directly to real-world decision-making problems due to their computational cost and lack of uncertainty quantification. To address these issues, we propose CL-BRUNO, a probabilistic, Neural Process-based CL model that performs scalable and tractable Bayesian update and prediction. Our proposed approach uses deep-generative models to create a unified probabilistic framework capable of handling different types of CL problems such as task- and class-incremental learning, allowing users to integrate information across different CL scenarios using a single model. Our approach is able to prevent catastrophic forgetting through distributional and functional regularisation without the need of retaining any previously seen samples, making it appealing to applications where data privacy or storage capacity is of concern. Experiments show that CL-BRUNO outperforms existing methods on both natural image and biomedical data sets, confirming its effectiveness in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。