用轻量生成器压缩记忆，显著降低持续学习的内存占用。
Leveraging Lightweight Generators for Memory Efficient Continual Learning
- 基于SVD的轻量生成器，仅需一次线性拟合，无需训练。
- 在少量样本下有效捕捉数据分布，平均准确率明显提升。
- 适合资源受限场景，尤其适用于记忆型持续学习任务。
灾难性遗忘可通过保存所有历史数据来缓解，因此在最小化内存开销的同时最大化信息保留至关重要。本文旨在降低基于记忆的持续学习算法所需的内存。我们探索从先前任务中提取最少信息以最大程度缓解遗忘的方案，提出利用基于奇异值分解（SVD）的轻量级生成器增强现有方法（如A-GEM和经验回放）。该生成器内存开销极小，无需训练时间，仅需一次线性时间拟合步骤即可从少量样本中有效捕捉数据分布。实验表明，根据数据集与网络架构不同，相比原方法平均准确率显著提升。本方法在降低记忆型持续学习算法内存开销方面具有巨大潜力。
原文摘要 · Abstract (English)
Catastrophic forgetting can be trivially alleviated by keeping all data from previous tasks in memory. Therefore, minimizing the memory footprint while maximizing the amount of relevant information is crucial to the challenge of continual learning. This paper aims to decrease required memory for memory-based continuous learning algorithms. We explore the options of extracting a minimal amount of information, while maximally alleviating forgetting. We propose the usage of lightweight generators based on Singular Value Decomposition to enhance existing continual learning methods, such as A-GEM and Experience Replay. These generators need a minimal amount of memory while being maximally effective. They require no training time, just a single linear-time fitting step, and can capture a distribution effectively from a small number of data samples. Depending on the dataset and network architecture, our results show a significant increase in average accuracy compared to the original methods. Our method shows great potential in minimizing the memory footprint of memory-based continual learning algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。