不用梯度和旧数据，用进化算法防止模型遗忘旧知识。
Gradient-Free Continual Learning
- 用进化算法更新网络，无需梯度或存储旧数据。
- 在多个基准上达到标准方法水平，内存受限下表现优异。
- 适合隐私敏感或内存受限场景的持续学习应用。
神经网络在学习新知识时容易遗忘旧技能，即灾难性遗忘问题。传统持续学习方法依赖保存旧数据或复杂梯度更新，但在内存或隐私限制下无法使用。为此，我们提出EvoCL，一种无梯度方法，利用进化算法更新网络，无需旧数据或梯度。EvoCL通过轻量级适配器模块，将过往任务的保存表示转换到当前模型空间，实现新任务学习的同时保留旧知识。在多个基准测试中，EvoCL在严格内存约束下表现匹配或超越标准方法，提供了一种简单灵活的持续学习新方向。代码已开源：https://github.com/grypesc/EvoCL。
原文摘要 · Abstract (English)
Neural networks are notorious for forgetting old skills when taught new ones - a problem known as catastrophic forgetting. Standard continual learning techniques try to fix this by saving old data or relying on complex gradient updates, but these methods fail when past data cannot be stored due to memory or privacy constraints. To solve this, we propose EvoCL, a gradient-free approach that uses evolutionary algorithms to update the network without needing old data or gradients. EvoCL uses a lightweight adapter module to translate saved representations from past tasks into the model's current space, allowing it to learn new tasks while keeping past knowledge intact. Across multiple benchmarks, EvoCL matches or exceeds standard performance under strict memory constraints, offering a simple and flexible new direction for continual learning. The code to reproduce these results is available at https://github.com/grypesc/EvoCL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。