提出新方法缓解持续学习中的遗忘问题，降低内存消耗90%。
Mitigating Forgetting in Continual Learning with Selective Gradient Projection
- 通过余弦相似度和分层门控动态调节梯度方向
- 在标准基准上实现高精度，内存减少90%
- 适合资源受限场景，兼顾学习新知识与保留旧知识
随着神经网络在动态环境中的广泛应用，其面临灾难性遗忘的挑战——在适应新任务时会覆盖先前学到的知识，导致旧任务性能严重下降。本文提出选择性遗忘感知优化（SFAO），一种动态方法，通过余弦相似度和分层门控调节梯度方向，实现可控遗忘的同时平衡可塑性与稳定性。SFAO利用可调机制选择性地投影、接受或丢弃更新，并采用高效的蒙特卡洛近似。在标准持续学习基准上的实验表明，SFAO在保持竞争力准确率的同时，内存成本降低90%，在MNIST数据集上显著改善遗忘问题，适用于资源受限场景。
原文摘要 · Abstract (English)
As neural networks are increasingly deployed in dynamic environments, they face the challenge of catastrophic forgetting, the tendency to overwrite previously learned knowledge when adapting to new tasks, resulting in severe performance degradation on earlier tasks. We propose Selective Forgetting-Aware Optimization (SFAO), a dynamic method that regulates gradient directions via cosine similarity and per-layer gating, enabling controlled forgetting while balancing plasticity and stability. SFAO selectively projects, accepts, or discards updates using a tunable mechanism with efficient Monte Carlo approximation. Experiments on standard continual learning benchmarks show that SFAO achieves competitive accuracy with markedly lower memory cost, a 90$\%$ reduction, and improved forgetting on MNIST datasets, making it suitable for resource-constrained scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。