arXiv:2507.21109cs.LGcs.AI2025-07

让神经网络像大学生一样复习,防止学新忘旧。

Task-Focused Consolidation with Spaced Recall: Making Neural Networks Learn like College Students

  • 用定期主动回忆机制检测模型对旧任务的掌握程度,稳定过往知识。
  • 在分割CIFAR-100上最终准确率达13.17%,远超标准经验回放的7.40%。
  • 适合资源受限场景,也适用于内存充足时提升学习效率。

深度神经网络常面临灾难性遗忘问题,即学习新任务后旧任务性能下降。本文提出受人类学习策略启发的持续学习方法——任务聚焦巩固与间隔重复(TFC-SR),在标准经验回放基础上引入主动回忆探针机制。该机制周期性、任务感知地评估模型记忆状态,稳定过去知识的表征。在分割MNIST和分割CIFAR-100基准上测试,TFC-SR显著优于主流正则化与回放基线方法。例如,在分割CIFAR-100上,其最终准确率达13.17%,而标准经验回放仅为7.40%。结果表明该优势源于探针本身的稳定作用,而非回放数据量差异。此外,分析了内存大小与性能的权衡:尽管在内存受限环境下表现更优,但当内存充足时,更大回放量仍更有效。结论表明TFC-SR是一种鲁棒高效的持续学习方法,强调将主动记忆检索机制融入系统的重要性。

原文摘要 · Abstract (English)

Deep neural networks often suffer from a critical limitation known as catastrophic forgetting, where performance on past tasks degrades after learning new ones. This paper introduces a novel continual learning approach inspired by human learning strategies like Active Recall, Deliberate Practice, and Spaced Repetition, named Task-Focused Consolidation with Spaced Recall (TFC-SR). TFC-SR enhances the standard experience replay framework with a mechanism we term the Active Recall Probe. It is a periodic, task-aware evaluation of the model's memory that stabilizes the representations of past knowledge. We test TFC-SR on the Split MNIST and the Split CIFAR-100 benchmarks against leading regularization-based and replay-based baselines. Our results show that TFC-SR performs significantly better than these methods. For instance, on the Split CIFAR-100, it achieves a final accuracy of 13.17% compared to Standard Experience Replay's 7.40%. We demonstrate that this advantage comes from the stabilizing effect of the probe itself, and not from the difference in replay volume. Additionally, we analyze the trade-off between memory size and performance and show that while TFC-SR performs better in memory-constrained environments, higher replay volume is still more effective when available memory is abundant. We conclude that TFC-SR is a robust and efficient approach, highlighting the importance of integrating active memory retrieval mechanisms into continual learning systems.

持续学习记忆巩固主动回忆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。