用快慢双网络模拟记忆巩固，缓解持续学习中的遗忘问题。
FSC-Net: Fast-Slow Consolidation Networks for Continual Learning
- 快网络快速学新任务，慢网络通过重放逐步巩固旧知识。
- 在分割MNIST上保留准确率达91.71%，比单网络提升4.27个百分点。
- 简单MLP比复杂结构更有效，说明方法比架构更重要。
持续学习面临灾难性遗忘问题，即神经网络在学习新任务时会丢失已有知识。受神经科学中记忆巩固机制启发，我们提出FSC-Net（快-慢巩固网络），一种双网络架构，将快速任务学习与渐进式知识巩固分离。快网络（NN1）用于即时适应新任务，慢网络（NN2）通过知识蒸馏和重放实现知识巩固。在评估的多种基于MLP的NN1变体中，巩固效果更多由方法决定而非架构复杂度——一个简单MLP反而比更复杂的相似性门控变体高出1.2个百分点。系统性超参数分析表明，仅使用重放而无蒸馏的巩固方式表现更优，支持了从快网络蒸馏会引入近期偏差的假设。在Split-MNIST（30个种子）上，FSC-Net达到91.71% ± 0.62%的保留准确率，相比仅快网络的87.43% ± 1.27%提升了4.27个百分点（配对t=23.585，p < 1e-10）。在Split-CIFAR-10（5个种子）上，本方法达到33.31% ± 0.38%的保留准确率，较快网络提升8.20个百分点（25.11% ± 1.61%，配对t=9.75，p < 1e-3），尽管绝对性能仍偏低且低于随机期望，凸显强骨干模型的必要性。结果为双时间尺度巩固机制在缓解灾难性遗忘中的核心作用提供了实证支持。
原文摘要 · Abstract (English)
Continual learning remains challenging due to catastrophic forgetting, where neural networks lose previously acquired knowledge when learning new tasks. Inspired by memory consolidation in neuroscience, we propose FSC-Net (Fast-Slow Consolidation Networks), a dual-network architecture that separates rapid task learning from gradual knowledge consolidation. Our method employs a fast network (NN1) for immediate adaptation to new tasks and a slow network (NN2) that consolidates knowledge through distillation and replay. Within the family of MLP-based NN1 variants we evaluated, consolidation effectiveness is driven more by methodology than architectural embellishments -- a simple MLP outperforms more complex similarity-gated variants by 1.2pp. Through systematic hyperparameter analysis, we observed empirically that pure replay without distillation during consolidation achieves superior performance, consistent with the hypothesis that distillation from the fast network introduces recency bias. On Split-MNIST (30 seeds), FSC-Net achieves 91.71% +/- 0.62% retention accuracy, a +4.27pp gain over the fast network alone (87.43% +/- 1.27%, paired t=23.585, p < 1e-10). On Split-CIFAR-10 (5 seeds), our method achieves 33.31% +/- 0.38% retention with an +8.20pp gain over the fast network alone (25.11% +/- 1.61%, paired t=9.75, p < 1e-3), demonstrating +8.20pp gain, though absolute performance (33.31%) remains modest and below random expectation, highlighting need for stronger backbones. Our results provide empirical evidence that the dual-timescale consolidation mechanism, rather than architectural complexity, is central to mitigating catastrophic forgetting in this setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。