提出新型网络架构，实现无需重放数据的持续学习
A Proposal for Networks Capable of Continual Learning
- 采用响应保持机制替代梯度更新，实现记忆保留
- 在MNIST和简单环境建模中成功实现无任务边界持续学习
- 适合需要长期学习且无法存储历史数据的场景
我们分析了计算单元在参数更新后保留过去响应的能力，这是系统级持续学习的关键特性。基于梯度下降训练的神经网络缺乏这一能力，因此我们提出Modelleyen，一种具备固有响应保持特性的替代方法。通过在简单环境动态建模和MNIST上的实验表明，尽管当前阶段存在计算复杂度较高和表征能力有限的问题，Modelleyen仍可在不依赖样本重放或预定义任务边界的情况下实现持续学习。
原文摘要 · Abstract (English)
We analyze the ability of computational units to retain past responses after parameter updates, a key property for system-wide continual learning. Neural networks trained with gradient descent lack this capability, prompting us to propose Modelleyen, an alternative approach with inherent response preservation. We demonstrate through experiments on modeling the dynamics of a simple environment and on MNIST that, despite increased computational complexity and some representational limitations at its current stage, Modelleyen achieves continual learning without relying on sample replay or predefined task boundaries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。