arXiv:2607.17944cs.LGcs.AI2026-07

不靠反向传播也能防遗忘,新模型在文本任务中表现远超传统方法。

The Art of Not Forgetting

论文配图:The Art of Not Forgetting
图 1 · 摘自论文原文
  • 用稀疏关系编码存储信息,全程本地更新无反向传播。
  • 跨15个文本领域测试,遗忘减少15到19倍,优于带EWC的Transformer。
  • 适合研究抗遗忘机制或非反向传播学习的学者参考。

我们提出CMP(认知记忆原语)架构,将输入表示为稀疏关系编码,存入两级竞争性记忆,并通过完全局部、无梯度的更新方式学习,网络中无任何反向传播。该架构用于验证一个假设:灾难性遗忘并非训练缺陷,而是反向传播分配信用的结构性结果;而一种从结构上局部且稀疏的学习规则,无需补丁即可抵抗遗忘。在15个文本领域的控制性增量任务中(三种子重复),CMP的后向迁移能力比同规模的在线EWC训练的Transformer高15-19倍,且在领域顺序控制实验中仍保持正向提升(+0.24至+0.44)。同时报告了与Transformer之间的显著准确率差距、在经典视觉基准上的零结果,以及尝试结合另一提升精度机制失败的诊断——因诚实披露负面结果更利于学术进步。核心论点明确可证伪:在特定条件下,非反向传播、局部且稀疏的学习,确实比标准反向传播加修复手段更能抵抗遗忘。

原文摘要 · Abstract (English)

We introduce CMP (Cognitive Memory Primitive), an architecture that represents inputs as sparse relational codes, stores them in a two-tier competitive memory, and learns entirely through local, gradient-free updates, with no backpropagation anywhere in the network. We use this architecture to test a specific hypothesis: that catastrophic forgetting, usually treated as a training-time defect to be patched with replay or regularization, is instead a structural consequence of how backpropagation assigns credit and that a learning rule that is local and sparse by construction should resist it without a patch. On a controlled domain-incremental protocol across 15 text domains, three-seed replicated, CMP's backward transfer is 15-19x better than a matched-size Transformer trained with online EWC, and the result survives a domain-order control (reported as a range, +0.24 to +0.44, rather than a single figure). We report this alongside a real, substantial accuracy gap versus the Transformer baseline, a null result on a recognized vision benchmark, and a diagnosed, unresolved failure attempting to combine this architecture with a separate mechanism that improves raw accuracy, disclosed because an honest negative result is more useful than an omitted one. The central claim is narrow and falsifiable: local, sparse, non-backpropagation learning measurably resists catastrophic forgetting better than backpropagation with its standard fix, under conditions we state precisely.

抗遗忘无反向传播稀疏学习记忆机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。