arXiv:2608.14634cs.LG2026-08

用生物突触可塑性思想改进神经网络,避免遗忘旧知识。

Metaplasticity as adaptive gradient preconditioning for incremental learning

论文配图:Metaplasticity as adaptive gradient preconditioning for incremental learning
图 1 · 摘自论文原文
  • 通过动态计算信息矩阵,自动调节梯度更新强度。
  • 在CIFAR-100上比EWC++高4倍准确率,遗忘率降低超10%。
  • 无需任务标签和大量内存,适合边缘设备持续学习。

生物智能通过互补学习系统理论自然防止灾难性遗忘,其微观机制是突触元可塑性:基于历史的局部神经调制。人工神经网络在非平稳环境中难以平衡稳定性与可塑性,现有方法常需任务标签或大量内存,偏离生物现实。本文将局部神经调制重新建模为优化驱动过程,提出SynGAP:基于自适应梯度预处理的无任务持续学习框架。SynGAP通过在连续数据流上维护费雪信息矩阵的指数移动平均,模拟实时元可塑性。优化时,动态元可塑状态转化为有界乘性掩码,选择性抑制关键历史参数的更新。实验表明,SynGAP显著缓解遗忘:在Split CIFAR-100上准确率较EWC++提升4倍,优于经验回放(ER)近10%,遗忘率降低超10%;在CORe50上达约68%,较基线提升10%。通过数学形式化连续生物元可塑性为稳定梯度正则化,SynGAP提供了一种高效、低内存的边缘自适应智能解决方案。

原文摘要 · Abstract (English)

Biological intelligence naturally prevents catastrophic forgetting through Complementary Learning Systems (CLS) theory, a macroscopic consolidation process driven at the local level by synaptic metaplasticity: the continuous, history-dependent neuromodulation of individual synapses. While artificial neural networks struggle with the stability-plasticity dilemma in non-stationary environments, existing solutions often require task labels or incur massive memory overhead, diverging from biological reality. Re-framing this localized neuromodulation as an optimization-driven process, we introduce $\textbf{SynGAP}$: $\textbf{Syn}$aptic $\textbf{G}$eometric $\textbf{A}$daptive $\textbf{P}$reconditioning. SynGAP is a task-free continual learning framework based on adaptive gradient preconditioning. Rather than relying on explicit episodic triggers, SynGAP simulates real-time metaplasticity by maintaining an exponential moving average of the Fisher Information Matrix over a continuous data stream. During the optimization step, these dynamic metaplastic states are translated into a bounded multiplicative mask that preconditions raw gradients, selectively attenuating updates to critical historical parameters. Empirical evaluations demonstrate SynGAP's superior ability to mitigate catastrophic forgetting compared to established baselines. On the Split CIFAR-100 benchmark, SynGAP delivers a $4\times$ increase in accuracy compared to EWC++ and outperforms Experience Replay (ER) by almost $10\%$, while reducing the forgetting measure by over $10\%$ against both methods. Furthermore, on the CORe50 benchmark, SynGAP achieves about $68\%$, a $10\%$ improvement over optimizer baselines. By mathematically formalizing continuous biological metaplasticity as stable gradient-based regularization, SynGAP offers a highly robust and memory-efficient solution for adaptive intelligence at the edge.

持续学习元可塑性梯度预处理边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。