arXiv:2603.15184cs.LGcs.AI2026-03中稿 · publication in the…被引 1

用动态阈值神经元实现无遗忘的类脑持续学习。

CATFormer: When Continual Learning Meets Spiking Transformers With Dynamic Thresholds

  • 引入动态阈值脉冲神经元,通过调节神经元兴奋性防止遗忘。
  • 在多个数据集上超越现有无重放方法,任务累积下仍保持高准确率。
  • 适合需要低功耗、长期学习的类脑计算系统开发者使用。

尽管深度神经网络在受控环境下表现优异,但在现实场景中面临数据非一次性到达的问题,模型需适应新数据分布,而旧知识常因后续更新导致灾难性遗忘。相比之下,大脑能持续学习而不发生遗忘。现有的用于类别增量学习(CIL)的脉冲神经网络(SNN)随着任务增加性能急剧下降。本文提出CATFormer(上下文自适应阈值变换器),一种可扩展的框架以克服此限制。我们发现防止遗忘的关键不仅在于突触可塑性,还在于调节神经元兴奋性。CATFormer的核心是动态阈值漏电积分-发放(DTLIF)神经元模型,利用上下文自适应阈值作为主要知识保留机制,并结合门控动态头选择(G-DHS)实现无任务依赖推理。在静态(CIFAR-10/100/Tiny-ImageNet)和类脑(CIFAR10-DVS/SHD)数据集上的广泛评估表明,CATFormer在多种任务划分下均优于现有无重放的CIL算法,确立了其在节能、真实类别增量学习中的理想架构地位。

原文摘要 · Abstract (English)

Although deep neural networks perform extremely well in controlled environments, they fail in real-world scenarios where data isn't available all at once, and the model must adapt to a new data distribution that may or may not follow the initial distribution. Previously acquired knowledge is lost during subsequent updates based on new data. a phenomenon commonly known as catastrophic forgetting. In contrast, the brain can learn without such catastrophic forgetting, irrespective of the number of tasks it encounters. Existing spiking neural networks (SNNs) for class-incremental learning (CIL) suffer a sharp performance drop as tasks accumulate. We here introduce CATFormer (Context Adaptive Threshold Transformer), a scalable framework that overcomes this limitation. We observe that the key to preventing forgetting in SNNs lies not only in synaptic plasticity but also in modulating neuronal excitability. At the core of CATFormer is the Dynamic Threshold Leaky Integrate-and-Fire (DTLIF) neuron model, which leverages context-adaptive thresholds as the primary mechanism for knowledge retention. This is paired with a Gated Dynamic Head Selection (G-DHS) mechanism for task-agnostic inference. Extensive evaluation on both static (CIFAR-10/100/Tiny-ImageNet) and neuromorphic (CIFAR10-DVS/SHD) datasets reveals that CATFormer outperforms existing rehearsal-free CIL algorithms across various task splits, establishing it as an ideal architecture for energy-efficient, true-class incremental learning.

脉冲神经网络持续学习类脑计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。