模仿大脑神经调制机制,提升模型持续学习的适应能力。
NeuMoSync: End-to-End Neuromodulatory Control for Plasticity and Adaptability in Continual Learning

- 为每个神经元引入可学习的动态调制向量,实时响应上下文状态。
- 在多个持续学习任务中显著提升前向与后向适应性能。
- 调制信号具可解释性,适合需要长期适应的AI系统研究者。
持续学习要求模型顺序学习任务,但深度神经网络常因可塑性下降和知识迁移差而影响长期适应能力。受大脑全局神经调制机制启发,我们提出NeuMoSync:一种将动态、神经元特异性调制整合进深度网络的新架构。该架构为每个神经元引入可学习的特征向量,追踪全网历史状态,并在更高抽象层级设置模块,基于当前输入与网络演化状态生成神经元特异性信号,自适应调节激活动态与突触可塑性。在多种持续学习基准上评估,包括记忆任务(随机标签CIFAR-10与MNIST)、概念漂移(打乱CIFAR-10与打乱Mini-ImageNet)、类别增量学习(类别分割ImageNet与类别分割CIFAR-100)及领域增量学习(置换MNIST),NeuMoSync在保持可塑性方面表现优异,相比现有方法在正向与反向适应上均有提升。消融实验验证各组件必要性,学习到的调制信号展现出可解释的任务间协调模式。本工作表明,将全局协调机制融入深度学习系统,有望推动更鲁棒、自适应的持续学习。代码公开于 https://github.com/RoozbehRazavi/NeuMoSync。
原文摘要 · Abstract (English)
Continual learning (CL) requires models to learn tasks sequentially, yet deep neural networks often suffer from plasticity loss and poor knowledge transfer, which can impede their long-term adaptability. Drawing high-level inspiration from global neuromodulatory mechanisms in the brain, we introduce Neuromodulation and Synchronization (NeuMoSync), a novel architecture that integrates dynamic, neuron-specific modulation into deep neural networks to enhance their adaptability and plasticity. NeuMoSync extends standard neural network architectures with learnable feature vectors for each neuron that track network-wide historical context and with a module operating at a higher level of abstraction. This module synthesizes neuron-specific signals, conditioned on both current inputs and the network's evolving state, to adaptively regulate activation dynamics and synaptic plasticity. Evaluated on diverse CL benchmarks, including memorization (Random Label CIFAR-10 and Random Label MNIST), concept drift (Shuffle CIFAR-10 and Shuffle Mini-ImageNet), class-incremental learning (Class Split ImageNet and Class Split CIFAR-100), and domain-incremental learning (Permuted MNIST), NeuMoSync demonstrates strong performance in retaining plasticity and achieves improvements in both forward and backward adaptation compared with existing methods. Ablation studies validate the necessity of each component, while analysis of the learned modulatory signals reveals interpretable coordination patterns across tasks. Our work underscores the potential of integrating global coordination mechanisms into deep learning systems to advance robust, adaptive continual learning. The code is publicly available at https://github.com/RoozbehRazavi/NeuMoSync.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。