arXiv:2604.27031cs.LGcs.AI2026-04

提出可自适应生长的神经网络,解决持续学习中容量不足或浪费的问题。

NORACL: Neurogenesis for Oracle-free Resource-Adaptive Continual Learning

  • 基于生物神经发生机制,动态增长网络以应对未知任务流。
  • 在多种任务设置下,性能优于或媲美已知最优静态模型,参数更少。
  • 生长模式可解释:不同任务类型触发不同层的扩展,提升可读性。

持续学习中,模型需兼具学习新任务的灵活性与保留旧知识的稳定性。现有方法受限于固定容量网络,隐含依赖未来任务未知的最优架构(即‘神谕’架构)。当任务关联弱时,固定网络逐渐耗尽可塑性资源;当任务少或高度重叠时,则过度配置。受生物神经发生启发,我们提出NORACL,通过监测表征与可塑性饱和度信号,在需要时仅动态增长网络。在不同任务数量与几何结构下评估显示,NORACL始终达到或超过神谕规模静态基线的平均准确率,同时使用更少参数。其生长具有可解释性:任务差异大时主增特征提取层,共享特征多时倾向增长特征融合层。分析揭示,固定容量网络随任务累积失去可塑性,而NORACL通过生长释放新容量。结果表明,自适应神经发生显著提升了持续学习的稳定性-可塑性权衡边界。

原文摘要 · Abstract (English)

In a continual learning setting, we require a model to be plastic enough to learn a new task and stable enough to not disturb previously learned capabilities. We argue that this dilemma has an architectural root. A finite network has limited representational and plastic resources, yet the required capacity depends on properties of the future task stream that are unknown: how many tasks will be encountered, and how much they overlap in feature space. Regularization-based methods preserve past knowledge within fixed-capacity architectures and therefore implicitly rely on an oracle architecture sized for this unknown future. When tasks are only weakly related, fixed architectures progressively run out of plastic resources; when tasks are few or strongly overlapping, models are often over-provisioned. Inspired by neurogenesis in biology, we propose NORACL to address the stability-plasticity dilemma by tackling the oracle architecture problem through neuronal growth. Starting from a compact network, NORACL grows only when needed by monitoring two complementary signals for representational and plasticity saturation. We evaluate NORACL against oracle-sized static baselines across varying task counts and geometries. Across all settings, NORACL achieves final average accuracies that are better than or on par with oracle-provisioned static baselines while using fewer parameters. Additionally, NORACL yields architectures with interpretable growth, i.e. dissimilar tasks predominantly expand feature-extraction layers, whereas tasks which rely on common features shift growth toward later feature-combination layers. Our analysis further explains why fixed-capacity networks lose plasticity as tasks accumulate, whereas NORACL creates fresh capacity for new tasks through growth. Together, these results show that adaptive neurogenesis pushes the stability-plasticity Pareto frontier of continual learning.

持续学习神经发生自适应架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。