arXiv:2604.24637cs.LGcs.AI2026-04

受大脑皮层启发,实现无监督任务分割与零遗忘持续学习。

Cortex-Inspired Continual Learning: Unsupervised Instantiation and Recovery of Functional Task Networks

论文配图:Cortex-Inspired Continual Learning: Unsupervised Instantiation and Recovery of Functional Task Networks
图 1 · 摘自论文原文
  • 用类树突结构的二值掩码动态选择小深度网络,实现参数隔离。
  • 在三个基准上几乎零遗忘,最快版本仅两步平滑即可恢复旧任务。
  • 适合追求高鲁棒性持续学习的科研与工业应用。

块序列式持续学习要求单一模型既能防止灾难性遗忘,又能在无任务标签时高效判断当前输入匹配哪个旧任务。我们提出功能任务网络(FTN),一种受哺乳动物新皮层结构与动力学特征启发的参数隔离方法。类似专家混合模型,该方法通过高维自组织二值掩码控制大规模小型深层网络群体,灵感来自锥体神经元的树突模型。掩码生成分三步:(1) 对连续掩码进行梯度下降以识别任务相关神经元,(2) 使用平滑核引导结果向空间连贯性偏移,(3) 固定容量预算下通过k-赢家通吃机制二值化。每个神经元是独立深层网络,互不重叠的掩码带来精确的梯度更新隔离,提供对抗灾难性遗忘的结构保障。该三步过程可在单步梯度更新中恢复先前训练任务的子网络,实现在推理时无监督的任务分割。我们在三个持续学习基准上测试:(1) 合成多任务分类/回归生成器,(2) 打乱类别标签的MNIST(纯概念漂移),(3) 变换像素顺序的Permuted MNIST(领域漂移)。所有实验中,细粒度平滑的FTN-Slow实现近乎零遗忘;使用大核且仅两步平滑的FTN-Fast牺牲部分保留率换取更高效率。我们证明,空间组织机制将掩码搜索复杂度从组合型的O(C(H,K))降低至近线性的O(H),仅需在紧凑皮层邻域内扫描,且可通过基于梯度的更新并行化。

原文摘要 · Abstract (English)

Block-sequential continual learning demands that a single model both protect prior solutions from catastrophic forgetting and efficiently infer at inference time which prior solution matches the current input without task labels. We present Functional Task Networks (FTN), a parameter-isolation method inspired by structural and dynamical motifs found in the mammalian neocortex. Similar to mixture-of-experts, this method uses a high dimensional, self-organizing binary mask over a large population of small but deep networks, inspired by dendritic models of pyramidal neurons. The mask is produced by a three-stage procedure: (1) gradient descent on a continuous mask identifies task-relevant neurons, (2) a smoothing kernel biases the result toward spatial contiguity, (3) and k-winner-take-all binarizes the resulting group at a fixed capacity budget. Like mixture-of-experts, each neuron is an independent deep network, so disjoint masks give exactly disjoint gradient updates, providing structural guarantees against catastrophic forgetting. This three-stage procedure recovers the sub-network of a previously-trained task in a single gradient step, providing unsupervised task segmentation at inference time. We test it on three continual-learning benchmarks: (1) a synthetic multi-task classification/regression generator, (2) MNIST with shuffled class labels (pure concept shift), and (3) Permuted MNIST (domain shift). On all three, FTN with fine grained smoothing (FTN-Slow) results in nearly zero forgetting. FTN with a large kernel and only 2 iterations of smoothing (FTN-Fast) trades off some retention for increased speed. We show that the spatial organization mechanism reduces the effective mask search from the combinatorial top-k subset problem in O(C(H,K)) to the complexity of a near-linear scan in O(H) over compact cortical neighborhoods, which is parallelized by the gradient-based update.

持续学习神经形态任务分割零遗忘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。