arXiv:2602.22479cs.LG2026-02被引 2

让大模型像人一样持续学习,不丢旧知识还高效。

Efficient Continual Learning in Language Models via Thalamically Routed Cortical Columns

  • 用类脑结构分层管理记忆,选择性更新信息。
  • 在多个语言任务流上减少累积遗忘,优于主流模型。
  • 适合需要长期演进的AI应用,如客服、智能助手。

部署在真实场景中的大语言模型需适应不断变化的数据、用户行为和任务组合,但现有方法常导致灾难性遗忘。传统稳定化手段成本高、难扩展。本文提出TRC²(丘脑路由皮层柱),一种解码器架构,将持续学习内嵌于主干网络。该设计结合堆叠皮层柱、丘脑调制通路(选择性跨柱通信)与海马体路径(事件选择性检索、延迟惊喜写入、重放驱动巩固),实现快速可塑性与慢速稳定计算并存。引入因果记忆更新机制与在线重放控制器,根据遗忘程度动态调节巩固强度。在覆盖C4、WikiText-103和GSM8K的任务序列流中,TRC²显著提升任务边界建模质量,大幅降低累积遗忘,优于Transformer、Mamba、MoE及持续学习基线。消融实验表明,丘脑与海马组件对保留性能至关重要,全模型仍保持高效吞吐与训练成本竞争力。

原文摘要 · Abstract (English)

Large language models deployed in the wild must adapt to evolving data, user behavior, and task mixtures without erasing previously acquired capabilities. In practice, this remains difficult: sequential updates induce catastrophic forgetting, while many stabilization methods rely on external procedures that are costly, brittle, or difficult to scale. We present TRC$^{2}$ (Thalamically Routed Cortical Columns), a decoder-only architecture that makes continual learning a property of the backbone itself. TRC$^{2}$ combines stacked cortical columns with a thalamic modulatory pathway for selective inter-column communication and a hippocampal pathway for event selective retrieval, delayed surprise-based writing, and replay-driven consolidation. This design localizes fast plasticity while preserving a slower stable computation pathway. We further introduce a causal memory-update scheme and an online replay controller that adjusts consolidation strength from measured forgetting. Across a task-sequential language-modeling stream over C4, WikiText-103, and GSM8K, TRC$^{2}$ consistently improves task-boundary modeling quality and substantially reduces cumulative forgetting relative to Transformer, Mamba, MoE, DeepSeek and continual learning baselines trained under the same pipeline. Ablations show that the thalamic and hippocampal components are central to the retention gains, while the full model remains competitive in throughput and training cost.

持续学习大模型类脑架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。