无需记忆缓冲或任务标签，实现大模型持续预训练不遗忘。
TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale
- 通过读写分离架构，仅更新特定参数块避免干扰原有知识。
- 在六类文本上保持99.59%梯度正交性，且跨域前向迁移提升超60%。
- 适合追求自主持续学习的大模型研发团队使用。
在不使用回放缓冲、任务标识符或正则化惩罚的情况下，对大规模语言模型进行无任务标签的持续预训练，仍是大模型规模下的未解难题。本文提出TFGN，一种用于Transformer语言模型的架构增强层，可实现输入相关的、参数高效的更新，同时保持原模型结构不变。在六个异构文本领域（散文、Python、数学、生物医学、中文、JavaScript）上，每阶段10亿词元数据，覆盖三个模型尺度（约398M、739M、9B）和两种训练模式（从头训练与重训），TFGN在LLaMA 3.1 8B重训中实现-0.007的后向迁移性能，HellaSwag保留率0.506/0.504/0.510，并确保不同领域间参数更新的L2正交性达99.59%以上，无需任何回放或任务标签。同一框架下，仅通过Python训练即可使未见的JavaScript模型困惑度降低26.8%（在LLaMA-8B重训中）和62.0%（在GPT-2 Medium从头训练中），体现显著跨域前向迁移能力。进一步扩展中，闭环元控制层（扩展A）在约398M模型上额外减少81%遗忘，对应系统A与系统M角色；操作级计划向量（扩展B）在30个源到目标映射中以99.96%余弦相似度重塑前向行为。核心思想是读/写分解：前向传播保持全连接，而跨领域参数更新被设计为不写入先前领域子空间。
原文摘要 · Abstract (English)
Continually pre-training a large language model on heterogeneous text domains, without replay or task labels, has remained an unsolved architectural problem at LLM scale. Existing methods rely on replay buffers, task identifiers, regularization penalties that scale poorly, or sentence-classification-scale evaluation. We introduce TFGN, an architectural overlay for transformer language models that produces input-conditioned, parameter-efficient updates while leaving the rest of the transformer unchanged. On six heterogeneous text domains (Prose, Python, Math, Biomedical, Chinese, JavaScript) at 1B tokens per phase across three model scales (~398M, ~739M, ~9B) and two regimes (From-Scratch and Retrofit), TFGN achieves backward transfer of -0.007 at LLaMA 3.1 8B Retrofit, HellaSwag retention 0.506/0.504/0.510, and >=99.59% L2-orthogonal gradient separation between domain pairs - with no replay, no task IDs, no Fisher penalty. The same matrices show positive cross-domain forward transfer: held-out JavaScript PPL drops 26.8% at LLaMA-8B Retrofit and 62.0% at GPT-2 Medium From-Scratch purely from Python training. Two extensions on the same substrate close further open problems. A closed-loop meta-control layer (Extension A) reduces forgetting by an additional 81% at ~398M, mapping onto the System A and System M roles of Dupoux et al. (arXiv:2603.15381). An operator-level plan vector (Extension B) reshapes forward-pass behavior at 99.96% cosine fidelity over 30 source->target pairs. The architectural insight is a Read/Write decomposition: the forward pass is fully dense, while cross-domain parameter updates are structured so prior-domain subspaces are not written to. To our knowledge, TFGN is the first architecture that simultaneously closes catastrophic forgetting at LLM scale, realizes a closed-loop autonomous-learning meta-controller, and carries an operator-level latent planner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。