让大模型持续学习多领域知识，不遗忘还能跨领域组合使用。
Brainstacks: Cross-Domain Cognitive Capabilities via Frozen MoE-LoRA Stacks for Continual LLM Learning
- 用冻结的模块化适配器堆叠，分阶段添加新领域知识。
- 比传统方法快2.5倍，且能恢复因叠加导致的生成质量下降。
- 发现领域模块蕴含通用认知能力，适合需要持续学习的场景。
我们提出Brainstacks，一种用于大语言模型持续多领域微调的模块化架构，将领域专长封装为在共享冻结基座上加性叠加的冻结适配器堆栈。五个核心组件：(1) 基于QLoRA 4-bit量化与rsLoRA缩放的Shazeer风格噪声top-2路由的MoE-LoRA，覆盖所有七个Transformer投影；(2) 内循环通过冻结已训练堆栈并新增堆栈实现残差增强；(3) 外循环按课程顺序训练领域特定堆栈，具有依赖关系；(4) 通过随机SVD进行零空间投影，使新堆栈约束在与先前方向正交的子空间，实现孤立状态下零遗忘；(5) 基于实证发现的领域组合目标训练的结果导向型sigmoid元路由器，选择性加权堆栈，支持跨域组合。两个边界实验：(6) 在随机初始化模型上进行PSN预训练；(7) 每领域强化学习（DPO/GRPO）验证与后SFT对齐兼容性。在TinyLlama-1.1B（4个领域，9个堆栈）和Gemma 3 12B IT（5个领域，10个堆栈）上验证，MoE-LoRA收敛速度达参数匹配单个LoRA的2.5倍，残差增强突破单堆栈上限，路由系统恢复了因无门控堆栈累积导致的生成质量损失。核心发现：结果导向路由器揭示，领域堆栈编码的是可迁移的认知原语（指令遵循清晰度、数值推理、程序逻辑、链式思维结构），而非领域特异性知识——医学提示在97%情况下路由至聊天+数学堆栈，尽管这些堆栈中未包含任何医学数据。
原文摘要 · Abstract (English)
We present Brainstacks, a modular architecture for continual multi-domain fine-tuning of large language models that packages domain expertise as frozen adapter stacks composing additively on a shared frozen base at inference. Five interlocking components: (1) MoE-LoRA with Shazeer-style noisy top-2 routing across all seven transformer projections under QLoRA 4-bit quantization with rsLoRA scaling; (2) an inner loop performing residual boosting by freezing trained stacks and adding new ones; (3) an outer loop training sequential domain-specific stacks with curriculum-ordered dependencies; (4) null-space projection via randomized SVD constraining new stacks to subspaces orthogonal to prior directions, achieving zero forgetting in isolation; (5) an outcome-based sigmoid meta-router trained on empirically discovered domain-combination targets that selectively weights stacks, enabling cross-domain composition. Two boundary experiments: (6) PSN pretraining on a randomly initialized model; (7) per-domain RL (DPO/GRPO) validating compatibility with post-SFT alignment. Validated on TinyLlama-1.1B (4 domains, 9 stacks) and Gemma 3 12B IT (5 domains, 10 stacks), MoE-LoRA achieves 2.5x faster convergence than parameter-matched single LoRA, residual boosting breaks through the single-stack ceiling, and the routed system recovers generation quality destroyed by ungated stack accumulation. The central finding: the outcome-based router discovers that domain stacks encode transferable cognitive primitives (instruction-following clarity, numerical reasoning, procedural logic, chain-of-thought structure) rather than domain-specific knowledge, with medical prompts routing to chat+math stacks in 97% of cases despite zero medical data in those stacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。