让大模型学新知识时不丢旧能力，还能少用算力。
Control LLM: Controlled Evolution for Intelligence Retention in LLM
- 用并行扩展的神经层和插值对齐，防止学习新知识时遗忘旧任务。
- 在数学推理和编程上提升超10%，多语言能力增30%以上。
- 适合想高效微调大模型且不希望性能下降的研究者和工程师。
大型语言模型（LLMs）需要大量计算资源，因此必须在不从头训练的前提下增强其能力。该领域的一个关键挑战是灾难性遗忘（CF），它会损害持续预训练（CPT）和持续监督微调（CSFT）中的性能。我们提出Control LLM，一种新型方法，利用并行预训练与扩展的Transformer块，通过插值策略对齐其隐藏状态，从而在保留原有任务表现的同时无缝融入新知识。大量实验表明，Control LLM在CPT和CSFT中均有效。在Llama3.1-8B-Instruct上，数学推理能力提升14.4%（Math-Hard），编程能力提升10%（MBPP-PLUS）。在Llama3.1-8B上，多语言能力显著提升：C-Eval +10.6%，CMMLU +6.8%，CMMLU-0shot-CoT +30.2%。该方法优于现有方案，在相同基底模型上达到开源模型最佳水平，且使用更少数据与算力。关键优势在于，原始能力损失极小（MMLU下降<4.3%），远低于其他开源数学与编程模型（>35%）。该技术已成功部署于LinkedIn的生成式AI求职与广告产品中。为支持后续研究，我们公开了训练与评估代码（https://github.com/linkedin/ControlLLM）及基于公开数据集训练的模型（https://huggingface.co/ControlLLM）。
原文摘要 · Abstract (English)
Large Language Models (LLMs) demand significant computational resources, making it essential to enhance their capabilities without retraining from scratch. A key challenge in this domain is \textit{catastrophic forgetting} (CF), which hampers performance during Continuous Pre-training (CPT) and Continuous Supervised Fine-Tuning (CSFT). We propose \textbf{Control LLM}, a novel approach that leverages parallel pre-trained and expanded transformer blocks, aligning their hidden-states through interpolation strategies This method effectively preserves performance on existing tasks while seamlessly integrating new knowledge. Extensive experiments demonstrate the effectiveness of Control LLM in both CPT and CSFT. On Llama3.1-8B-Instruct, it achieves significant improvements in mathematical reasoning ($+14.4\%$ on Math-Hard) and coding performance ($+10\%$ on MBPP-PLUS). On Llama3.1-8B, it enhances multilingual capabilities ($+10.6\%$ on C-Eval, $+6.8\%$ on CMMLU, and $+30.2\%$ on CMMLU-0shot-CoT). It surpasses existing methods and achieves SOTA among open-source models tuned from the same base model, using substantially less data and compute. Crucially, these gains are realized while preserving strong original capabilities, with minimal degradation ($<4.3\% \text{on MMLU}$) compared to $>35\%$ in open-source Math and Coding models. This approach has been successfully deployed in LinkedIn's GenAI-powered job seeker and Ads unit products. To support further research, we release the training and evaluation code (https://github.com/linkedin/ControlLLM) along with models trained on public datasets (https://huggingface.co/ControlLLM) to the community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。