如何让大模型持续学习新知识而不遗忘旧知识？
Collaboratively adding new knowledge to an LLM
- 用LoRA微调，比全参数微调更擅长保留已有知识
- 在无数据场景下，正交子空间学习效果最佳
- 适合需要长期更新知识的大模型应用
本文研究如何在不丢失已有知识的前提下，持续向大语言模型添加新知识。考虑半合作与全合作两种场景：在半合作场景（训练后数据不可用）中，基于正交子空间的序列学习、MoE混合和模型融合表现良好；在全合作场景（数据仍可用）中，联合训练与带回放的序列训练均有效，且LoRA微调优于全参数微调。所有实验结果均可通过开源代码复现。
原文摘要 · Abstract (English)
We address the question of how to successively add new knowledge to an LLM whilst retaining previously-added knowledge. We consider two settings, semi-cooperative and fully-cooperative. Overall, LoRA performs better in most cases than full-fine tuning of all parameters when both new knowledge acquisition and retention of old, including recent, knowledge are taken into account. In the semi-cooperative setting, where datasets are not available after training, MOE mixing, model merging, and LoRA-based orthogonal subspace sequential learning, using a small weight on the orthogonality term, perform well. In the fully-cooperative setting where datasets remain available, joint training and sequential training with replay are both effective approaches with LoRA training generally preferable to full fine-tuning. The codes needed to reproduce the results are provided in an open source repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。