提出分层逐元素正则化，缓解大模型微调时的知识遗忘问题。
How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization
- 按参数重要性逐元素加约束，保护预训练知识
- 在科学、医疗等任务上显著减少遗忘，适应性更强
- 效率极高：速度比以往快20倍，存储仅需10%-15%
大型语言模型具备强大的通用语言能力，但在特定领域任务微调时易出现灾难性遗忘，即覆盖或丢失预训练阶段获得的关键知识，严重限制其广泛应用。为此，本文提出一种新方法，用于计算微调过程中对保留通用知识至关重要的参数的逐元素重要性。该方法采用双目标优化策略：(1) 基于参数重要性的正则化损失，约束对通用知识关键参数的更新；(2) 交叉熵损失以适应特定领域任务。同时引入分层系数，考虑各层贡献差异，动态平衡双目标优化。在GPT-J和LLaMA-3上针对科学、医学和物理任务的大量实验表明，该方法有效缓解灾难性遗忘，提升模型适应性。相比先前方法，本方案约快20倍，存储仅需10%-15%，兼具高效与实用优势。代码将公开。
原文摘要 · Abstract (English)
Large Language Models (LLMs) exhibit strong general language capabilities. However, fine-tuning these models on domain-specific tasks often leads to catastrophic forgetting, where the model overwrites or loses essential knowledge acquired during pretraining. This phenomenon significantly limits the broader applicability of LLMs. To address this challenge, we propose a novel approach to compute the element-wise importance of model parameters crucial for preserving general knowledge during fine-tuning. Our method utilizes a dual-objective optimization strategy: (1) regularization loss based on element-wise parameter importance, which constrains the updates to parameters crucial for general knowledge; (2) cross-entropy loss to adapt to domain-specific tasks. Additionally, we introduce layer-wise coefficients to account for the varying contributions of different layers, dynamically balancing the dual-objective optimization. Extensive experiments on scientific, medical, and physical tasks using GPT-J and LLaMA-3 demonstrate that our approach mitigates catastrophic forgetting while enhancing model adaptability. Compared to previous methods, our solution is approximately 20 times faster and requires only 10-15% of the storage, highlighting the practical efficiency. The code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。