通过自适应SVD约束参数更新,实现无额外参数的持续学习。
Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning
- 用自适应SVD动态识别任务专属低秩子空间,限制更新方向。
- 在多个任务上平均准确率比O-LoRA高7%,遗忘几乎可忽略。
- 适合需要长期学习且不希望增加参数量的LLM应用。
大语言模型在持续学习中易发生灾难性遗忘,即在新任务上适应后显著降低旧任务性能。现有方法多依赖低秩、参数高效更新,限制模型表达能力并引入每任务额外参数,导致可扩展性差。为此,我们提出一种新型全量微调方法,基于自适应奇异值分解(SVD),动态识别任务特定的低秩参数子空间,并将更新约束为与先前任务关键方向正交,从而有效减少干扰,无需额外参数或存储历史梯度。我们在T5-Large和LLaMA-2 7B两种模型上,涵盖分类、生成和推理等多样化任务的标准持续学习基准上进行了全面评估。实验表明,该方法达到当前最优效果,平均准确率比O-LoRA等近期基线高出最多7%,且显著维持了模型的通用语言能力、指令遵循准确率和安全性,遗忘程度接近于零。自适应SVD框架在模型可塑性与知识保留间取得良好平衡,为大语言模型的持续学习提供了实用、理论坚实且计算可扩展的解决方案。
原文摘要 · Abstract (English)
Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones. Existing methods typically rely on low-rank, parameter-efficient updates that limit the model's expressivity and introduce additional parameters per task, leading to scalability issues. To address these limitations, we propose a novel continual full fine-tuning approach leveraging adaptive singular value decomposition (SVD). Our method dynamically identifies task-specific low-rank parameter subspaces and constrains updates to be orthogonal to critical directions associated with prior tasks, thus effectively minimizing interference without additional parameter overhead or storing previous task gradients. We evaluate our approach extensively on standard continual learning benchmarks using both encoder-decoder (T5-Large) and decoder-only (LLaMA-2 7B) models, spanning diverse tasks including classification, generation, and reasoning. Empirically, our method achieves state-of-the-art results, up to 7% higher average accuracy than recent baselines like O-LoRA, and notably maintains the model's general linguistic capabilities, instruction-following accuracy, and safety throughout the continual learning process by reducing forgetting to near-negligible levels. Our adaptive SVD framework effectively balances model plasticity and knowledge retention, providing a practical, theoretically grounded, and computationally scalable solution for continual learning scenarios in large language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。