提出动态正交微调，解决大模型持续学习中的灾难性遗忘问题。
Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
- 追踪任务间功能方向漂移，动态更新以保持历史知识。
- 使新任务梯度与旧任务方向正交，减少任务间干扰。
- 在多个基准上优于现有方法，适合长期模型微调场景。
灾难性遗忘仍是大语言模型持续学习的核心挑战，即在无历史数据访问的情况下，模型在顺序学习新任务时难以保持对旧任务的性能。本文首次揭示，现有基于正则化的方法在长期持续学习中失效的关键原因是功能方向在微调过程中发生漂移。为此，我们提出动态正交持续微调（DOC），该方法实时追踪这些功能方向的漂移，并在微调过程中动态更新。通过将新任务参数的梯度调整为与历史功能方向正交，有效缓解了新旧任务间的干扰。在多个大语言模型持续学习基准上的大量实验表明，该方法显著降低了灾难性遗忘，提升了模型连续学习的鲁棒性。代码已开源：https://github.com/meloxxxxxx/DOC。
原文摘要 · Abstract (English)
Catastrophic forgetting remains a critical challenge in continual learning for large language models (LLMs), where models struggle to retain performance on historical tasks when fine-tuning on new sequential data without access to past datasets. In this paper, we first reveal that the drift of functional directions during the fine-tuning process is a key reason why existing regularization-based methods fail in long-term LLM continual learning. To address this, we propose Dynamic Orthogonal Continual (DOC) fine-tuning, a novel approach that tracks the drift of these functional directions and dynamically updates them during the fine-tuning process. Furthermore, by adjusting the gradients of new task parameters to be orthogonal to the tracked historical function directions, our method mitigates interference between new and old tasks. Extensive experiments on various LLM continual learning benchmarks demonstrate that this approach outperforms prior methods, effectively reducing catastrophic forgetting and providing a robust tool for continuous LLM fine-tuning. Our code is available at https://github.com/meloxxxxxx/DOC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。