提出PLAN框架，让大模型在持续学习中高效适应新任务且不遗忘旧知识。
PLAN: Proactive Low-Rank Allocation for Continual Learning
- 通过正向分配低秩子空间，动态管理每个任务的参数更新方向。
- 在多个基准上超越现有方法，显著降低对旧知识的干扰。
- 适合需要长期迭代训练的大模型应用，如智能客服、多阶段推理系统。
持续学习（CL）要求模型在不遗忘旧知识的前提下持续适应新任务。本文提出一种名为PLAN（Proactive Low-Rank Allocation）的框架，将低秩适配（LoRA）扩展至持续学习场景，实现大预训练模型的高效且抗干扰的微调。PLAN通过为每个任务引入正交基向量，并采用基于扰动的优化策略，最小化与已有参数的冲突。此外，框架设计了一种新颖的选择机制，识别并分配对干扰敏感度最低的基向量，从而在保持对新任务高效适应的同时，有效降低对旧知识的破坏风险。在标准持续学习基准上的实验表明，PLAN持续优于现有方法，为基于基础模型的持续学习建立了新的性能标杆。
原文摘要 · Abstract (English)
Continual learning (CL) requires models to continuously adapt to new tasks without forgetting past knowledge. In this work, we propose \underline{P}roactive \underline{L}ow-rank \underline{A}llocatio\underline{N} (PLAN), a framework that extends Low-Rank Adaptation (LoRA) to enable efficient and interference-aware fine-tuning of large pre-trained models in CL settings. PLAN proactively manages the allocation of task-specific subspaces by introducing orthogonal basis vectors for each task and optimizing them through a perturbation-based strategy that minimizes conflicts with previously learned parameters. Furthermore, PLAN incorporates a novel selection mechanism that identifies and assigns basis vectors with minimal sensitivity to interference, reducing the risk of degrading past knowledge while maintaining efficient adaptation to new tasks. Empirical results on standard CL benchmarks demonstrate that PLAN consistently outperforms existing methods, establishing a new state-of-the-art for continual learning with foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。