提出SABER框架，让模型在持续学习中安全回传知识，避免遗忘。
Turning Back Without Forgetting: Selective Backward Refinement for Parameter-Efficient Continual Learning
- 基于提示梯度与损失分布的互补判断，决定何时回传知识
- 仅在不干扰的参数方向更新提示，防止负面回传
- 在T5、LLaMA等模型上均实现正向回传且性能稳定
提示驱动的参数高效持续学习虽能缓解灾难性遗忘，但任务提示的隔离也限制了后期任务对早期任务的改进，导致反向知识迁移未被充分探索。本文提出SABER（Selective bAckward refinement for positive Backward knowledge transfER），一种无需重放的框架，实现可控的反向知识迁移。SABER通过提示梯度几何与损失分布相似性构建互补任务相关性判据，判断何时进行反向优化；并通过限制更新方向至非干扰区域，确保优化安全。在多个持续学习基准及多种预训练模型（包括T5-Large、LLaMA、Qwen）上的实验表明，SABER在保持优异平均性能的同时，始终实现正向反向迁移。代码已开源。
原文摘要 · Abstract (English)
While prompt-based parameter-efficient continual learning mitigates catastrophic forgetting by isolating task-specific prompts, this isolation also limits later tasks from improving earlier ones, leaving backward knowledge transfer underexplored. We address this limitation by proposing Selective bAckward refinement for positive Backward knowledge transfER (SABER), a replay-free framework that enables controlled backward transfer in prompt-based continual learning. SABER determines when backward refinement is beneficial using complementary task-correlation criteria based on prompt-gradient geometry and loss-distribution similarity, and how to perform refinement safely by restricting updates to non-interfering directions in the prompt parameter space. Extensive experiments across multiple continual learning benchmarks and diverse pretrained backbones, including T5-Large, LLaMA, and Qwen, demonstrate that SABER consistently achieves positive backward transfer while maintaining strong overall average performance. Code is available at https://github.com/OptMN-Lab/SABER-ICML-2026/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。