用旧任务提示缓解大模型遗忘，实现持续学习。
Joint Flashback Adaptation for Forgetting-Resistant Instruction Tuning
- 引入旧任务提示作为记忆回溯，约束输出偏差。
- 通过隐空间插值联合学习新旧任务，提升知识共享。
- 无需原始数据，适配任意新任务，适合持续训练场景。
大语言模型在各类任务中表现卓越，但增量学习新任务时面临灾难性遗忘问题。现有方法依赖经验回放、优化约束或任务区分，实际应用受限。本文提出联合回溯适应（Joint Flashback Adaptation）：在适应新任务时引入少量旧任务提示，并约束模型输出与原输出的偏差；通过在隐空间中插值旧任务与新任务，实现对相关隐任务、新任务及旧提示的联合学习，缓解旧提示数据稀疏问题，促进知识迁移。本方法仅需少量回溯提示，无需重访原始数据，且不依赖具体任务。我们在1000+个指令遵循、算术推理与通用推理任务上对前沿大模型进行实验，结果表明该方法显著提升了新任务泛化能力，有效减少旧任务遗忘。
原文摘要 · Abstract (English)
Large language models have achieved remarkable success in various tasks. However, it is challenging for them to learn new tasks incrementally due to catastrophic forgetting. Existing approaches rely on experience replay, optimization constraints, or task differentiation, which encounter strict limitations in real-world scenarios. To address these issues, we propose Joint Flashback Adaptation. We first introduce flashbacks -- a limited number of prompts from old tasks -- when adapting to new tasks and constrain the deviations of the model outputs compared to the original one. We then interpolate latent tasks between flashbacks and new tasks to enable jointly learning relevant latent tasks, new tasks, and flashbacks, alleviating data sparsity in flashbacks and facilitating knowledge sharing for smooth adaptation. Our method requires only a limited number of flashbacks without access to the replay data and is task-agnostic. We conduct extensive experiments on state-of-the-art large language models across 1000+ instruction-following tasks, arithmetic reasoning tasks, and general reasoning tasks. The results demonstrate the superior performance of our method in improving generalization on new tasks and reducing forgetting in old tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。