语言模型可用自生成数据缓解遗忘，提升微调效率。
Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
- 利用模型自身生成的数据作为回放样本，替代传统存储的旧数据。
- 容量充足时，高学习率微调结合回放可几乎消除遗忘。
- 适合需要快速微调且避免知识丢失的研究者与工程师。
在新任务上训练的语言模型通常会退化旧任务表现，即遗忘现象。传统缓解方法依赖存储并回放旧任务样本，但往往不切实际。而语言模型能从自身训练分布中采样，我们发现这些自生成样本可作为有效回放数据，几乎消除遗忘。当模型接近容量饱和时，遗忘仍存在：无法在不覆盖旧知识的前提下吸收新信息。当容量非瓶颈时，低学习率虽能减少遗忘，但需大量训练步数。回放机制打破此权衡，使高学习率、快速微调成为可能且无遗忘。
原文摘要 · Abstract (English)
Models trained on a new task typically degrade on prior tasks, a phenomenon known as forgetting. Traditionally, mitigating forgetting has required replaying stored exemplars from prior tasks, which is often impractical. By contrast, language models can sample from their own training distribution, and we show that these self-generated samples serve as effective replay data, nearly eliminating forgetting. We find that forgetting nonetheless persists when the model has little remaining capacity: models pretrained close to saturation cannot absorb new information without overwriting prior knowledge. When capacity is not the limiting factor, low learning rates reduce forgetting but require substantially more training steps. Replay breaks this tradeoff, enabling fast, high-learning-rate finetuning without forgetting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。