用自蒸馏让模型从示范中持续学习，不遗忘旧技能。
Self-Distillation Enables Continual Learning
- 用自身演示作为教师，生成在线学习信号
- 新任务准确率更高，遗忘程度大幅降低
- 适合需要长期积累知识的AI系统
持续学习使基础模型能在不损害已有能力的前提下不断获取新技能和知识，仍是重大挑战。尽管基于策略的强化学习可减少遗忘，但需显式奖励函数，往往不可得。当前主流替代方法是监督微调（SFT），但属于离线策略。本文提出自蒸馏微调（SDFT），一种直接从示范中实现在线学习的简单方法。SDFT通过上下文学习，以示范条件化的模型作为自身教师，生成保留原有能力的同时习得新技能的在线训练信号。在多个技能学习与知识获取任务中，SDFT始终优于SFT，新任务准确率更高，且显著减少灾难性遗忘。序列学习实验表明，单一模型可通过SDFT持续积累多技能而无性能下降，证明了基于示范的在线蒸馏是持续学习的可行路径。
原文摘要 · Abstract (English)
Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting, it requires explicit reward functions that are often unavailable. Learning from expert demonstrations, the primary alternative, is dominated by supervised fine-tuning (SFT), which is inherently off-policy. We introduce Self-Distillation Fine-Tuning (SDFT), a simple method that enables on-policy learning directly from demonstrations. SDFT leverages in-context learning by using a demonstration-conditioned model as its own teacher, generating on-policy training signals that preserve prior capabilities while acquiring new skills. Across skill learning and knowledge acquisition tasks, SDFT consistently outperforms SFT, achieving higher new-task accuracy while substantially reducing catastrophic forgetting. In sequential learning experiments, SDFT enables a single model to accumulate multiple skills over time without performance regression, establishing on-policy distillation as a practical path to continual learning from demonstrations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。