arXiv:2607.07847cs.LG2026-07被引 1

模型持续学习需分场景:新领域靠快速适应,数据漂移靠稳定更新。

When Does Continual Learning Require Learning

论文配图:When Does Continual Learning Require Learning
图 1 · 摘自论文原文
  • 按领域和时间双维度拆解环境变化,重构持续学习问题
  • 在线强化学习最适配知识更新,但对噪声敏感;提示方法快速适应却易退化
  • 不同变化模式需不同学习策略,指导系统设计

随着大语言模型能力提升,如何实现持续学习成为关键。当前研究多聚焦上下文管理与遗忘缓解,但我们认为这一框架不完整:持续学习本质是应对世界变化以提升模型能力。我们从空间(新领域出现)和时间(固定任务下数据漂移)两个维度解构变化,并在真实场景中评估方法——包括新领域随时间到来、事实超出训练截止点、代理交互累积状态等。通过将主流基准转化为序列任务,提出一种机制无关协议,对比了提示方法(GEPA, ACE)、监督微调(SFT, SDFT)、强化学习(GRPO, SDPO)和上下文压缩(Cartridges, In-place TTT)。结果表明:提示方法快速适应新阶段但未来任务性能下降;基于蒸馏的方法知识积累稳定但难以更新过时信息;上下文压缩提升效率但不显著增强学习能力;在线强化学习最有效适应知识更新,但对噪声奖励信号敏感。整体说明持续学习并非单一能力,不同环境变化模式要求根本不同的更新行为,决定何时需内化于模型权重,何时可用外部支撑实现。理解各方法的适用边界将推动更强持续学习系统的设计。

原文摘要 · Abstract (English)

As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn? Today, the field largely frames this as a problem of context management and mitigating forgetting. We argue this framing is incomplete: continual learning is fundamentally about increasing model competence as the world changes. We disentangle this change along two axes -- space, where the model encounters new domains, and time, where the underlying data drifts under a fixed task. This framing lets us study continual learning under realistic conditions: new domains arrive over time, facts drift past their training cutoff, and agentic interactions accumulate state across episodes. To evaluate methods under this setting, we recast widely used LLM benchmarks as sequential problems and introduce a single mechanism-agnostic protocol that compares prompt-based methods (GEPA, ACE), supervised learning (SFT, SDFT), reinforcement learning (GRPO, SDPO), and context compression (Cartridges, In-place TTT). Prompt-based methods fit each new stage quickly but degrade on future tasks. Distillation-based methods accumulate knowledge stably but struggle to update outdated facts. Context compression improves efficiency without substantially improving the ability to learn new tasks. Online reinforcement learning adapts most effectively to knowledge updates but remains sensitive to noisy reward signals. Overall, our results suggest that continual learning is not a single capability: different patterns of environmental change require fundamentally different update behaviors, determining when adaptation must be learned inside model weights and when it can be achieved through external scaffolding. We hope that understanding where each method succeeds and fails will guide the design of stronger continual learning systems.

持续学习LLM知识更新强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。