让部署中的智能体通过用户反馈持续学习，无需修改核心模型。
Learning on the Job: Continual Learning from Deployment Feedback for Frozen-Weights Agents
- 用外部记忆存储每轮交互的自然语言规则，实现持续学习。
- 仅凭成功/失败反馈，任务成功率提升至基线1.6倍，纠错反馈达2.6倍。
- 适用于需数据本地化的开源模型，且可跨模型共享学习成果。
AI智能体在每次运行中都面临学习机会，但部署后模型冻结，导致相同问题再次出现时仍从零开始。然而日常运行已产生反馈信号：结果判断和事后修正。本文表明，当冻结模型搭配外部记忆系统，将每轮交互提炼为可检索的自然语言规则时，这些反馈足以支持持续学习。在τ-bench银行领域测试中，仅使用一比特结果反馈，单次尝试成功率提升至基线1.6倍；结合修正信息则达2.6倍，使原本基线无法解决的84个任务中有22个被攻克。该方法在Mistral Large（可自托管的开源模型）与Claude Sonnet 5等前沿模型上均验证有效，且积累的记忆可跨模型迁移——任一模型读取另一模型的存储，均优于自身无记忆基线。相关框架、协议与数据已公开。
原文摘要 · Abstract (English)
AI agents encounter learning opportunities in every episode they run, and discard nearly all of them: the underlying models are frozen at deployment, so an agent that resolves a difficult request today starts from zero when it recurs tomorrow. Yet ordinary operation already produces feedback, in the form of outcome verdicts and after-the-fact corrections. We show that this feedback is a sufficient signal for continual learning when the frozen model is paired with an external memory that distils each episode into retrievable natural-language rules. On the banking domain of $τ$-bench, against a static-RAG control retrieving over the complete policy corpus, learning from the one-bit outcome verdict lifts single-trial success to 1.6$\times$ the baseline, and learning from corrections to 2.6$\times$, converting 22 of the 84 tasks the baseline never solves. The result spans the deployment spectrum, measured on Mistral Large, an open-weights model that organisations with data sovereignty requirements can self-host, and replicated on a frontier model, Claude Sonnet 5. The accumulated memory also transfers: each model, reading the store built by the other, rises above its own no-memory baseline. The harness, protocol, and data are released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。