让智能体从自身经验中高效学习，不花额外环境成本。
Sample-Efficient Learning from Agent Experience
- 将交互经验提炼为模型参数，实现无额外交互的持续学习。
- 在软件任务和文字游戏上保留64.8%的上下文学习效果。
- 相比传统强化学习，节省9.6倍以上环境采样次数。
真实世界中的智能体学习常受限于昂贵的环境交互，如耗时实验或人工反馈。上下文学习虽能高效利用历史经验，但一旦上下文清除，收益即消失。上下文蒸馏可将上下文信息内化至模型权重，但将其应用于智能体交互历史且不牺牲样本效率仍待探索。本文提出经验蒸馏（Experience Distillation）方法，仅需已有交互数据，无需额外环境交互。在749个精选软件工程任务与6个文本冒险游戏上的实验表明，该方法在两个领域均保留至少64.8%的上下文学习增益，而直接监督微调仅恢复3.8%。相较于经典强化学习基线，基于试错经验的上下文学习结合经验蒸馏，性能相当，但环境样本减少至少9.6倍。
原文摘要 · Abstract (English)
Real-world agent learning is often constrained by costly environment interactions, such as running time-consuming experiments or obtaining human feedback. In-context learning offers a highly sample-efficient way for agents to learn from their own interaction histories, but its gains disappear once that experience is removed from the context. Separately, context distillation provides a mechanism for internalizing contextual information into model weights. However, applying it to agents' interaction histories without sacrificing environment sample efficiency remains underexplored. We term this problem Experience Distillation and develop an implementation that requires no further environment interaction beyond the collected experience. Experiments on 749 curated software-engineering tasks and six text-adventure games show that it retains at least 64.8\% of the gains from in-context learning across both domains, whereas direct supervised fine-tuning on the collected experience recovers only 3.8\%. Compared with classical reinforcement-learning baselines, in-context learning from trial-and-error experience followed by Experience Distillation matches their performance with at least \(9.6\times\) fewer environment samples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。