让AI通过反思过往经验,自动生成更贴合当前任务的上下文。
CLEAR: Context Augmentation from Contrastive Learning of Experience via Agentic Reflection
- 用反思代理对比历史执行轨迹,提炼有用上下文
- 在AppWorld上任务完成率从72.62%提升至81.15%
- 适合需要动态生成上下文的复杂任务智能体
大型语言模型智能体依赖有效的上下文获取任务相关资讯以进行决策。现有方法多依赖过往经验生成的上下文与检索机制复用这些内容,但检索到的上下文需由执行智能体适配新情境,增加底层LLM的推理负担。为此,我们提出基于代理反思的对比学习经验上下文增强框架(CLEAR)。CLEAR首先利用反思代理对过往执行轨迹进行对比分析,并总结每项任务的有用上下文;这些总结作为监督微调数据,用于训练上下文增强模型(CAM)。随后通过强化学习进一步优化CAM,奖励信号来自运行任务执行智能体。通过学习生成特定任务知识而非从过往中检索,CAM生成的上下文更契合当前任务。我们在AppWorld和WebShop基准上进行了全面评估。实验结果表明,CLEAR持续优于强基线:在AppWorld测试集上任务完成率从72.62%提升至81.15%,在WebShop子集上平均奖励从0.68提升至0.74。代码已公开于https://github.com/awslabs/CLEAR。
原文摘要 · Abstract (English)
Large language model agents rely on effective model context to obtain task-relevant information for decision-making. Many existing context engineering approaches primarily rely on the context generated from the past experience and retrieval mechanisms that reuse these context. However, retrieved context from past tasks must be adapted by the execution agent to fit new situations, placing additional reasoning burden on the underlying LLM. To address this limitation, we propose a generative context augmentation framework using Contrastive Learning of Experience via Agentic Reflection (CLEAR). CLEAR first employs a reflection agent to perform contrastive analysis over past execution trajectories and summarize useful context for each observed task. These summaries are then used as supervised fine-tuning data to train a context augmentation model (CAM). Then we further optimize CAM using reinforcement learning, where the reward signal is obtained by running the task execution agent. By learning to generate task-specific knowledge rather than retrieve knowledge from the past, CAM produces context that is better tailored to the current task. We conduct comprehensive evaluations on the AppWorld and WebShop benchmarks. Experimental results show that CLEAR consistently outperforms strong baselines. It improves task completion rate from 72.62% to 81.15% on AppWorld test set and averaged reward from 0.68 to 0.74 on a subset of WebShop, compared with baseline agent. Our code is publicly available at https://github.com/awslabs/CLEAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。