让智能体自动优化记忆使用策略，提升长期表现。
AEL: Evolving Agent Harness in Open-Ended Environments
- 快慢双时标机制：快速选策略，慢速反思改进。
- 在投资组合任务上提升夏普比率27%，方差最低。
- 适合需要持续学习的开放环境应用，如客服路由。
传统大模型智能体的调用框架由人工设计且固定不变，导致智能体虽积累经验却无法学会如何有效利用记忆——例如何时检索、何时判断信息误导、何时调整策略。本文提出智能体演化学习（AEL），采用双时标框架：快速的汤普森采样老虎机每轮选择记忆检索策略；慢速的大模型反思则遵循‘先诊断后处方’原则，解释性能下降原因，并在策略停滞时注入新检索策略作为新选项。AEL在序列投资组合基准上超越10个自改进及非大模型基线，使夏普比率提升27%，且所有随机方法中波动最小；在客服工单路由任务中，准确率较无反思的汤普森采样提升18%，较最优前序基线提升51%。机制分析表明收益具有因果性：仅在需切换策略的模式下反思有效，策略稳定时无损无益。代码与数据见：https://github.com/WujiangXu/AEL。
原文摘要 · Abstract (English)
LLM Agents Harnesses are hand-designed and stay fixed, so agents accumulate experience but never learn how to use it: which memories to retrieve, when retrieved evidence is misleading, and when the retrieval strategy itself should change. We introduce Agent Evolving Learning (AEL), a two-timescale framework that evolves the harness, recasting memory use as online policy selection. A fast Thompson Sampling bandit selects among memory-retrieval policies episode by episode, while slow LLM reflection follows a diagnose-before-prescribe principle: it first explains why performance degraded, then injects a targeted new retrieval policy as a bandit arm when the current pool plateaus. AEL outperforms ten self-improving and nonLLM baselines on a sequential portfolio benchmark, lifting Sharpe by 27% over the strongest memory-only variant with the lowest variance among all stochastic methods, and generalizes to a support-ticket routing stream, where it improves accuracy by 18% over reflectionfree Thompson Sampling and by 51% over the best prior baseline. Mechanism studies further show that the gains are causal: reflection helps precisely when regimes demand different retrieval behavior, and is provably no-harm/nogain when the best policy is stable. Code and data: https://github.com/WujiangXu/AEL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。