让智能体在对手不断变化时,有依据地选择技能更新。
Evolving in the Agent Jungle via History-Informed Opponent Awareness

- 用历史对手策略快照做对照实验,判断新技能是否真有效。
- 在两种博弈场景中,比基线更接近最优均衡,且更新次数少40%以上。
- 适合需要长期稳定博弈的AI代理,如拍卖、市场竞争场景。
通过交互学习适应策略是实现更通用、自主的大语言模型智能体的关键步骤。现有方法通常通过修改技能库实现行为适应,但在多智能体环境中,对手可能同时更新策略,导致环境持续演化。在静态环境下设计的技能修订方法在此类场景中相当于基于过时参考进行更新。为此,我们提出OASE(对手感知的择优演化),在动态多智能体环境中识别并采纳真正有益的技能更新。具体而言,OASE在历史对手策略快照所锚定的相同条件下,对候选技能与当前技能进行成对比较,仅当候选技能估计带来的收益增益超过接受阈值时才采纳。我们在两种决策场景中评估OASE:第一价格拍卖和私有成本古诺竞争。实验结果表明,相比基于反思(Reflexion)的基线方法,OASE在两个环境中均实现了更低的最终均衡距离,同时显著减少技能更新次数,抑制了缺乏充分收益支撑的策略变更。因此,OASE以证据为锚点的筛选机制取代盲目更新,使智能体即使在对手持续演化的情况下也能稳定、高效地适应。
原文摘要 · Abstract (English)
Learning to adapt strategies through interaction is a key step toward more general and autonomous LLM agents. Existing approaches typically achieve behavioral adaptation by revising skill libraries. However, in multi-agent environments, opponents may simultaneously update their strategies, causing the environment itself to evolve continuously. Applying skill-revision methods designed for static environments in such settings therefore amounts to updating against an obsolete reference. To address this challenge, we introduce OASE (Opponent-Aware Selective Evolution), which identifies and adopts genuinely beneficial skill revisions in dynamic multi-agent environments. Specifically, OASE conducts paired comparisons between a candidate skill and the incumbent under identical conditions anchored by historical snapshots of opponent strategies, and adopts the candidate only when its estimated payoff gain exceeds an acceptance threshold. We evaluate OASE in two decision-making scenarios: first-price auctions and private-cost Cournot competition. Experimental results show that, compared with a Reflexion-style baseline, OASE achieves a lower final equilibrium distance in both environments while accepting substantially fewer skill revisions, thereby suppressing strategy changes that lack sufficient payoff support. OASE therefore replaces blind updating with evidence-anchored selection, allowing agents to adapt stably and efficiently even as opponents continuously evolve.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。