通过记忆增强优化上下文，让大模型在多轮多人对战中更稳定、更强。
MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Games
- 用记忆库存储自对弈经验，作为后续推理的先验知识
- 结合不确定性选择与优先重播，提升策略探索效率，赢率最高提升24.4个百分点
- 特别适合谈判和信息不完全的游戏，开源代码可复现
多轮、多智能体的大模型对战评估常出现显著的运行间波动。在长周期交互中，早期微小偏差随回合累积，并被多智能体耦合放大，导致胜率估计偏移，使重复比赛中的排名不可靠。提示词选择进一步加剧这一问题，产生不同有效策略。我们提出MEMO（记忆增强模型上下文优化），一种自对弈框架，通过耦合保留与探索来优化推理时的上下文。保留机制维护持久记忆库，存储自对弈轨迹中的结构化洞察，并在后续对弈中作为先验注入；探索机制采用基于TrueSkill的不确定性感知提示进化，结合优先重播机制重新访问稀有且关键的状态。在五个文本类游戏中，MEMO将GPT-4o-mini的平均胜率从25.1%提升至49.5%，Qwen-2.5-7B-Instruct从20.9%提升至44.3%，每任务使用2,000场自对弈游戏。运行间方差显著降低，提示变化下的排名更加稳定。结果表明，通过上下文优化，多智能体大模型对战的表现与鲁棒性仍有巨大提升空间。MEMO在谈判和不完全信息游戏中收益最大，而强化学习在完全信息场景仍更有效。所有代码开源：https://github.com/openverse-ai/MEMO
原文摘要 · Abstract (English)
Multi-turn, multi-agent LLM game evaluations often exhibit substantial run-to-run variance. In long-horizon interactions, small early deviations compound across turns and are amplified by multi-agent coupling. This biases win rate estimates and makes rankings unreliable across repeated tournaments. Prompt choice worsens this further by producing different effective policies. We address both instability and underperformance with MEMO (Memory-augmented MOdel context optimization), a self-play framework that optimizes inference-time context by coupling retention and exploration. Retention maintains a persistent memory bank that stores structured insights from self-play trajectories and injects them as priors during later play. Exploration runs tournament-style prompt evolution with uncertainty-aware selection via TrueSkill, and uses prioritized replay to revisit rare and decisive states. Across five text-based games, MEMO raises mean win rate from 25.1% to 49.5% for GPT-4o-mini and from 20.9% to 44.3% for Qwen-2.5-7B-Instruct, using $2,000$ self-play games per task. Run-to-run variance also drops, giving more stable rankings across prompt variations. These results suggest that multi-agent LLM game performance and robustness have substantial room for improvement through context optimization. MEMO achieves the largest gains in negotiation and imperfect-information games, while RL remains more effective in perfect-information settings. All code is open-source and available here: https://github.com/openverse-ai/MEMO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。