arXiv:2602.16928cs.GTcs.AI2026-02被引 3

用大模型自动发现新多智能体强化学习算法,提炼出更简洁高效的版本。

Discovering Multiagent Learning Algorithms with Large Language Models

  • 用大模型驱动的进化框架搜索博弈论算法空间,发现两种新算法。
  • 新算法在18种游戏中表现媲美顶尖人工设计方法,且可泛化到新环境。
  • 通过简化发现核心机制,得到结构更简单但性能更强的最小化求解器。

多智能体强化学习(MARL)在不完美信息博弈中的进展长期依赖人工反复调试算法基线。近期,基于大语言模型(LLM)的演化编码智能体成为自动化发现新算法的强大工具。本文采用名为 AlphaEvolve 的代理框架,在两类博弈论范式——反事实后悔最小化(CFR)与策略空间响应最优(PSRO)——的设计空间中进行探索。该自动化搜索发现了两种新算法:波动率自适应折扣(VAD-)CFR 和平滑混合乐观后悔(SHOR-)PSRO。它们在涵盖德州扑克、戈弗斯尔、说谎骰子、布洛托和战舰变体等18个游戏的评估套件中,持续表现出与最先进人工设计基线相当的性能。然而,由于LLM仅在特定训练集上优化适应性,所生成的机制往往高度协同且复杂,专为这些环境量身定制。通过系统消融实验,我们证明尽管机制紧密耦合,真正推动泛化能力的核心在于一个极简的算法内核。通过将大模型的发现提炼为最基础原理,我们得到了两个最小化求解器:热启动乐观预测(WOP-)CFR 和投影匹配(PM-)PSRO。这些精简版本在泛化性能上更优,且结构复杂度显著降低,为利用大模型进行算法发现提供了清晰的方法论。

原文摘要 · Abstract (English)

Much of the advancement in Multi-Agent Reinforcement Learning (MARL) for imperfect-information games has historically depended on the manual, iterative refinement of algorithmic baselines. Recently, evolutionary coding agents powered by Large Language Models (LLMs) have emerged as powerful tools to automate this discovery process. In this work, we deploy one of such agentic frameworks, AlphaEvolve, to navigate the design spaces of two distinct game-theoretic paradigms: counterfactual regret minimization (CFR) and policy-space response oracles (PSRO). This automated search yielded two algorithms: Volatility-Adaptive Discounted (VAD-) CFR and Smoothed Hybrid Optimistic Regret (SHOR-) PSRO, which are consistently competitive with state-of-the-art human-designed baselines across an 18-game evaluation suite spanning Poker, Goofspiel, Liar's Dice, Blotto, and Battleship variants. However, because the LLM optimizes for fitness on a specific training set, it often constructs highly synergistic, complex mechanisms tailored to those environments. Through systematic ablation studies, we demonstrate that while these mechanisms are tightly coupled, the true driver of generalization lies in a minimal algorithmic core. By distilling the LLM's discoveries down to their most fundamental principles, we produce two minimal solvers: Warm-started Optimistic Predictive (WOP-)CFR and Projection Matching (PM-)PSRO. These distilled versions achieve superior performance on generalization with greatly reduced structural complexity, providing a clear methodology for using LLMs in algorithmic discovery.

多智能体强化学习大模型算法发现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。