用大模型生成可读代码，让多智能体策略变得透明可解释
Code-Space Response Oracles: Generating Interpretable Multi-Agent Policies with Large Language Models
- 把最优响应计算转为代码生成任务，用大模型直接输出人类可读策略
- 在多个博弈场景中达到与传统方法相当的性能，且策略多样可解释
- 适合需要理解智能体行为逻辑的研究者或高风险应用领域
多智能体强化学习中的策略空间响应正则化（PSRO）已在复杂领域中近似求解博弈论均衡。然而，该方法依赖深度强化学习生成的‘黑箱’神经网络策略，难以解释、信任或调试。本文提出代码空间响应正则化（CSRO），用大语言模型（LLM）替代原有强化学习代理。将最优响应计算重构为代码生成任务，通过提示词使大模型直接生成可读代码形式的策略。该方法不仅实现内在可解释性，还利用大模型预训练知识发现复杂且类人的策略。我们探索了零样本提示、迭代优化及一种名为AlphaEvolve的分布式大模型演化系统来构建和增强基于大模型的代理。实验表明，CSRO在性能上媲美基线，同时生成多样化可解释策略。本工作为多智能体学习提供了新视角，从优化不可见的策略参数转向合成可解释的算法行为。
原文摘要 · Abstract (English)
Recent advances in multi-agent reinforcement learning, particularly Policy-Space Response Oracles (PSRO), have enabled the computation of approximate game-theoretic equilibria in increasingly complex domains. However, these methods rely on deep reinforcement learning oracles that produce `black-box' neural network policies, making them difficult to interpret, trust or debug. We introduce Code-Space Response Oracles (CSRO), a novel framework that addresses this challenge by replacing RL oracles with Large Language Models (LLMs). CSRO reframes the best response computation as a code generation task, prompting an LLM to generate policies directly as human-readable code. This approach not only yields inherently interpretable policies but also leverages the LLM's pretrained knowledge to discover complex, human-like strategies. We explore multiple ways to construct and enhance an LLM-based oracle: zero-shot prompting, iterative refinement and \emph{AlphaEvolve}, a distributed LLM-based evolutionary system. We demonstrate that CSRO achieves performance competitive with baselines while producing a diverse set of explainable policies. Our work presents a new perspective on multi-agent learning, shifting the focus from optimizing opaque policy parameters to synthesizing interpretable algorithmic behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。