用大模型统筹多智能体策略，实现高效协作与类人行为。
Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution
- 大模型制定战略，强化学习负责具体执行，分层协同决策。
- 在2对2对抗环境中胜率46.4%，接近人工规则系统（51.5%）。
- 用户评价显示其行为更像人类，具备灵活战术与适应性。
强化学习在序列决策中表现强劲，但在复杂多智能体环境中仍面临奖励稀疏、状态动作空间庞大及协调策略难学等挑战。本文提出一种分层架构：预训练大语言模型（LLM）作为中央战略控制器，从多个专用强化学习技能策略中选择合适方案，而各强化学习策略则负责实时反应式执行。在竞争性2v2‘王座之巅’环境中，该混合系统性能与手工设计的行为树（BT）相当（胜率46.4% vs 51.5%，p=0.103），显著优于无技能分解的端到端强化学习（Flat RL）。用户研究（n=15）显示，60%参与者认为LLM+RL智能体最像人类（p=0.027），因其具备行为适应性和战术多样性。结果表明，预训练大模型可有效调度预训练强化学习技能，在无需人工规则工程的情况下实现有竞争力的多智能体协作与更高可信度的类人行为。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has achieved strong performance in sequential decision-making, yet scaling to complex multi-agent environments remains challenging due to sparse rewards, large state-action spaces, and the difficulty of learning coordinated strategies. We propose a hierarchical architecture where a pretrained large language model (LLM) acts as a centralized strategic controller that selects among specialized RL skill policies for a team of agents, while RL policies handle reactive low-level execution. We evaluate this hybrid system in a competitive 2v2 King of the Hill environment against behavior tree (BT) and \emph{``Flat''} RL (end-to-end training without skill decomposition) baselines. The LLM+RL system achieves task performance statistically equivalent to hand-crafted BT (46.4\% vs 51.5\% win rate, $p=0.103$) while both significantly outperform Flat RL trained without skill decomposition. A user study ($n=15$) reveals that 60\% of participants perceive LLM+RL agents as the most human-like ($p=0.027$), citing behavioral adaptability and tactical variability. These results demonstrate that pretrained LLM reasoning can effectively orchestrate pretrained RL skills, achieving competitive multi-agent coordination and superior perceived believability without manual rule engineering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。