用大模型自动生成多机器人策略和奖励函数，提升训练效率
LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generation
- 用大模型自动生成先验策略和奖励函数，免去人工设计
- 仿真与实测显示样本效率提升185.9%，任务完成率更高
- 适合想降低人工调参成本的多智能体系统研究者
尽管多智能体强化学习（MARL）在复杂多机器人任务中表现有效，但其存在样本效率低、需反复人工调整奖励函数的问题。大语言模型（LLM）在单机器人场景中已展现潜力，但在多机器人系统中的应用仍不充分。本文提出一种新型的LLM辅助多智能体强化学习（LAMARL）方法，将MARL与LLM结合，显著提升样本效率且无需手动设计。LAMARL包含两个模块：第一模块利用LLM全自动生成先验策略与奖励函数；第二模块为MARL，使用生成的函数有效指导机器人策略训练。在形状组装基准测试中，仿真与真实世界实验均验证了该方法的独特优势。消融实验表明，先验策略使样本效率平均提升185.9%，并提高任务完成率；基于思维链（CoT）与基础API的结构化提示可使LLM输出成功率提升28.5%-67.5%。视频与代码见 https://windylab.github.io/LAMARL/
原文摘要 · Abstract (English)
Although Multi-Agent Reinforcement Learning (MARL) is effective for complex multi-robot tasks, it suffers from low sample efficiency and requires iterative manual reward tuning. Large Language Models (LLMs) have shown promise in single-robot settings, but their application in multi-robot systems remains largely unexplored. This paper introduces a novel LLM-Aided MARL (LAMARL) approach, which integrates MARL with LLMs, significantly enhancing sample efficiency without requiring manual design. LAMARL consists of two modules: the first module leverages LLMs to fully automate the generation of prior policy and reward functions. The second module is MARL, which uses the generated functions to guide robot policy training effectively. On a shape assembly benchmark, both simulation and real-world experiments demonstrate the unique advantages of LAMARL. Ablation studies show that the prior policy improves sample efficiency by an average of 185.9% and enhances task completion, while structured prompts based on Chain-of-Thought (CoT) and basic APIs improve LLM output success rates by 28.5%-67.5%. Videos and code are available at https://windylab.github.io/LAMARL/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。