用语言协商引导机器人强化学习,让训练更安全高效
MARLIN: Multi-Agent Reinforcement Learning Guided by Language-Based Inter-Robot Negotiation
- 用大模型先做语言协商,生成动作计划指导学习
- 早期训练性能提升,最终效果不降反而更好
- 适合需要安全探索的多机器人系统研究者
多智能体强化学习是训练多机器人系统的关键方法。通过在多轮训练中根据表现奖励或惩罚机器人,使其最终部署于现实环境。然而,训练初期策略不佳可能导致不安全行为。本文提出基于语言交互的多智能体强化学习框架MARLIN,利用大语言模型在强化学习策略尚未有效时提供高层规划。机器人通过语言模型协商行动并生成指导策略学习的计划。系统在训练中动态切换强化学习与语言模型协商模式,实现更安全、高效的探索。在仿真和真实机器人上使用本地与远程语言模型进行评估,结果表明:相比标准多智能体强化学习,该混合方法在早期训练中性能更高,且不影响最终性能。代码已开源:https://github.com/SooratiLab/MARLIN。
原文摘要 · Abstract (English)
Multi-agent reinforcement learning is a key method for training multi-robot systems. Through rewarding or punishing robots over a series of episodes according to their performance, they can be trained and then deployed in the real world. However, poorly trained policies can lead to unsafe behaviour during early training stages. We introduce Multi-Agent Reinforcement Learning guided by language-based Inter-robot Negotiation (MARLIN), a hybrid framework in which large language models provide high-level planning before the reinforcement learning policy has learned effective behaviours. Robots use language models to negotiate actions and generate plans that guide policy learning. The system dynamically switches between reinforcement learning and language-model-based negotiation during training, enabling safer and more effective exploration. MARLIN is evaluated using both simulated and physical robots with local and remote language models. Results show that, compared to standard multi-agent reinforcement learning, the hybrid approach achieves higher performance in early training without reducing final performance. The code is available at https://github.com/SooratiLab/MARLIN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。