首个能通用于所有汉牌游戏设置的通用协作智能体
A Generalist Hanabi Agent
- 用文本重构任务，结合分布式强化学习应对动态观测与动作空间
- 可同时适配2至5人游戏，且跨设置策略迁移成功
- 能与无法协作的算法伙伴配合，适合研究通用协作智能体的人
传统多智能体强化学习（MARL）系统虽可通过反复交互发展出合作策略，但仅在训练环境中表现良好，难以适应新场景或与陌生合作者协作。这一问题在汉牌（Hanabi）基准中尤为明显——这是一个需要复杂推理和精准协助的2至5人合作卡牌游戏。现有MARL代理只能学习特定游戏设置（如2人游戏），且仅能与相同算法的代理协作。相比之下，人类可快速调整策略以适应新伙伴或情境。本文提出递归回放相关性分布式DQN（R3D2），一种面向汉牌的通用智能体，通过引入文本表示提升泛化能力，并设计分布式MARL算法以应对动态观测与动作空间。该智能体首次实现所有游戏设置的并行运行，并能将一种设置中学到的策略迁移到其他设置。结果表明，它还能与原本无法协作的算法代理成功合作。代码已开源。
原文摘要 · Abstract (English)
Traditional multi-agent reinforcement learning (MARL) systems can develop cooperative strategies through repeated interactions. However, these systems are unable to perform well on any other setting than the one they have been trained on, and struggle to successfully cooperate with unfamiliar collaborators. This is particularly visible in the Hanabi benchmark, a popular 2-to-5 player cooperative card-game which requires complex reasoning and precise assistance to other agents. Current MARL agents for Hanabi can only learn one specific game-setting (e.g., 2-player games), and play with the same algorithmic agents. This is in stark contrast to humans, who can quickly adjust their strategies to work with unfamiliar partners or situations. In this paper, we introduce Recurrent Replay Relevance Distributed DQN (R3D2), a generalist agent for Hanabi, designed to overcome these limitations. We reformulate the task using text, as language has been shown to improve transfer. We then propose a distributed MARL algorithm that copes with the resulting dynamic observation- and action-space. In doing so, our agent is the first that can play all game settings concurrently, and extend strategies learned from one setting to other ones. As a consequence, our agent also demonstrates the ability to collaborate with different algorithmic agents -- agents that are themselves unable to do so. The implementation code is available at: $\href{https://github.com/chandar-lab/R3D2-A-Generalist-Hanabi-Agent}{R3D2-A-Generalist-Hanabi-Agent}$
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。