新环境ColorGrid挑战智能体推断人类隐藏目标的能力
ColorGrid: A Multi-Agent Non-Stationary Environment for Goal Inference and Assistance
- 设计可调非平稳、不对称的多智能体环境
- IPPO算法在复杂目标冲突下表现不佳，任务未解决
- 适合研究人机协作与动态目标推理的学者
自主智能体与人类的交互日益关注适应其变化的偏好，以提升现实任务中的辅助效果。高效智能体需准确推断通常隐藏的人类目标以实现良好协作。然而，现有多智能体强化学习（MARL）环境缺乏评估此类学习能力所需的特性。为此，我们提出ColorGrid，一种可定制非平稳性、不对称性和奖励结构的新型MARL环境。通过大量消融实验，我们发现独立近端策略优化（IPPO）——当前最先进的MARL算法——在领导者（代表人类）与追随者（助手）同时存在非平稳和不对称目标时，无法解决ColorGrid任务。为支持未来MARL算法的基准测试，我们公开发布环境代码、模型检查点及轨迹可视化，地址：https://github.com/andreyrisukhin/ColorGrid。
原文摘要 · Abstract (English)
Autonomous agents' interactions with humans are increasingly focused on adapting to their changing preferences in order to improve assistance in real-world tasks. Effective agents must learn to accurately infer human goals, which are often hidden, to collaborate well. However, existing Multi-Agent Reinforcement Learning (MARL) environments lack the necessary attributes required to rigorously evaluate these agents' learning capabilities. To this end, we introduce ColorGrid, a novel MARL environment with customizable non-stationarity, asymmetry, and reward structure. We investigate the performance of Independent Proximal Policy Optimization (IPPO), a state-of-the-art (SOTA) MARL algorithm, in ColorGrid and find through extensive ablations that, particularly with simultaneous non-stationary and asymmetric goals between a ``leader'' agent representing a human and a ``follower'' assistant agent, ColorGrid is unsolved by IPPO. To support benchmarking future MARL algorithms, we release our environment code, model checkpoints, and trajectory visualizations at https://github.com/andreyrisukhin/ColorGrid.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。