PLATO让智能体在任务和智能体数量动态变化时仍能高效协作。
PLATO: Pointer Learner for Agent and Task Openness

- 用指针网络直接生成当前任务集上的动作分布,无需填充或重训练。
- 在野火扑救环境中表现优于现有方法,零样本泛化能力更强。
- 适合需要灵活应对新任务和新智能体的开放多智能体系统研究者。
开放智能体系统(OASYS)在现实场景中日益普遍,其智能体和任务集合会随时间不可预测地变化。这种开放性包括智能体开放(AO)和任务开放(TO),对传统多智能体强化学习(MARL)构成根本挑战,因后者通常假设状态和动作空间固定。现有方法仅部分解决此问题:填充与掩码引入人工边界,而基于图或超图的方法虽处理一维开放性,但仍依赖严格假设。本文提出指针学习器(PLATO),一种基于指针网络的演员,配合中心化图神经网络(GNN)评价值,采用集中训练、分散执行范式,结合多智能体近端策略优化进行训练。其指针式演员直接对当前任务集输出动作分布,无需掩码或重训练即可支持变化的动作空间。其GNN评价值将智能体-任务交互编码为随任务与智能体组成动态变化的图。两者共同实现对AO和TO的无界建模。我们形式化了PLATO在任务与智能体开放马尔可夫博弈(TaAgO-MG)框架下,扩展了先前的任务开放设定,并证明其在由此产生的无界状态与动作空间上定义良好。我们在面向开放智能体系统评估的野火扑救环境(MOASEI)中评估了PLATO,结果表明其性能强于现有最优基线,且零样本泛化更稳定。
原文摘要 · Abstract (English)
Open agent systems (OASYS) are increasingly prevalent in real-world domains where the sets of agents and tasks change unpredictably over time. Such openness, including agent openness (AO) and task openness (TO), poses a fundamental challenge to multi-agent reinforcement learning (MARL), which typically assumes fixed state and action spaces. Existing methods address openness only partially: padding and masking approaches introduce artificial bounds, while recent graph-based or hypergraph methods handle one dimension of openness but still depend on restrictive assumptions. In this paper, we introduce Pointer Learner for Agent and Task Openness (PLATO), a pointer-network-based actor combined with a centralized graph neural network (GNN) critic, trained with multi-agent proximal policy optimization under a centralized training and decentralized execution paradigm. Our pointer-based actor outputs distributions directly over the current task set. This directly supports changing action spaces without masking or retraining. Our GNN critic encodes agent-task interactions as a graph that changes shape with task and agent composition. Together, these components consider AO and TO without the boundedness of existing approaches. We formalize PLATO in a Task-and-Agent-Open Markov Game (TaAgO-MG), extending prior task-open formulations, and prove it is well-defined over the resulting unbounded state and action spaces. We evaluate PLATO with the Methods for Open Agent Systems Evaluation Initiative (MOASEI) wildfire suppression domain, an environment designed for open multi-agent system evaluation, and we demonstrate strong performance and more consistent zero-shot generalization than state-of-the-art baselines in OASYS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。