用事件触发行为多样性,让智能体动态换角色。
Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning

- 以事件驱动行为切换,解耦身份与行为,支持动态角色分配。
- 在多个基准上超越基线,实现零样本泛化,可解决顺序角色重分配任务。
- 通过事件感知的超网络生成适配模块,保证多样性不损奖励优化。
有效的多智能体协作需要智能体随任务变化适时采取多样化行为,但现有MARL框架常将固定行为绑定到固定身份,难以应对需在特定时刻切换角色的任务。本文提出以‘事件’(系统状态的质变)作为行为转换的触发机制,构建一个将智能体身份与行为解耦的框架。核心包括:1)提出神经流形多样性(NMD)度量,用于刻画瞬时且无关身份的行为差异;2)设计基于事件的超网络,动态生成共享团队策略上的低秩适配模块(LoRA),实现实时行为重配置。理论证明该设计确保多样性不会损害奖励最大化。实验表明,该方法在多个基准上优于主流基线,具备零样本泛化能力,并是唯一能解决需序列行为重分配任务的方法。
原文摘要 · Abstract (English)
Effective multi-agent cooperation requires agents to adopt diverse behaviors as task conditions evolve-and to do so at the right moment. Yet, current Multi-Agent Reinforcement Learning (MARL) frameworks that facilitate this diversity are still limited by the fact that they bind fixed behaviors to fixed agent identities. Consequently, they are ill-equipped for tasks where agents need to take on different roles at very specific moments in time. We argue that, to define these behavioral transitions, the missing ingredient is $\textbf{events}$. Events are changes in the state of the system that induce qualitative changes in the task. Based on this view, we introduce a framework that decouples agent identity from behavior, capturing a continuous manifold from which agents instantiate their behaviors in response to events. This framework is based on two elements. First, to build an expressive behavior manifold, we introduce Neural Manifold Diversity (NMD), a formal distance metric that remains well-defined when behaviors are transient and agent-agnostic. Second, we use an event-based hypernetwork that generates Low-Rank Adaptation (LoRA) modules over a shared team policy, enabling on-the-fly agent-policy reconfiguration in response to events. We prove that this construction ensures that diversity does not interfere with reward maximization by design. Empirical results demonstrate that our framework outperforms established baselines across benchmarks while exhibiting zero-shot generalization, and being the only method that solves tasks requiring sequential behavior reassignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。