多智能体系统在合作任务中自发形成依赖层级，由能力与努力动态决定。
Emergence of Hierarchies in Multi-Agent Self-Organizing Systems Pursuing a Joint Objective
- 通过计算智能体间动作梯度量化依赖关系，分析层级演化。
- 层级随任务需求动态变化，不依赖预设规则。
- 适合研究自组织系统、协作智能体的学者与工程师。
多智能体自组织系统（MASOS）具有可扩展性、适应性、灵活性和鲁棒性等关键特性，已广泛应用于多个领域。然而，其自组织特性也带来了行为不可预测性。本文聚焦于任务执行过程中依赖层级的涌现，旨在理解这些层级如何从智能体共同追求目标中产生、如何动态演变，以及受何种因素驱动。采用多智能体强化学习（MARL）训练系统完成协作推箱任务，通过计算每个智能体动作对其他智能体状态的梯度，量化智能体间依赖关系，并通过聚合依赖关系分析层级结构的形成。结果表明，层级在智能体协同推进共同目标时动态涌现，且随任务需求不断演化。值得注意的是，这些依赖层级是响应共享目标而有机生成的，而非预设规则或可调参数的结果。此外，层级的形成受任务环境和网络初始化条件影响。层级的涌现源于智能体在环境中的‘能力’与‘努力’之间的动态互动：‘能力’决定初始影响力，而持续的‘努力’使智能体能够调整自身角色与地位。
原文摘要 · Abstract (English)
Multi-agent self-organizing systems (MASOS) exhibit key characteristics including scalability, adaptability, flexibility, and robustness, which have contributed to their extensive application across various fields. However, the self-organizing nature of MASOS also introduces elements of unpredictability in their emergent behaviors. This paper focuses on the emergence of dependency hierarchies during task execution, aiming to understand how such hierarchies arise from agents' collective pursuit of the joint objective, how they evolve dynamically, and what factors govern their development. To investigate this phenomenon, multi-agent reinforcement learning (MARL) is employed to train MASOS for a collaborative box-pushing task. By calculating the gradients of each agent's actions in relation to the states of other agents, the inter-agent dependencies are quantified, and the emergence of hierarchies is analyzed through the aggregation of these dependencies. Our results demonstrate that hierarchies emerge dynamically as agents work towards a joint objective, with these hierarchies evolving in response to changing task requirements. Notably, these dependency hierarchies emerge organically in response to the shared objective, rather than being a consequence of pre-configured rules or parameters that can be fine-tuned to achieve specific results. Furthermore, the emergence of hierarchies is influenced by the task environment and network initialization conditions. Additionally, hierarchies in MASOS emerge from the dynamic interplay between agents' "Talent" and "Effort" within the "Environment." "Talent" determines an agent's initial influence on collective decision-making, while continuous "Effort" within the "Environment" enables agents to shift their roles and positions within the system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。