arXiv:2510.14401cs.MAcs.AI2025-10中稿 · the 25th Internati…被引 7

用社会学习和集体规范机制,让大模型在无奖励下自发合作。

The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems

  • 移除显式奖励,通过模仿成功同伴和规范惩罚来驱动合作。
  • 在资源丰富/稀缺、利他/利己环境下,不同模型合作能力差异明显。
  • 适合研究AI社会中合作机制,为可信治理提供设计参考。

大量基于大模型的多智能体研究探讨了在混合动机情境下规范与合作如何涌现,其中追求个体利益可能损害集体福祉。以往工作多在情境丰富或简化博弈环境中进行,且多数大模型系统为智能体提供直接关联行动的显式奖励函数。而人类合作常无需了解收益结构或行动长期后果,而是依赖启发式、沟通与执行机制。本文提出一种去除显式奖励信号的公共资源(CPR)模拟框架,融入文化演化机制:社会学习(从成功同伴处采纳策略与信念)与基于奥斯特罗姆资源治理原则的规范性惩罚。智能体还通过环境反馈个体学习采收、监控与惩罚的后果,使规范能内生形成。我们通过复现已有研究中的人类行为关键发现验证了该框架的有效性。在此基础上,分析了在资源丰富/稀缺、利他/自私两种初始条件构成的2×2网格中规范演化情况,并对比不同大模型组成的社会在这些条件下的表现。结果揭示模型间在维持合作与规范形成上的系统性差异,表明该框架可作为研究混合动机下大模型社会中涌现规范的严谨测试平台。此类分析有助于设计部署于社会与组织场景中的对齐协作型人工智能系统,以保障稳定性、公平性与治理有效性。

原文摘要 · Abstract (English)

A growing body of multi-agent studies with LLMs explores how norms and cooperation emerge in mixed-motive scenarios, where pursuing individual gain can undermine the collective good. While prior work has explored these dynamics in both richly contextualized simulations and simplified game-theoretic environments, most LLM systems featuring common-pool resource (CPR) games provide agents with explicit reward functions directly tied to their actions. In contrast, human cooperation often emerges without explicit knowledge of the payoff structure or how individual actions translate into long-run outcomes, relying instead on heuristics, communication, and enforcement. We introduce a CPR simulation framework that removes explicit reward signals and embeds cultural-evolutionary mechanisms: social learning (adopting strategies and beliefs from successful peers) and norm-based punishment, grounded in Ostrom's principles of resource governance. Agents also individually learn from the consequences of harvesting, monitoring, and punishing via environmental feedback, enabling norms to emerge endogenously. We establish the validity of our simulation by reproducing key findings from existing studies on human behavior. Building on this, we examine norm evolution across a $2\times2$ grid of environmental and social initialisations (resource-rich vs. resource-scarce; altruistic vs. selfish) and benchmark how agentic societies comprised of different LLMs perform under these conditions. Our results reveal systematic model differences in sustaining cooperation and norm formation, positioning the framework as a rigorous testbed for studying emergent norms in mixed-motive LLM societies. Such analysis can inform the design of AI systems deployed in social and organizational contexts, where alignment with cooperative norms is critical for stability, fairness, and effective governance of AI-mediated environments.

多智能体合作机制大模型社会学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。