分析先进AI多智能体系统的三大风险及成因，提出安全治理新框架。
Multi-Agent Risks from Advanced AI
- 基于激励机制识别出错配、冲突、串通三种失效模式。
- 提炼七类风险因素，如信息不对称与涌现自主性，揭示系统复杂性根源。
- 结合真实案例与实验，为安全治理提供可操作方向，适合政策制定者参考。
先进AI智能体的快速发展与大规模部署将催生前所未有的复杂多智能体系统，带来新型且未充分研究的风险。本报告通过分析智能体的激励机制，构建了三类关键失效模式（错配、冲突、串通）的系统性分类框架，并识别出七项核心风险因素：信息不对称、网络效应、选择压力、失稳动态、承诺难题、涌现自主性以及多智能体安全问题。报告列举了各类风险的具体实例，并探讨潜在缓解路径。通过锚定真实世界案例与实验证据，阐明多智能体系统带来的独特挑战，及其对高级AI的安全性、治理与伦理的深远影响。
原文摘要 · Abstract (English)
The rapid development of advanced AI agents and the imminent deployment of many instances of these agents will give rise to multi-agent systems of unprecedented complexity. These systems pose novel and under-explored risks. In this report, we provide a structured taxonomy of these risks by identifying three key failure modes (miscoordination, conflict, and collusion) based on agents' incentives, as well as seven key risk factors (information asymmetries, network effects, selection pressures, destabilising dynamics, commitment problems, emergent agency, and multi-agent security) that can underpin them. We highlight several important instances of each risk, as well as promising directions to help mitigate them. By anchoring our analysis in a range of real-world examples and experimental evidence, we illustrate the distinct challenges posed by multi-agent systems and their implications for the safety, governance, and ethics of advanced AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。