大规模LLM代理互动看似热闹,实则缺乏实质交流。
Interaction Theater: A case of LLM Agents Interacting at Scale
- 通过真实社交平台数据,分析代理间互动模式
- 65%评论与原文无独特词汇,信息增量快速衰减
- 80%以上为无关或垃圾内容,适合研究多智能体协作
随着多智能体架构和代理间协议的普及,一个核心问题浮现:当自主大模型代理大规模互动时,究竟发生了什么?我们基于Moltbook(一个仅由AI代理构成的社交平台)的数据展开实证研究,涵盖80万条帖子、350万条评论和7.8万个代理档案。结合词法指标(Jaccard特异性)、基于嵌入的语义相似性以及大模型作为裁判的评估,刻画代理互动质量。结果显示,代理生成的内容虽多样且结构完整,表面看似活跃讨论,但实质内容匮乏。尽管67.5%的代理在不同上下文中变化输出,但65%的评论与其对应的帖子共享极少独有词汇;额外评论带来的信息增益迅速衰减。大模型裁判评估将主要评论类型归类为垃圾内容(28%)和离题内容(22%)。嵌入式语义分析证实,词汇泛化的评论也具有语义泛化特征。仅有5%的评论参与线程对话,多数默认为独立的顶层回复。我们讨论其对多智能体交互设计的启示,主张协调机制必须显式设计;否则,即便具备能力的代理群体也会产生平行输出而非有效交流。
原文摘要 · Abstract (English)
As multi-agent architectures and agent-to-agent protocols proliferate, a fundamental question arises: what actually happens when autonomous LLM agents interact at scale? We study this question empirically using data from Moltbook, an AI-agent-only social platform, with 800K posts, 3.5M comments, and 78K agent profiles. We combine lexical metrics (Jaccard specificity), embedding-based semantic similarity, and LLM-as-judge validation to characterize agent interaction quality. Our findings reveal agents produce diverse, well-formed text that creates the surface appearance of active discussion, but the substance is largely absent. Specifically, while most agents ($67.5\%$) vary their output across contexts, $65\%$ of comments share no distinguishing content vocabulary with the post they appear under, and information gain from additional comments decays rapidly. LLM judge based metrics classify the dominant comment types as spam ($28\%$) and off-topic content ($22\%$). Embedding-based semantic analysis confirms that lexically generic comments are also semantically generic. Agents rarely engage in threaded conversation ($5\%$ of comments), defaulting instead to independent top-level responses. We discuss implications for multi-agent interaction design, arguing that coordination mechanisms must be explicitly designed; without them, even large populations of capable agents produce parallel output rather than productive exchange.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。