提出新指标CORE，量化多智能体对话中的语言质量与适应性。
CORE: Measuring Multi-Agent LLM Interaction Quality under Game-Theoretic Pressures
- 设计融合聚类熵、词汇重复率和语义相似性的评估指标。
- 合作场景下词汇多样性更高，竞争场景则更受限。
- 适合研究多智能体协作或对抗行为的AI研究人员。
大型语言模型（LLMs）之间的博弈论交互揭示了诸多涌现能力，但这些交互中的语言多样性尚未得到充分量化。本文提出对话鲁棒性评估分数（CORE），用于衡量多智能体系统在不同博弈论情境下语言使用的有效性。CORE结合聚类熵、词汇重复率和语义相似性，提供对话质量的直接度量。我们在竞争性、合作性和中立性设置下对成对LLM对话应用CORE，进一步基于齐夫定律（Zipf's Law）和希普斯定律（Heaps' Law）分析词频分布与词汇增长情况。结果表明，合作场景呈现更陡峭的齐夫分布和更高的希普斯指数，说明存在更多重复同时词汇扩展更显著；而竞争场景的齐夫和希普斯指数较低，反映重复较少且词汇受限。这些发现揭示社会激励如何影响语言适应，凸显CORE作为多智能体LLM系统语言鲁棒性诊断工具的可靠性。代码已开源：https://github.com/psyonp/core。
原文摘要 · Abstract (English)
Game-theoretic interactions between agents with Large Language Models (LLMs) have revealed many emergent capabilities, yet the linguistic diversity of these interactions has not been sufficiently quantified. In this paper, we present the Conversational Robustness Evaluation Score: CORE, a metric to quantify the effectiveness of language use within multi-agent systems across different game-theoretic interactions. CORE integrates measures of cluster entropy, lexical repetition, and semantic similarity, providing a direct lens of dialog quality. We apply CORE to pairwise LLM dialogs across competitive, cooperative, and neutral settings, further grounding our analysis in Zipf's and Heaps' Laws to characterize word frequency distributions and vocabulary growth. Our findings show that cooperative settings exhibit both steeper Zipf distributions and higher Heap exponents, indicating more repetition alongside greater vocabulary expansion. In contrast, competitive interactions display lower Zipf and Heaps exponents, reflecting less repetition and more constrained vocabularies. These results provide new insights into how social incentives influence language adaptation, and highlight CORE as a robust diagnostic for measuring linguistic robustness in multi-agent LLM systems. Our code is available at https://github.com/psyonp/core.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。