用即兴文字游戏测试AI社交智能，看它能否理解他人认知状态。
Improvisational Games as a Benchmark for Social Intelligence of AI Agents: The Case of Connections
- 设计即兴文字游戏Connections，融合知识检索与认知推断。
- 模型需在有限沟通中判断他人理解程度，体现社会智能。
- 适合研究具身智能、多智能体协作的学者参考。
我们正式引入一种即兴文字游戏Connections，用于探索基于语言模型的AI代理的推理能力。该游戏要求玩家结合知识检索、摘要生成以及对其他智能体认知状态的意识。研究表明，该游戏能有效评估语言模型代理超越自身记忆和演绎推理的社交智能，尤其体现在对其他代理理解能力的判断上。此外，在受限沟通环境中，通过与其他智能体互动，AI代理必须展现社交意识与智能，以实现协作目标。
原文摘要 · Abstract (English)
We formally introduce a improvisational wordplay game called Connections to explore reasoning capabilities of AI agents. Playing Connections combines skills in knowledge retrieval, summarization and awareness of cognitive states of other agents. We show how the game serves as a good benchmark for social intelligence abilities of language model based agents that go beyond the agents' own memory and deductive reasoning and also involve gauging the understanding capabilities of other agents. Finally, we show how through communication with other agents in a constrained environment, AI agents must demonstrate social awareness and intelligence in games involving collaboration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。