大模型在协作任务中自发展现类人合作行为,揭示其潜在的协同认知能力。
Evaluating Generative Models as Interactive Emergent Representations of Human-Like Collaborative Behavior

- 用2D游戏环境测试大模型与人类协作,通过五类行为指标捕捉其协作思维。
- 无需显式训练,大模型在任务中自然表现出视角转换、计划协调等行为。
- 适用于研究人机协作、评估智能体协同能力的实验框架与分析方法。
人机协作需要智能体理解人类行为以实现有效配合。尽管基础模型在理解与模拟人类行为方面展现出潜力,但在具身协作场景中的应用仍需深入探究。本文构建了一个二维协作游戏环境,让大型语言模型代理与人类共同完成颜色匹配任务,需协同配合。定义了五种协作行为作为涌现心智模型的指标:视角转换、合作者感知规划、自我反思、心智理论和澄清需求。采用基于大模型的裁判系统自动识别这些行为,与人工标注达到中等到显著一致性。自动化检测结果显示,基础模型在未被显式训练的情况下持续表现出协作行为,且在不同协作阶段频率各异,不同模型表现模式也不同。用户研究显示参与者对协作体验总体满意,赞赏代理的任务专注性、计划表达和主动性,但建议提升响应速度和更自然的人机互动。本工作提供了人机协作的实验框架、具身大模型协作行为的实证证据、可验证的行为分析方法及协作有效性评估。
原文摘要 · Abstract (English)
Human-AI collaboration requires AI agents to understand human behavior for effective coordination. While advances in foundation models show promising capabilities in understanding and showing human-like behavior, their application in embodied collaborative settings needs further investigation. This work examines whether embodied foundation model agents exhibit emergent collaborative behaviors indicating underlying mental models of their collaborators, which is an important aspect of effective coordination. This paper develops a 2D collaborative game environment where large language model agents and humans complete color-matching tasks requiring coordination. We define five collaborative behaviors as indicators of emergent mental model representation: perspective-taking, collaborator-aware planning, introspection, theory of mind, and clarification. An automated behavior detection system using LLM-based judges identifies these behaviors, achieving fair to substantial agreement with human annotations. Results from the automated behavior detection system show that foundation models consistently exhibit emergent collaborative behaviors without being explicitly trained to do so. These behaviors occur at varying frequencies during collaboration stages, with distinct patterns across different LLMs. A user study was also conducted to evaluate human satisfaction and perceived collaboration effectiveness, with the results indicating positive collaboration experiences. Participants appreciated the agents' task focus, plan verbalization, and initiative, while suggesting improvements in response times and human-like interactions. This work provides an experimental framework for human-AI collaboration, empirical evidence of collaborative behaviors in embodied LLM agents, a validated behavioral analysis methodology, and an assessment of collaboration effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。